As a baseball fan, I follow hundreds of baseball personalities on Twitter. One of the hottest debates between the old guard and the new is whether professional baseball teams should rely on the intuition of coaches and scouts (“Gut”) or the analysis of geeks crunching statistics (“Data”). It can sometimes be phrased as “experience” versus “analytics.”

The Data movement was popularized by Michael Lewis’ Moneyball, which framed the Oakland A’s as hapless and money-less, until underdog general manager Billy Beane and his analyst, Paul DePodesta used historical game data to determine which statistical outcomes within the game of baseball most closely correlated to wins. The A’s built an organization that consistently contended by finding players that the rest of the league did not want, and were therefore cheap, but had a propensity to generate these secret outcomes. Once the book became a bestseller, every other team caught on, and teams with more money, like the Boston Red Sox, combined a large wallet with analytics to best the A’s.

Fast forward to 2018, when the Houston Astros won an improbable world series after years of disastrous performance. Sports Illustrated’s Ben Reiter wrote a book called Astroball which depicts the Astros strategy of not only identifying predictive statistics, but of finding measurable, physical attributes of players that correlated with those statistics. For example, they put defense equipment meant for detecting projectiles in all of their minor league ballparks so they could measure the angle and velocity of balls off a hitter’s bat, the spin rate of a ball leaving the pitcher’s hand, and the range of ground covered by a fielder. You can imagine the myriad measurable attributes they capture based on every wrinkle in the game.

Last month Ben Cohen of the Wall Street Journal wrote an article called Computers Are the New Basketball Coaches. The title is not as tongue-in-cheek as it sounds. He highlights an NBA player named Admiral Schofield. Admiral played collegiate ball at University of Tennessee, where he was a subpar 3-point shooter, making only 30% as a freshman. A camera based shot tracking tool, called Noah, used in the school’s practice facility showed that the angle at which the ball approached the rim was suboptimal. The diagnosis: Admiral needed more arc on his shot, but not too much. He made the tweak and his 3-pointers started going in at an exceptional rate of 41%.

All of that said, is there still value in intuition? Some scouts in both games have watched thousands of games. They saw the old pros do things that worked. Driveline Baseball, a training club dedicated to developing baseball prospects into stars, utilizes data very heavily. They regularly insist on the Driveline Podcast, that a scout’s gut is data. It is information that should be used, just like the data points used by the analysts in the front office. Reiter made the same point in Astroball. So while spin rate is predictive of a pitcher’s success, developing the next Game 7 starter of the World Series is not simply finding the pitcher with the best spin rate.

Potential Problems with too Much Data Focus

In western psychology, the prevailing view until the 1970s was that human judgment was basically rational and logically consistent. Amos Tversky and Daniel Kahneman (“T&K”) tore asunder that premise. T&K showed that humans, in order to judge situations perfectly, would have to do algorithm calculations. We don’t. Instead, our minds take shortcuts. In some cases, our minds reach quickly for the data that is most readily available to us, a phenomenon called Availability Heuristic.

For example, perhaps the history we study is not based on the significance of historical events, but rather the availability of information about certain historical societies or personages. I have begun a long journey through Arnold Toynbee’s 12 volume A Study in History. Toynbee begins the series with a critique of modern Western thought and its influence on history. He argues that the Industrial Revolution, while no doubt a wonderful step of progress for society, is responsible for a ruthless emphasis on efficiency.

“Its method of operation is to maintain, up to the maximum of its productive capacity, an incessant output of such articles as can be manufactured from raw materials by the mechanically coordinated work of a number of human beings.”

Dr. Toynbee wrote this in the 1930s, but before we write off his anachronistic view of industry, he gives a compelling practical example in his field. The death of Alexander the Great led to the break-up of the Achaemenid Empire. In modern Egypt, the Ptolemaic Dynasty rose to power and build a great society, but its impact on the world today is minimal. Elsewhere in Asia, however, the Seleucid Monarchy emerged and built a society whose echoes we still hear today. Within the Seleucid Empire were Hellenic and Syriac civilizations who eventually became the Roman Empire, and the breeding ground for Christianity and Islam.

At the time, much was written about the Ptolemaic Dynasty because of the availability of archaeological evidence and manuscripts. Sadly, not much of the Seleucid Monarchy has been preserved, meaning very little can be studied.

“Owing, however, to a climatic accident, the amount of raw information regarding these two monarchies which happens to be accessible to us is in inverse ratio to their intrinsic importance in history.”

He goes on to note that Gibbon’s famous The History of the Decline and Fall of the Roman Empire, is an example of great industrial feat. The amount of data processed as raw material into articles of finished goods as multi-volume history books is amazing. Its sheer size is perhaps commendable. But is it useful? Have we, as a society, placed too much emphasis on scale and efficiency?

In fields where knowledge is the raw material, do we place too much emphasis on the availability of data?

The Jury is Out

There is no use debating whether data is more useful than gut. Gut is data. The real debate should be over whether the data being used is actually meaningful. In e-commerce, it is difficult to name a single digitally native brand that has been acquired for $1 billion. In pitch decks, however, they tout LTV to CAC numbers that are impressive, but the long-term economic model rarely works out.

In consumer private equity, there is a firm called CircleUp that has built a machine learning tool that can supposedly capture predictive data on brands. I really respect the team at CircleUp, but the jury is out as to whether their quant-heavy approach is actually capable of capturing data that is truly predictive of great outcomes.

The tech industry is filled with unicorns (private companies with $1 billion valuations) setting the financial world on fire with $100 million of annual revenue and cash burn equal to 1.0x to 1.5x their annual revenue rate.

There is no debate: information is better than intuition when it comes to business. It takes only a modicum of experience to learn quickly that betting on your own intellect too often leads to eventual mistakes. Pride comes before the fall. That said, there can be a pride of intellect that unduly applies skepticism to intuition when its availability is just as good as data. We should learn to be aware of that, lest our life’s work become like dissertations on the Ptolemaic Dynasty.