I met Billy Beane at a conference and I asked him if he had any regrets. After all, his secret was out. Beane is the baseball executive whose story was the subject of the bestselling book "Moneyball" and who was portrayed by Brad Pitt in the film of the same name. He is now the most famous executive in sports with a second career as a conference speaker for being the cultural icon on how to use advanced analytics to drive success.
Beane, with the help of Paul DePodesta, quietly put analyst/guru Bill James’ statistical work on baseball into action with amazing results. But the sensational success of the book and the movie made Beane’s covert strategy common knowledge and every major league baseball team followed his lead hiring highly skilled analytic experts in a battle for the analytical high ground.
Beane (who by the way is a great guy) smiled and said that after the success of the book and movie, all the major league teams did indeed hire high-priced Ivy League educated statistical people in an effort to take Bill James’ analytics, called Sabermetrics, to the next level. But he added that when push came to shove, many of the teams in the league did not use those analytical recommendations and relied on old metrics and established conventional wisdom when making decisions.
Beane said he had no regrets. He had the spoils from the success of the book and the movie and still retained an analytical competitive edge in baseball because many of the teams resisted the use of advanced analytics in their actual day-to-day operations.
The response from this analytic icon got me thinking on what is the real adoption rate of analytic investments in our industry. Big data and analytics are mentioned in almost every food industry article on how to gain a competitive advantage or optimize business performance, but how many people really believe that?
Part of the resistance to analytics is caused by overconfidence in our own abilities and our understanding of our world. As individuals, we rate ourselves as being better than average and often in the top percentile in a whole host of areas including driving skills, sense of humor, physical appearance and judgment. In business, we are all self-ascribed experts in pricing, marketing, packaging design and advertising. Having good instincts is important, but relying solely on gut to provide reliable guidance on very complex issues that have an enormous business impact is risky behavior.
This overconfidence carries over into numbers. Our historical hunter/gatherer brain is often ill-equipped to manage the tsunami of numbers in our complex business world. The ability of Moore’s Law to deliver facts and figures at an exponential amount and frequency can be more overwhelming than we would like to admit.
Making sense of this inundation of data and teasing out the signal from the noise requires more dedicated time and focus than our ever-shortening attention spans are capable of handling. It also requires an acknowledgment of the organizational limitations and cultural barriers that may exist which prevent a company from fully leveraging the findings from advanced analytics.
It is difficult to step outside of ourselves and weigh the impact of our internal biases on our conclusions. It feels as if we come to decisions and opinions based on the deliberate review of a wide array of facts, but studies show that this happens in the opposite order. We actually derive opinions and then look for information that supports those opinions.
To compound the problem we will often disregard credible information that runs counter to our established beliefs. In baseball, there are vocal detractors of Billy Beane’s use of analytics who point to Oakland’s superior pitching in the one season outlined in "Moneyball." In doing so, these critics disregard an exhaustive statistical analysis that examined every play, run and out in every game over decades.
What often gets lost in the buzz around big data, analytics and slick software is that the real value in these advancements is in adopting the tools to course-correct strategy and tactics. If different action is not taken on the findings from analytics it is a significant waste of time, money and energy. Competitors that do take action can gain a sustained advantage.
A good but difficult question to ask before initiating any data or analytic project is: Does our company have the culture and discipline to absorb and act on findings from these projects, even if the findings challenge conventional wisdom or go against our most strongly held strategies and processes? For many companies and baseball teams, the truthful answer is, unfortunately, not yet.