Blogging the Bookshelf

Blogging my bookshelf – one book at a time

Blogging the Bookshelf header image 2

"Super Crunchers: How Anything Can Be Predicted", Ian Ayres

July 6th, 2009 · 1 Comment · Economics, ICT, Policy

supercrunchersSynopsis: Technological advances that dramatically reduce the costs of collecting and storing data combined with vast increases in computer processing power has made data driven decision making both more powerful and more feasible.

My Take: Ian Ayres is a great advocate. Perhaps the reason for this is that he divides his time and expertise between the law and economics (He is both the William K. Townsend Professor at Yale Law School and a Professor at Yale’s School of Management). As a result of these divided loyalties, as a writer, Ayres retains the enthusiasm of an amateur as well as a lawyer’s focus in prosecuting a case. This mix can produce some engaging and exciting advocacy, but it can also leave him somewhat blind to the limitations and obstacles to his cause. “Super Crunchers”, is no exception. In this book Ayres makes an enthusiastic and compelling case for the potential of data driven analysis, while completely overlooking the not insubstantial obstacles to realisation of this promise.

The core premise of “Super Crunchers” is a good one. The emergence of technologies that allow the collection of extraordinarily large datasets combined with the computer processing power that allows for the easy use of regression and randomisation trials to analyse these data sets does create an enormous opportunity for data driven decision making. And as Ayres demonstrates over and again in “Super Crunchers” the decisions informed by statistical algorithms frequently outperform those made by highly educated, but more intuitive, subject area experts.

The seminal example Ayres gives of how good data and a better algorithm can best the experts in the seemingly most subjective of fields is the story of Princeton economist, Orley Ashenfelter and his Liquid Assets wine newsletter. Armed only with an algorithm informed by time-series regression analysis Ashenfelter was able to predict the quality of Bordeaux vintages using only data on the vintage’s winter rainfall, average growing season temperature and harvest rainfall. While originally the subject of scorn and derision, Ashenfelter’s predictions proved to both gazump and better those of the connoisseurs on a consistent basis. Ashenfelter’s success and the publicity his cause attracted went on to inspire other data geeks to apply their statistical toolkits to other areas, most notably Billy Beane, the general manager of the Oakland Athletics, who applied the tools to the analysis of baseball prospects. Since the release of “Super Crunchers”, statisticians have achieved public success in a range of other fields including Daryl Morey, the General Manager of the Houston Rockets (“the Dork Elvis of the NBA”) in basketball and Nate Silver in politics.

However, while there are plenty of success stories and while the potential is real, the fact is that a utopia of ubiquitous, rational, data-driven decision making is a long way from reality (For a good Australian take on this see Andrew Leigh here). As David Leonhardt pointed out in the New York Times review of the book:

“Evidence-based medical treatment, to take one of his favorite examples, is still far from the norm in this country.”

While Ayres correctly identifies the potential of “Super Crunching”, I’m afraid he vastly under-estimates the social and institutional barriers to its proliferation.

For starters, there is a thicket of government regulation around privacy and data protection that stands in the way of data collection in a number of fields not the least of which, Medicine.

But moreso, the most significant barriers to the adoption of data-driven decision making are social and institutional. The ‘experts’ that Ayres anticipates being usurped by Super Crunching will not go slowly into the night.  These people currently hold positions of significant respect, power and influence by virtue of their ‘intuitive’ expertise. As has been seen in most of the examples that Ayres cites in his book, they will use their privileged and powerful positions to protect their current status whether via formalised professional standards or informal marginalisation of data geeks. While Ayres is cheerily optimistic about the ability for the demonstrably better Super Crunchers to naturally outperform and usurp the experts, I’m not as convinced. For the moment at least, the majority of fields of expertise are not quantitatively measurable. In a lot of areas, it’s not possible to decisively determine a winner and a loser in a contest between a Super Cruncher and a traditional expert. In these situations, the status quo will be a powerful obstacle to the adoption of wide spread data driven decision making.

Highlight:

“We are in a historic moment of horse-versus-locomotive competition, where intuitive and experiential expertise is losing out time and time again to number crunching.”

Tags: