The power of statistics

About three decades ago statistics a la USA arrived in rugby league. This had some interesting effects. One year (1976?) there was an epic battle between Ray Higgs (Parra)   and Terry Randall (Manly) for a big prize for the leading tackler in the comp. Then someone realised that the club statisticians were putting their thumbs on the scale to inflate the count for their man, so the competition was called off.

Despite the rude remarks that I have made about econometrics, I appreciate the power of statistics, properly used. Regression analysis is  a tool and it is no better than the questions that are asked when it is used to extract answers.

This book about baseball appears to be  a fantastic illustration of the proper way to use statistics  (but check out Tony T’s review, comment #1 for balance).  It seems that baseball until very recently was full of myths that persisted despite the highly visible nature of the game and the results, the size of the stakes and the intelligence and experience of the players, coaches, commentators and everyone else involved in playing and watching the game.

It is the story of Billy Bean, a star recruit who never actually made it in the big league, then  became a highly effective  administrator.

Beane was a much better baseball analyst than baseball player, and he quickly moved up the Oakland club’s hierarchy. He became interested in a simple question: what is the most efficient way to spend money on baseball players? The origins of Beane’s iconoclastic answers can be found in the writings of Bill James, a once obscure but now legendary baseball writer-statistician. While working as a night watchman for a pork-and-beans factory, James decided that he wanted to write about baseball in a way that would illuminate what really happened and why. In his view, conventional statistics were insufficiently helpful and sometimes downright misleading. Consider the area of defensive play. When a player mishandles a ball or makes a bad throw, he can be assigned an “error.” A player who accumulates a lot of errors seems like a bad fielder, whereas one with few errors seems really good. The problem is that a player may accumulate errors in part because he is unusually good at getting to the ball. If you do not get to the ball, you do not get an error (according to the chapter on scoring in The Book). So errors are a crude measure of fielding ability.

Or consider walks. Since the late nineteenth century, walks have been treated, in official statistics, as neutral–neither good nor bad. According to a nineteenthcentury expert whose advice is followed to the present day, “There is but one true criterion of skill at the bat, and that is the number of times bases are made on clean hits.” Of course, many people realized that a walk is a positive event for the hitting team and a negative event for the team in the field, but this commonsense notion was not incorporated into baseball’s most common measure of batting skill, the batting average, which leaves walks out.

The statistical method was the only way for Beane to solve a serious problem: obtaining first-rate talent without a lot of money. After all, the New York Yankees had three times the budget of the Oakland Athletics. And if Beane did find good players, and they performed well, they would be bid away by richer teams. Owing to his low payroll, he would be forced to replace his own greatest successes. In 2001, Oakland won 102 games in the regular season, the second-highest total in baseball. They lost three players widely regarded as their best, and they were expected by many to have a catastrophic fall. Instead they used statistical methods to try to replace the lost players with new ones who would provide statistical equivalents–and they ended up winning 103 games, the most in baseball. Their payroll for that year was $34 million, less than half that of their division rivals the Seattle Mariners. In Lewis’s account, Beane was able to succeed because “the market for baseball players was so inefficient, and the general grasp of sound baseball strategy so weak, that superior management could still run circles around taller piles of cash.”

There is an even larger puzzle. Why didn’t someone like Beane come along sooner? Why didn’t baseball executives start using statistics a decade, or two decades, or three decades, earlier? Why have falsehoods and mistakes persisted? The economic stakes are extremely high, after all, and if Lewis is correct, the management of most baseball teams could have saved many millions of dollars simply by making more rational personnel decisions. Nor was the important information hard to find. James’s arguments have been around for nearly two decades. In a market as competitive as major league baseball, surely the information should have been used, and fast. What went wrong?

The problem is not that baseball professionals are stupid; it is that they are human. Like most people, including experts, they tend to rely on simple rules of thumb, on traditions, on habits, on what other experts seem to believe. Even when the stakes are high, rational behavior does not always emerge. It takes time and effort to switch from simple intuitions to careful assessments of evidence.