The Signal and the Noise
Nate Silver shot to fame during the 2008 election, when out of the welter of political polling he correctly predicted the outcome of the presidential race in 49 out of 50 states. Since then his blog FiveThirtyEight — the name comes from the number of votes in the Electoral College — has been subsumed into The New York Times, where he nailed almost all the 2010 congressional and gubernatorial races. His role is somewhere between a commentator and a bookie. Political types might worship him, but for Silver politics is a purely quantitative undertaking, not so far from his original beat of fantasy baseball.
His new book, The Signal and the Noise, applies the FiveThirtyEight model of rigorous data analysis to a whole range of fields, from poker playing to hurricane forecasting. But readers expecting the second coming of Freakonomics will (I’m happy to say) be very disappointed. There are no wacky results from deep data mining, nor unintuitive outcomes tailor-made for dinner party fodder. Data itself is not Silver’s story; prediction is. We are producing more information, and faster, than ever before, and yet, “if the quantity of information is increasing by 2.5 quintillion bytes per day, the amount of useful information certainly isn’t.”
The financial ratings agencies, deluged with data, predicted that CDOs were as safe as AAA bonds. Health organizations failed to foresee the recent epidemics of bird flu and swine flu. The massive earthquake in L’Aquila in 2009 caught Italian authorities unawares. And the predictions of innumerate TV pundits “display about as much political acumen as a barbershop quartet,” with no better than an even chance of being correct. The rise of big data has not been accompanied by better predictions; if anything, we’re getting worse. (Silver naturally has a quantitative demonstration of this: in the corpus of scientific literature, instances of the word “predictable” overtook those of the word “unpredictable” in the 1950s; it kept climbing after that, but now “unpredictable” is making a comeback.)
Silver’s satisfyingly geeky solution to uncertainty is “Bayesian reasoning” – basically, updating predictions over time with new information. Bayes’ Theorem, devised in the eighteenth century, is a simple algebraic formula (although Silver exiles the equation to a chart on page 245): given the probability of a future event prior to some new occurrence and some other fundamentals, you can calculate the event’s probability posterior to that occurrence. Silver gives a litany of serious, not to say unpleasant, examples — is your husband cheating on you? Do you have cancer? Are planes going to fly into the World Trade Center? — to make a point beyond politics or sports betting: we have to “think probabilistically about the world, even when it comes to issues that we don’t like to think of as matters of chance.”
To an ideologue that sounds frustrating if not heretical. But Silver isn’t claiming that all events are random. On the contrary, he’s saying that there are limits to our knowledge, and the best way to understand the world given our epistemological blinders is to take new information as it comes and refine our perceptions accordingly. Poker players do this all the time: once your opponents bet or fold, you can reassess the relative strength of your own hand, and win a pretty sum if you do it right.
In the waning days of this presidential election — which he has consistently called in favor of the president, whom he now gives about an 80 percent chance of victory — Silver has taken serious stick from blowhards such as Joe Scarborough and the Politico crew, who don’t seem to understand that a close election doesn’t have to be an unpredictable one. (Time, last week, actually ran a column by their in-house pundit Joe Klein with the headline “I Don’t Know.”) Silver doesn’t just infuriate them because he’s almost always right, but because he’s right without any special information. Although the FiveThirtyEight algorithm has grown more sophisticated with the years, Silver isn’t a pollster, and he has no inside intelligence. And while Silver’s politics-as-fantasy-baseball stance can sometimes grate, his analysis is a far sight better than the ruminations of the perpetually inaccurate pundit class.
On Tuesday his model will face another test. Silver doesn’t know the outcome already, only the likelihood of possible outcomes. The problem, as he concedes in The Signal and the Noise, is that while the odds of an event may be borne out over 10,000 or 100,000 instances, the actual vote happens just once. Writing about poker, a game he played for years, Silver observes: “If we make a prediction and it goes badly, we can never really be certain whether it was our fault or not, whether our model was flawed or we just got unlucky.” The same, alas, goes for elections.