Known Unknowns

‘The Signal and the Noise,’ by Nate Silver

Nate Silver has lived a preposterously interesting life. In 2002, while toiling away as a lowly consultant for the accounting firm KPMG, he hatched a revolutionary method for predicting the performance of baseball players, which the Web site Baseball Prospectus subsequently acquired. The following year, he took up poker in his spare time and quit his job after winning $15,000 in six months. (His annual poker winnings soon ran into the six-figures.) Then, in early 2008, Silver noticed that most political prognostication was bunk. Silver promptly reinvented that field, too. His predictive powers were such that at one point the Obama campaign turned to him for guidance.

These triumphs have built Silver a loyal following among fantasy-baseball aficionados and the political buffs who flock to his New York Times blog, FiveThirty­Eight. His signature approach is to concentrate enormous amounts of data on questions that lend themselves to pious blather. For example: television blowhards are fond of proclaiming that the winner of the Iowa caucuses enjoys a big bounce in the New Hampshire primary. Silver crunched numbers dating back to the 1970s and found that the bounce comes less from winning Iowa than from exceeding expectations there.

Silver’s method is completely straightforward: how else would you approach a question like this if not by considering every previous example? But the method is so empowering that it’s intoxicating — as if there’s no question he couldn’t answer with a big enough spreadsheet.

Which is why it’s slightly heartbreaking to read in the introduction to Silver’s new book, “The Signal and the Noise,” that, having set out to write a geek-conquers-world tell-all in the vein of “Moneyball” and “Freakonomics,” Silver decided to write an altogether different book. This one isn’t so much about his rise to statistical godliness, though it includes a smidgen of back story. It’s largely about evaluating predictions in a variety of fields, from finance to weather to epidemiology. We learn about a handful of successes: when, for instance, meteorologists predict a hurricane’s landfall 72 hours in advance, they now come within a 100-mile radius, whereas the radius was 350 miles a quarter-century ago. But mostly we learn about failures. It turns out we’re not even close to predicting the next catastrophic earthquake or the spread of the next killer bird flu, despite the enormous amounts of brainpower trained on these questions in the past few decades.

As science, this investigation is wholly satisfying. As a literary proposition, it’s a bit disappointing. It’s always more gripping to read about how we might achieve the improbable than why we can’t. And when books about statistical wizardry succeed, it’s generally on the strength of their narrative elements, not their analytical rigor. “Moneyball” was a classic underdog tale about the cash-deprived Oakland A’s; “Freakonomics” read like a series of detective stories. Silver’s volume is more like an engagingly written user’s manual, with forays into topics like dynamic nonlinear systems (the guts of chaos theory) and Bayes’s theorem (a tool for figuring out how likely a particular hunch is right in light of the evidence we observe).

Photo

Credit
Illustration by Timothy Goodman

And yet, while “The Signal and the Noise” doesn’t chronicle Silver’s rise, it marks an important milestone in his ascent. For that reason, it could turn out to be one of the more momentous books of the decade. Journalism is in a strange place these days. Cable and the Internet crippled the old media establishment; political polarization dealt it a death blow. In the meantime, no new establishment has risen up to take its place. What we have is a growing sense of intellectual nihilism. The right-wing media speak only to true believers. Liberal journalists are often more fact-conscious but equally partisan, while mainstream outlets have a rapidly dwindling audience. Few media institutions command widespread credibility.

I think Silver — or at least Silver-ism — has the potential to fill the void. Silver uses statistics to scrutinize the claims of people who don’t always have an incentive to be accurate. Until now, he took aim mostly at sports pundits and political handicappers. But the book hints at his ambitions to take on weightier questions. There’s no better example of this than his chapter on climate change. In recent years, the most sophisticated global-warming skeptics have seized on errors in the forecasts of the United Nations’ International Panel on Climate Change (I.P.C.C.) in order to undermine efforts at greenhouse gas reduction. These skeptics note that global temperatures have increased at only about half the rate the I.P.C.C. predicted in 1990, and that they flatlined in the 2000s (albeit after rising sharply in the late ’90s).

Silver runs the numbers to show that the past few decades of data are still highly consistent with the hypothesis of man-made global warming. He shows how, at the rate that carbon dioxide is accumulating, a single decade of flat temperatures is hardly invalidating. On the other hand, Silver demonstrates that projecting temperature increases decades into the future is a dicey proposition. He chides some environmental activists for their certainty — observing that overambitious predictions can undermine a cause when they don’t come to pass — without descending into false equivalence.

An error has occurred. Please try again later.

You are already subscribed to this email.

What Silver is doing here is playing the role of public statistician — bringing simple but powerful empirical methods to bear on a controversial policy question, and making the results accessible to anyone with a high-school level of numeracy. The exercise is not so different in spirit from the way public intellectuals like John Kenneth Galbraith once shaped discussions of economic policy and public figures like Walter Cronkite helped sway opinion on the Vietnam War. Except that their authority was based to varying degrees on their establishment credentials, whereas Silver’s derives from his data savvy in the age of the stats nerd.

That Silver is taking this on is, by and large, a welcome development. Few journalists have the statistical chops; most scientists and social scientists are too abstruse. Though his approach doesn’t apply to every issue, it’s not hard to imagine Silver and his ilk one day letting the air out of an inflating housing bubble, or unmasking tobacco-company spin, by appealing to nothing but the numbers.

Still, I can’t help feeling a twinge of ambivalence. Silver is scrupulous about not claiming more certainty than he has. He echoes the famous line from Donald Rumsfeld about “unknown unknowns” — knowledge gaps that we aren’t aware of because we haven’t even thought to ask the right questions. As he and his fellow stat-heads colonize more disciplines, will they know which questions to ask? Sorting through the numbers on climate change is a much more daunting challenge than figuring out which shortstops will hit for power or which candidate will carry Ohio. There are nuances in scientific and financial data — to say nothing about how we discuss the data in the context of a raging political debate — that people spend their careers assimilating. And, of course, the stakes are much higher when we’re talking policy.

As long as someone’s going to fill the role of public statistician, I’m glad it’s Nate Silver. But I do worry about the appearance of accuracy without the real thing. Statistics can dazzle with their aura of authority, yet reality is relentlessly messy. Genuine understanding, as even Silver knows, is more than a numbers game.

THE SIGNAL AND THE NOISE

Why So Many Predictions Fail — but Some Don’t

By Nate Silver

Illustrated. 534 pp. The Penguin Press. $27.95.

Noam Scheiber is a senior editor at The New Republic and the author of “The Escape Artists.”

A version of this review appears in print on November 4, 2012, on Page BR12 of the Sunday Book Review with the headline: Known Unknowns. Today's Paper|Subscribe