The Signal and the Noise: Why So Many Predictions Fail--but Some Don't

by Nate Silver

Overview

"One of the more momentous books of the decade." —The New York Times Book Review

Nate Silver built an innovative system for predicting baseball performance, predicted the 2008 election within a hair’s breadth, and became a national sensation as a blogger—all by the time he was thirty. He solidified his standing as the nation's foremost political forecaster with his near perfect prediction of the 2012 election. Silver is the founder of the website FiveThirtyEight. 
 
Drawing on his own groundbreaking work, Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.

In keeping with his own aim to seek truth from data, Silver visits the most successful forecasters in a range of areas, from hurricanes to baseball to global pandemics, from the poker table to the stock market, from Capitol Hill to the NBA. He explains and evaluates how these forecasters think and what bonds they share. What lies behind their success? Are they good—or just lucky? What patterns have they unraveled? And are their forecasts really right? He explores unanticipated commonalities and exposes unexpected juxtapositions. And sometimes, it is not so much how good a prediction is in an absolute sense that matters but how good it is relative to the competition. In other cases, prediction is still a very rudimentary—and dangerous—science.

Silver observes that the most accurate forecasters tend to have a superior command of probability, and they tend to be both humble and hardworking. They distinguish the predictable from the unpredictable, and they notice a thousand little details that lead them closer to the truth. Because of their appreciation of probability, they can distinguish the signal from the noise.

With everything from the health of the global economy to our ability to fight"terrorism"dependent on the quality of our predictions, Nate Silver’s insights are an essential read.

Product Details

ISBN-13:	9780143125082
Publisher:	Penguin Publishing Group
Publication date:	02/03/2015
Pages:	576
Sales rank:	102,388
Product dimensions:	5.40(w) x 8.40(h) x 1.30(d)
Age Range:	18 Years

About the Author

Nate Silver is the founder and editor in chief of FiveThirtyEight.com.

Read an Excerpt

At about the time The Signal and the Noise was first published in September 2012, “Big Data” was on its way becoming a Big Idea. Google searches for the term doubled over the course of a year,1 as did mentions of it in the news media.2 Hundreds of books were published on the subject. If you picked up any business periodical in 2013, advertisements for Big Data were as ubiquitous as cigarettes in an episode of Mad Men.
(Continues…)

---

Excerpted from "The Signal and the Noise"
by .
Copyright © 2015 Nate Silver.
Excerpted by permission of Penguin Publishing Group.
All rights reserved. No part of this excerpt may be reproduced or reprinted without permission in writing from the publisher.
Excerpts are provided by Dial-A-Book Inc. solely for the personal use of visitors to this web site.

---

Table of Contents

Preface to the Paperback Edition xiii

Introduction 1

1 A Catastrophic Failure of Prediction 19

2 Are You Smarter Than a Television Pundit? 47

3 All I Care About Is W's and Ls 74

4 For Years You've Been Telling Us That Rain Is Green 108

5 Desperately Seeking Signal 142

6 How to Drown in Three Feet of Water 176

7 Role Models 204

8 Less and Less and Less Wrong 232

9 Rage Against the Machines 262

10 The Poker Bubble 294

11 If You Can't Beat'em … 329

12 A Climate of Healthy Skepticism 370

13 What you Don't Know Can Hurt You 412

Conclusion 446

Acknowledgments 455

Notes 459

Index 517

Editorial Reviews

Not so different in spirit from the way public intellectuals like John Kenneth Galbraith once shaped discussions of economic policy and public figures like Walter Cronkite helped sway opinion on the Vietnam War . . . could turn out to be one of the more momentous books of the decade.” —New York Times Book Review

“Mr. Silver, just 34, is an expert at finding signal in noise . . . Lively prose—from energetic to outraged . . . illustrates his dos and don’ts through a series of interesting essays that examine how predictions are made in fields including chess, baseball, weather forecasting, earthquake analysis and politics… [the] chapter on global warming is one of the most objective and honest analyses I’ve seen . . . even the noise makes for a good read.” —New York Times

"A serious treatise about the craft of prediction—without academic mathematics—cheerily aimed at lay readers. Silver's coverage is polymathic, ranging from poker and earthquakes to climate change and"terrorism"" —New York Review of Books

"Mr. Silver's breezy style makes even the most difficult statistical material accessible. What is more, his arguments and examples are painstakingly researched . . ." —Wall Street Journal

"Nate Silver is the Kurt Cobain of statistics . . . His ambitious new book, The Signal and the Noise, is a practical handbook and a philosophical manifesto in one, following the theme of prediction through a series of case studies ranging from hurricane tracking to professional poker to counte"terrorism" It will be a supremely valuable resource for anyone who wants to make good guesses about the future, or who wants to assess the guesses made by others. In other words, everyone." —The Boston Globe

"Silver delivers an improbably breezy read on what is essentially a primer on making predictions." —Washington Post
 
“The Signal and the Noise is many things—an introduction to the Bayesian theory of probability, a meditation on luck and character, a commentary on poker's insights into life—but it's most important function is its most basic and absolutely necessary one right now: a guide to detecting and avoiding bullshit dressed up as data . . . What is most refreshing . . . is its humility. Sometimes we have to deal with not knowing, and we need somebody to tell us that.” —Esquire

“[An] entertaining popularization of a subject that scares many people off . . .Silver’s journey from consulting to baseball analytics to professional poker to political prognosticating is very much that of a restless and curious mind. And this, more than number-crunching, is where real forecasting prowess comes from.” —Slate

“Nate Silver serves as a sort of Zen master to American election-watchers . . . In the spirit of Nassim Nicholas Taleb’s widely read The Black Swan, Mr. Silver asserts that humans are overconfident in their predictive abilities, that they struggle to think in probabilistic terms and build models that do not allow for uncertainty.” —The Economist

"Silver explores our attempts at forecasting stocks, storms, sports, and anything else not set in stone." —Wired

"The Signal and the Noise is essential reading in the era of Big Data that touches every business, every sports event, and every policymaker." —Forbes.com

“Laser sharp. Surprisingly, statistics in Silver’s hands is not without some fun.” —Smithsonian Magazine

“A substantial, wide-ranging, and potentially important gauntlet of probabilistic thinking based on actual data thrown at the feet of a culture determined to sweep away silly liberal notions like ‘facts.’” —The Village Voice

“Silver shines a light on 600 years of human intelligence-gathering—from the advent of the printing press all the way through the Industrial Revolution and up to the current day—and he finds that it's been an inspiring climb. We've learned so much, and we still have so much left to learn.” —MLB.com

“Nate Silver’s The Signal and the Noise is The Soul of a New Machine for the 21st century (a century we thought we’d be a lot better at predicting than we actually are). Our political discourse is already better informed and more data-driven because of Nate’s influence. But here he shows us what he has always been able to see in the numbers—the heart and the ethical imperative of getting the quantitative questions right. A wonderful read—totally engrossing." —Rachel Maddow, author of Drift
 
“Yogi Berra was right: ‘forecasting is hard, especially about the future.’ In this important book, Nate Silver explains why the performance of experts varies from prescient to useless and why we must plan for the unexpected. Must reading for anyone who cares about what might happen next.” —Richard Thaler, co-author of Nudge

From the Publisher