The Signal and the Noise: Why Most Predictions Fail-but Some Don't

The Signal and the Noise: Why Most Predictions Fail-but Some Don't

3.8 42
by Nate Silver
     
 

View All Available Formats & Editions

One of Wall Street Journal's Best Ten Works of Nonfiction in 2012
 
New York Times Bestseller

“Not so different in spirit from the way public intellectuals like John Kenneth Galbraith once shaped discussions of economic policy and public figures like Walter Cronkite

Overview

One of Wall Street Journal's Best Ten Works of Nonfiction in 2012
 
New York Times Bestseller

“Not so different in spirit from the way public intellectuals like John Kenneth Galbraith once shaped discussions of economic policy and public figures like Walter Cronkite helped sway opinion on the Vietnam War…could turn out to be one of the more momentous books of the decade.”
New York Times Book Review
 
"Nate Silver's The Signal and the Noise is The Soul of a New Machine for the 21st century."
—Rachel Maddow, author of Drift

"A serious treatise about the craft of prediction—without academic mathematics—cheerily aimed at lay readers. Silver's coverage is polymathic, ranging from poker and earthquakes to climate change and terrorism."
—New York Review of Books

Nate Silver built an innovative system for predicting baseball performance, predicted the 2008 election within a hair’s breadth, and became a national sensation as a blogger—all by the time he was thirty. He solidified his standing as the nation's foremost political forecaster with his near perfect prediction of the 2012 election. Silver is the founder and editor in chief of FiveThirtyEight.com.
 
Drawing on his own groundbreaking work, Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the “prediction paradox”: The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.

In keeping with his own aim to seek truth from data, Silver visits the most successful forecasters in a range of areas, from hurricanes to baseball, from the poker table to the stock market, from Capitol Hill to the NBA. He explains and evaluates how these forecasters think and what bonds they share. What lies behind their success? Are they good—or just lucky? What patterns have they unraveled? And are their forecasts really right? He explores unanticipated commonalities and exposes unexpected juxtapositions. And sometimes, it is not so much how good a prediction is in an absolute sense that matters but how good it is relative to the competition. In other cases, prediction is still a very rudimentary—and dangerous—science.

Silver observes that the most accurate forecasters tend to have a superior command of probability, and they tend to be both humble and hardworking. They distinguish the predictable from the unpredictable, and they notice a thousand little details that lead them closer to the truth. Because of their appreciation of probability, they can distinguish the signal from the noise.

With everything from the health of the global economy to our ability to fight terrorism dependent on the quality of our predictions, Nate Silver’s insights are an essential read.


From the Trade Paperback edition.

Product Details

ISBN-13:
9781101595954
Publisher:
Penguin Publishing Group
Publication date:
09/27/2012
Sold by:
Penguin Group
Format:
NOOK Book
Pages:
544
Sales rank:
73,886
File size:
8 MB

What People are saying about this

Bill James
Projection, prediction, assumption, trepidation, anticipation, expectation, estimation... we wouldn't have 80 words like this in the English language if it wasn't central to our lives. We tend not to take prediction seriously because, on some level, we know that we don't know. Silver shows us how this inevitable part of life goes awry when projected on a grand scale into the murky worlds of politics, science and economics. Dancing through chess, sports, snowstorms, global warming and the McLaughlin Group, he makes a serious and systematic effort to show us how to clean the noise off the signal.
—Bill James (author of The Bill James Baseball Abstracts)
Peter Orszag
Here's a prediction: after you read The Signal and the Noise, you'll have much more insight into why some models work well-and also why many don't. You'll learn to pay more attention to weather forecasts for the coming week-and none at all for weather forecasts beyond that. Nate Silver takes a complex, difficult subject and makes it fun, interesting, and relevant.
—Peter Orszag (former director of the Office of Management and Budget)
Rachel Maddow
Nate Silver's The Signal and the Noise is The Soul of a New Machine for the 21st century (a century we thought we'd be a lot better at predicting than we actually are). Our political discourse is already better informed and more data-driven because of Nate's influence. But here he shows us what he has always been able to see in the numbers-the heart and the ethical imperative of getting the quantitative questions right. A wonderful read-totally engrossing.
—Rachel Maddow (author of Drift)
Richard Thaler
Yogi Berra was right: 'forecasting is hard, especially about the future.' In this important book, Nate Silver explains why the performance of experts varies from prescient to useless and why we must plan for the unexpected. Must reading for anyone who cares about what might happen next.
—Richard Thaler (co-author of Nudge)
Governor Jon Huntsman
Making predictions in the era of 'big data' is not what you might imagine. Nate Silver's refreshing and original book provides unpredictably illuminating insights differentiating objective and subjective realities in forecasting our future. He reminds us that the human element is still essential in predicting advances in science, technology and even politics... if we were only wise enough to learn from our mistakes.
—Governor Jon Huntsman

Meet the Author

Nate Silver is the founder and editor in chief of FiveThirtyEight.com.

Customer Reviews

Average Review:

Write a Review

and post it to your social network

     

Most Helpful Customer Reviews

See all customer reviews >

The Signal and the Noise: Why So Many Predictions Fail but Some Don't 3.8 out of 5 based on 0 ratings. 41 reviews.
popscipopulizer More than 1 year ago
*A full executive summary of this book will be available at newbooksinbrief . wordpress . com, on or before Monday, October 15, 2012. Making decisions based on an assessment of future outcomes is a natural and inescapable part of the human condition. Indeed, as Nate Silver points out, "prediction is indispensable to our lives. Every time we choose a route to work, decide whether to go on a second date, or set money aside for a rainy day, we are making a forecast about how the future will proceed--and how our plans will affect the odds for a favorable outcome" (loc. 285). And over and above these private decisions, prognosticating does, of course, bleed over into the public realm; as indeed whole industries from weather forecasting, to sports betting, to financial investing are built on the premise that predictions of future outcomes are not only possible, but can be made reliable. As Silver points out, though, there is a wide discrepancy across industries and also between individuals regarding just how accurate these predictions are. In his new book `The Signal and the Noise: Why So Many Predictions Fail--but Some Don't' Silver attempts to get to the bottom of all of this prediction-making to uncover what separates the accurate from the misguided. In doing so, the author first takes us on a journey through financial crashes, political elections, baseball games, weather reports, earthquakes, disease epidemics, sports bets, chess matches, poker tables, and the good ol' American economy, as we explore what goes into a well-made prediction and its opposite. The key teaching of this journey is that wise predictions come out of self-awareness, humility, and attention to detail: lack of self-awareness causes us to make predictions that tell us what we'd like to hear, rather than what is true (or most likely the case); lack of humility causes us to feel more certain than is warranted, leading us to rash decisions; and lack of attention to detail (in conjunction with self-serving bias and rashness) leads us to miss the key variables that make all the difference. Attention to detail is what we need to capture the signal in the noise (the key variable[s] in the sea of data and information that are integral in determining future outcomes), but without self-awareness and humility, we don't even stand a chance. In the final stage of the book Silver explores how the lessons that he lays out can be applied to such issues as global warming, terrorism and bubbles in financial markets. Unfortunately, each of these fields is a lot noisier than many of us would like to think (thus making them very difficult to predict precisely). Nevertheless, the author argues, within each there are certain signals that can help us make better predictions regarding them, and which should help make the world a safer and more livable place. If you are hoping that this book will make you a fool-proof prognosticator, you are going to be disappointed. A key tenet of the book is that this is simply not possible (no matter what field you are in). That being said, Silver makes a very strong argument that by applying a few simple principles (and putting in a lot of hard work in identifying key variables) our predictive powers should take a great boost indeed. A full executive summary of this book will be available at newbooksinbrief . wordpress . com, on or before Monday, October 15; a podcast discussion of the book will be available shortly thereafter.
ProfJane More than 1 year ago
I teach probability and statistics and really wanted to like this book, but Nate Silver is NOT on expert on statistics and probability and makes some pretty big errors as a consequence. Chapter 8 completely fails in its explanation of the controversy between classical and Bayesian methods. In other chapters, Silver seems to believe that anyone who revises his/her beliefs based on new information is using Bayes's theorem. Bayes's theorem involves reasoning based on how likely the information is if the hypothesis is true versus how likely the information is if the hypothesis is false.
Anonymous More than 1 year ago
Silver draws on a wide range of activities with examples from poker, sports, finance, etc. that involve his own experiences and interviews with others. His goal is to uncover significant principles and factors and not to expound a specific theory. The book is well organized and easy to read.
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago
I enjoyed reading this book and the author's efforts to sort out all the predictions that we encounter in daily life.  I needed to learn to put down the book after finishing a chapter so that I could absorb the points being made. Wish that the book would have been around when I was taking statistics.  Certainly he gave a lot of encouragement to thinking about probabilities rather than just doing counts.
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Georgia_writer More than 1 year ago
A very well-written and fascinating presentation on why some predictions work and many others fail. If you're at all interested in politics, economics, disease spread, terrorism, the stock market, weather, earthquakes, baseball, chess or poker you'll be captivated by The Signal and the Noise. I'm primarily a fiction reader, but found this book to be as much or more of a page turner than many "thrillers." Although the book is written for the layman, you'll have an easier time with it if you have a solid grounding in fundamental mathematics and statistics. Bottom line: if you're curious about the world we live in and why some events are easier to predict than others, I highly recommend this book.
Anonymous More than 1 year ago
BeeMD More than 1 year ago
I was inspired to do some research on Bayes' theorem after finishing the text. Footnotes and references make up a large part of the book, but are gathered at the end, not within the text.
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Nate Silver, has taken time out to write an eye opening book. Before reading this book I had taken prediction for granted, but after reading it, I have come to appreciate the effort that goes into prediction and also how we are surrounded with predictions. It has also helped improve my prediction ability, because those that predict rightly, controls the future and ultimately wins in the long run.
Excusemesir More than 1 year ago
I finished reading "The Signal and the Noise" a couple of weeks ago, and although I enjoyed it as I read it, I find that it is already fading into the background of other related books that I have read. Occasionally I come across a book that requires re-reading and pondering, but this is not one of them. On the other hand, if you are new to this discussion, Mr. Silver does a good job of organizing and summarizing the problems with making accurate and reliable predictions. My summary of the book (and of the field of predicting the future): Predict that tomorrow will be pretty much like today, and you will have a very high accuracy rate, but you'll be BOOOOORRRIIINNNGG. Predict great and dramatic changes, and you will usually miss, but if you do it in an authoritative and entertaining fashion, you'll get lots of attention anyway.
Anonymous More than 1 year ago
Some very interesting and useful concepts are explained. In particular the usefulness of Bayes Theorum for refining predictions is clearly set forth. The book needs editing. The same ground seems to be covered over and over. If you are short of time you could read just those chapters that deal with an area of your interest: gambling, weather, epidemics, or terrorism. I predict that 20% of those who start the book will not finish it.
Anonymous More than 1 year ago
Anonymous More than 1 year ago
Anonymous More than 1 year ago