1 of 1 people found the following review helpful
on 25 July 2013
Nate Silver has shot to fame as the oracular figure who decoded political polling data into plain English and successfully predicted the US election. His debut book brings him back down to earth, using familiar examples as diverse as moneyball and warfare to demonstrate the sore lack of and need for better prediction in our lives, and the path to improvement through critical thinking and Bayesian reasoning.
Each chapter uses a particular area of prediction to teach broader lessons. The book opens with great momentum, using the financial crisis as a set of unambiguous examples of how not to make predictions before drilling into the all-too-human reasons that political commentators make poor election forecasts. There are good lessons here about how the need to feel confident and a single-minded focus on a few issues can lead one astray; he turns back to the financial crisis to emphasise the same failures there.
It's not all about the human factors, though, and Silver then turns to "moneyball" - statistics-based sports recruitment - to provide an overview of the more technical aspects of the art of prognostication. The idea of a predictive model is well articulated and applied to common-sense issues with surprising complications. With the reader warmed up, he spends several chapters digging into the fundimental reasons why level-headed and critically thinking scientists are unable to predict earthquakes. Some things - weather, disease, tectonic plates - are inherently challenging to forecast for interesting reasons, and he is equally quick to emphasise the technical traps that researchers can fall into in building their models.
The heart of the book, however, is Bayesian reasoning: the idea that we should take new predictions as adjustments to whatever our existing prediction said, as a sort of rolling improvement to our models. As a simple illustration, a test result indicating that one may have a rare disease should be combined with the low probability that one had the disease before the test results were in. Even if the test is 95% accurate, if the disease only affects one in a million people then the odds are far, far lower than 95% that one actually has the condition.
This is the tool Silver uses in the latter half of the book to show the way to better predictions, while still taking the time to illuminate other forecasting challenges. Whether it's poker or chess, the stockmarket or the battlefield, making a good model and refining it with new data is the key to victory. He lays out how the problems rise in these fields, be it a new raft of human frailties or the hefty challenge of trying to beat the "wisdom of the crowds", sets out how these failures in prediction can be capitalised by good agents or bad, and suggests Bayesian solutions.
A chapter on climate change in a book aimed at at those in big business has a huge potential to be a train wreck but Silver manages to weave a fairly acceptable course through the problem. This chapter acts to draw the book together, forcing together issues of complex models, noisy new data, and incentives to mislead, with Bayesian reasoning as the knight in shining armour. The overall theme is that climate models are difficult to make for fundamental reasons, and the warming consensus that has come out from those models has stood up to new results - despite the claims of think tanks who wish it otherwise.
This section has annoyed commentators on both sides of the issue. Silver manages to make good points without falling into the many huge rhetorical traps that the denialist movement has laid in any writer's path, but he's never particularly strong on the issue either. I liked the unspoken conclusion that less-confident predictions - 95% confidence rather than 99%, say - are more resilient to contradictory data in a Bayesian world, and Silver does not make false equivalencies and is unambiguous in supporting global warming. However this is not a strong introduction into climate science, or a real challenge to many of the incorrect claims made by denialists.
Truth be told this is a deliberate stylistic choice and potential issue throughout the book. Silver avoids bringing in controversies in the fundimental results that feed forecasts, except where it is directly relevant to a chapter's lesson. In the section on the financial crisis, human incentives are raised as a source of bias, but the humans responsible are hardly taken to task. If you want to find out about the failures of reasoning that permitted the 9/11 attacks, you'll have to read elsewhere. (Donald Rumsfeld appears but only as a lead into the "unknown unknowns" idea.) The implications of Scott Armstrong's work with the notoriously vociferous anti-climate-change Heartland Institute are left for the reader to find out about on their own.
This will variously come across as refreshingly expedient, frustratingly wishy-washy, focussed or cowardly depending on your reading preferences and ideological views. Consider yourself forewarned and take the book on its own terms.
The Signal and the Noise is certainly cleanly written and well-structured. Silver's introduction sets the book up as a toolbox, first outlining the failures of prediction and their causes before moving onto the successes and the processes that enable them, but in truth he allows the book to digress around the broader themes raised in each chapter, be it the problems and benefits of the "wisdom of the crowds" or the failure to properly, quantitatively account for the uncertainty in the prediction. These digressions are brief and enlightening, and echo back and forth between the chapters to make a more cohesive whole.
With the aforementioned caveat this is a superb route into the whole issue of modeling and forecasting. It's accessible, clearly written, technically sound and meticulously reasoned. It's recommended as reading on a difficult subject, although it's probably not going to prove to be the definitive work.
(If you want an primer to thinking about statistics before you dig into this I strongly recommend Darrell Huff's "How to Lie with Statistics". It's inexpensive, funny, brief, and makes a good companion piece.)
35 of 41 people found the following review helpful
on 27 September 2012
*A full executive summary of this book is available at newbooksinbrief dot com.
Making decisions based on an assessment of future outcomes is a natural and inescapable part of the human condition. Indeed, as Nate Silver points out, "prediction is indispensable to our lives. Every time we choose a route to work, decide whether to go on a second date, or set money aside for a rainy day, we are making a forecast about how the future will proceed--and how our plans will affect the odds for a favorable outcome" (loc. 285). And over and above these private decisions, prognosticating does, of course, bleed over into the public realm; as indeed whole industries from weather forecasting, to sports betting, to financial investing are built on the premise that predictions of future outcomes are not only possible, but can be made reliable. As Silver points out, though, there is a wide discrepancy across industries and also between individuals regarding just how accurate these predictions are. In his new book `The Signal and the Noise: Why So Many Predictions Fail--but Some Don't' Silver attempts to get to the bottom of all of this prediction-making to uncover what separates the accurate from the misguided.
In doing so, the author first takes us on a journey through financial crashes, political elections, baseball games, weather reports, earthquakes, disease epidemics, sports bets, chess matches, poker tables, and the good ol' American economy, as we explore what goes into a well-made prediction and its opposite. The key teaching of this journey is that wise predictions come out of self-awareness, humility, and attention to detail: lack of self-awareness causes us to make predictions that tell us what we'd like to hear, rather than what is true (or most likely the case); lack of humility causes us to feel more certain than is warranted, leading us to rash decisions; and lack of attention to detail (in conjunction with self-serving bias and rashness) leads us to miss the key variables that make all the difference. Attention to detail is what we need to capture the signal in the noise (the key variable[s] in the sea of data and information that are integral in determining future outcomes), but without self-awareness and humility, we don't even stand a chance.
While self-awareness requires us to make an honest assessment of our particular biases, humility requires us to take a probabilistic approach to our predictions. Specifically, Silver advises a Bayesian approach. Bayes' theorem has it that when it comes to making a prediction, the most prudent way to proceed is to first come up with an initial probability of a particular event occurring (rather than a black and white prediction of the form `I believe x will occur'). Next, we must continually adjust this initial probability as new information filters in.
The level of certainty that we can place on our initial estimate of the probability of a particular event (and the degree to which we can accurately refine it moving forward) is limited by the complexity of the field in which we are making our prediction, and also the amount and quality of the information that we have access to. For instance, in a field like baseball, where wins and losses mostly comes down to two variables (the skill of the pitchers, and the skill of the hitters), and where there is an enormous wealth of precise data, prediction is relatively straightforward (but still not easy). On the other hand, in a dynamic field such as the American economy, where the outcomes are influenced by an enormous number of variables, and where the interactions between these variables can become incredibly complex (due to things like positive and negative feedback), probabilities become a whole lot more difficult to pin down precisely (though they often remain possible on a general and/or long-term scale).
It is also important to recognize that while additional information can help us no matter what field we are trying to make our prediction in, we must be careful not to think that information can stand on its own. Indeed, additional information (when it is not met with insightful analysis) often does nothing more than draw our attention away from the key variables that truly make a difference. In other words, it creates more noise, which can make it more difficult to identify the signal. It is for this reason that predictive models that rely on statistics and statistics alone are often not very effective (though they do often help a seasoned expert who is able to apply insightful analysis to them).
In the final stage of the book Silver explores how the lessons that he lays out can be applied to such issues as global warming, terrorism and bubbles in financial markets. Unfortunately, each of these fields is a lot noisier than many of us would like to think (thus making them very difficult to predict precisely). Nevertheless, the author argues, within each there are certain signals that can help us make better predictions regarding them, and which should help make the world a safer and more livable place.
If you are hoping that this book will make you a fool-proof prognosticator, you are going to be disappointed. A key tenet of the book is that this is simply not possible (no matter what field you are in). That being said, Silver makes a very strong argument that by applying a few simple principles (and putting in a lot of hard work in identifying key variables) our predictive powers should take a great boost indeed. A full executive summary of this book is available at newbooksinbrief dot com; a podcast discussion of Silver's treatment of Bayes' theorem is also available.
2 of 2 people found the following review helpful
on 21 December 2012
Nate Silver's book is very well written and accessible read. It is also wide ranging in coverage though with a distinct American flavour. As he has a well deserved reputation as a successful Bayesian practitioner my expectations for a more insightful book were somewhat frustrated. However this is nevertheless a good enough and friendly introduction to the "Art and Science" of prediction.
2 of 2 people found the following review helpful
on 6 July 2014
If you are really into measuring signals in noise this isn't for you. If you want to be told, page after page, how brilliant the author is at, for example, football (US) statistics then you will find something of interest. Don't expect to find any useful information on regression; Bayes; predictor-corrector; Kalman; entropy; ... or just about anything to do with prediction.
1 of 1 people found the following review helpful
NS has achieved fame for his success in predictions in housing,stock market, elections, baseball, weather, earthquakes, and terrorist attacks particularly the US presideantial elections. But don't expect the book to enable you to do likewise as he doesnt disclose any personal algoriths but does reveal the kind of mental attitude which he adopts. He is very modest in the accuracy he believes can be achieved and warns "if you can't make a good prdiction it is harmful to pretend you can". His book explains how to think probabilistically and he aims to strike a balance between ciriosity and scepticism and believes the world is fundsmentally unknowable and that we should scrutunise and test our theories. This all sounds like common sense but the devil is in the detail and the book should be read alongside Taleb, Black Swan.
5 out of 5
1 of 1 people found the following review helpful
The review title is my main takeaway from the book (and it sounds like an obvious takeaway from the book).
Nate Silver is a wunderkind, a genius and, probably, as the website [ ... ] says a witch. He correctly predicted 50 results in 50 states during the last American Election. (So what's the point of holding an election? Just pass the data to Nate, he'll crunch the numbers and anoint the winner. It'll save billions of dollars. Let the witch decide. What can go wrong?)
If you're looking for a step by step chart of Silver's meteoric rise you won't find it here. What you will find is a really fascinating exploration behind the art and science (and witchery) of prediction and why predictions about the future are difficult.
28 of 33 people found the following review helpful
on 8 November 2012
It's readable in the early stages, and provides an easy introduction to Bayesian probability. The stumbling blocks are the interminable sections on baseball and poker. Unless you understand and are interested in these two diversions, you'll lose the will to live. That's a pity, because the man obviously knows what he's talking about, witness his success in analysing poll data for the US elections.
on 10 November 2014
The things which stuck out were:
- exposition of Bayes theorem: useful for financial market investors to ward against emotional decision making, because so often the hardest part of investing is sticking with your view when the market is going against you. Normal behaviour is to give greater weight to recent data: "the market is still doing X unlike I predicted, so I must be wrong", rather than to only slightly modify it "this doesn't change anything really, my original hypothesis is intact as anyway it was not predicated on immediate turning points being achieved".
- Kasparov vs Big Blue: how computers aren't as amazing as (non computer savvy) people often think they are - that they are merely extensions of human's own minds, not separate entities altogether.
- about earthquakes: that Tohoku was 'normal' if you take a long enough time horizon, and that many parts of the world will likely experience very large earthquakes despite not thinking they are earthquake zones. The recency of most earthquake information makes far too many complacent ("it only happen in Japan and California" for example)
- poker: that it's half luck and half skill, rather than mostly skill as many seem to think, and probably harder than making money in financial markets.
The book covers a lot of ground, is written in a fairly pleasant journalistic style, and offers a well rounded, humble take on the world of prediction. However it probably doesn't say anything much new, but is more a review or collection of existing information on various prediction related topics. There's plenty of name dropping from all the interviews with high level people like Rumsfeld, but it's not clear how much insight the interviews afforded. The lack of depth made me want to skim much of the book in search of real insights. Given how much the author seems to believe in the Bayesian approach, more elaboration on using it in practice, examples etc could have been much more rewarding.
on 2 July 2014
So I picked up this book thinking that it could help me in my stock picking choices by providing techniques, background, information etc on separating signals from noise. I realise after reading it that this is not that kind of a book at all, rather it is chapter after chapter of how past events could/should have been predicted and how people failed to do so. This is not really helpful as it is super easy in hindsight to see where things went wrong and then point out why we should have guessed at the time that it would happen. I found myself skipping chapters (e.g. the one about weather forecasting) simply because it was uninteresting to me and irrelevant to my life.
This book reminded me of Freakonomics in that it offered some different and interesting points of view on real life events but overall wasn't that interesting or useful.
The bulk of this book is devoted to proving that Bayesian theory works. To me this just sounded like common sense - asses the situation and make a prediction. Then change your prediction according to new information as it comes along.
With regards to the chapter on the stock market, what I took away from this is that all investors should just just buy an index tracker because no one (amateurs and professionals alike) can beat the market over the long term (the long term being decades). Investors might beat the market in the short term but evidence shows that this is off-set by losses that come along over the next month/year/decade etc and so once it is all averaged out no one beats the "efficient market".
on 23 March 2014
A difficult, challenging book, rather like (as we learn) its subject.
Prediction, it turns out, in many fields, is best done by people who analyse a wide range of data, who seek consensus with others, who prefer to assign a range of probabilities rather than suggest one outcome, and who revise their predictions in the light of their success: in other words, boring, geeky people.
It's done worst by those who trust their gut feelings, are simplistic, and appear on TV, and who when wrong, blame it on unusual circumstances.
Most predictions (done by either group) are over-confident, economic prediction being only slightly less successful than forecasting earthquakes (which is useless). The best, chewed-over, consensual economic forecasts of annual GDP growth are massively, catastrophically wrong a third of the time. If you want them to be right 90% of the time you must assume they could be wrong on their GDP numbers by a massive plus or minus 3.2 points. So the UK's predicted (?) 2.6% this year could actually turn out as a recession, or an Asian-style 5.8%: meaningless. Once a decade, actual GDP will exceed even that range (probably through a recession) And yet the government must plan using these ridiculously wayward numbers.
Weather forecasting is one of the few areas that has improved over the years.
All this Nate Silver describes in deep but probably necessary detail over several areas of his particular expertise (poker, baseball, politics and the stock market), as well as a few areas that he is probably not so at home with (earthquakes and climate change). A strength of the book is the statistical heft behind his assertions; a weakness, possibly, is that by the end you feel you have read a textbook rather a work of popular science. i'm not sure that's a criticism, but you may as well know what you are letting yourself in for.
Also, the book makes absolutely no concessions to a non-American audience, not even bothering to explain baseball terms or what Moneyball is or was (for example), another example perhaps of editors being useless, overworked, non-existent or cowed. What are publishers for these days? It's hard to tell.
Some conclusions I came away with:
1. Most forecasts of most things should be taken with a pinch of salt.
2. It's (broadly) impossible or at the least not worth trying to beat the stock market by trading or stock-picking; an index tracker will put you within a cat's whisker of the best over the long term.
3. Be very suspicious of silver bullets
4. Not all forecasts are terrible. Some people still make big money on sports gambling, for example, ideally (and this book will help greatly) get an understanding of the forecasting record in a particular field before you commit yourself to it; some are much more reliable than others.
For an example of the last point, Nate Silver has compared polling data with outcomes and one of his data points says that if someone is 20 points ahead in the polls, six months out from (in this case a senatorial) election, there is a 93% chance they will win. As I write this, the vote for Scottish independence from the UK is six months in the future. The no-to-independence people are (I think) about 20 points ahead. Nate Silver himself has said the yes camp has 'almost no chance' of winning, and he correctly predicted every state in the last presidential election. in Scotland the talk is of undecideds, shifting sands, late surges. Don't believe it. The bookies will give you odds of 4-1 for a no vote, so a little punt on a yes (if you betted, which I don't) would be a nice earner.
An insightful, even indispensible, but far from easy book.