Discover millions of ebooks, audiobooks, and so much more with a free trial

Only $11.99/month after trial. Cancel anytime.

Unavailable
The Signal and the Noise: Why So Many Predictions Fail-but Some Don't
Unavailable
The Signal and the Noise: Why So Many Predictions Fail-but Some Don't
Unavailable
The Signal and the Noise: Why So Many Predictions Fail-but Some Don't
Audiobook16 hours

The Signal and the Noise: Why So Many Predictions Fail-but Some Don't

Written by Nate Silver

Narrated by Mike Chamberlain

Rating: 4.5 out of 5 stars

4.5/5

()

Currently unavailable

Currently unavailable

About this audiobook

Nate Silver built an innovative system for predicting baseball performance, predicted the 2008 election within a hair's breadth, and became a national sensation as a blogger-all by the time he was thirty. He solidified his standing as the nation's foremost political forecaster with his near perfect prediction of the 2012 election. Silver is the founder and editor in chief of FiveThirtyEight.com.

Drawing on his own groundbreaking work, Silver examines the world of prediction, investigating how we can distinguish a true signal from a universe of noisy data. Most predictions fail, often at great cost to society, because most of us have a poor understanding of probability and uncertainty. Both experts and laypeople mistake more confident predictions for more accurate ones. But overconfidence is often the reason for failure. If our appreciation of uncertainty improves, our predictions can get better too. This is the "prediction paradox": The more humility we have about our ability to make predictions, the more successful we can be in planning for the future.

In keeping with his own aim to seek truth from data, Silver visits the most successful forecasters in a range of areas, from hurricanes to baseball, from the poker table to the stock market, from Capitol Hill to the NBA. He explains and evaluates how these forecasters think and what bonds they share. What lies behind their success? Are they good-or just lucky? What patterns have they unraveled? And are their forecasts really right? He explores unanticipated commonalities and exposes unexpected juxtapositions. And sometimes, it is not so much how good a prediction is in an absolute sense that matters but how good it is relative to the competition. In other cases, prediction is still a very rudimentary -- and dangerous -- science.

Silver observes that the most accurate forecasters tend to have a superior command of probability, and they tend to be both humble and hardworking. They distinguish the predictable from the unpredictable, and they notice a thousand little details that lead them closer to the truth. Because of their appreciation of probability, they can distinguish the signal from the noise.

Editor's Note

Smart statistics…

Whether in baseball or elections, Nate Silver’s data-driven approach & crazily accurate predictions have made him a household name. He expands on his successful approach in this fun & direct guide to predictions.

LanguageEnglish
Release dateSep 27, 2012
ISBN9781101590072
Unavailable
The Signal and the Noise: Why So Many Predictions Fail-but Some Don't
Author

Nate Silver

Nate Silver is the founder and editor in chief of FiveThirtyEight.com.

Related to The Signal and the Noise

Related audiobooks

Politics For You

View More

Related articles

Reviews for The Signal and the Noise

Rating: 4.3884297520661155 out of 5 stars
4.5/5

121 ratings31 reviews

What did you think?

Tap to rate

Review must be at least 10 words

  • Rating: 5 out of 5 stars
    5/5
    Nate Silver has become famous for his highly accurate predictions in the 2008 and 2012 US presidential elections. In this excellent book he talks more about the limitations of predictions than about his successes - pleasingly self effacing (particularly after recently reading Taleb's Black Swan). After skewering the ratings agencies for their fundamental role in the global financial crisis of 2008, he polishes off the fatuous talking heads and pundits of TV and other media before moving to a success story - weather forecasting. Through it all he makes clear where forecasting can help, and where its potential is limited. A balanced, nuanced and carefully written book which, unfortunately, won't change the world as much as it deserves.Read January 2014.
  • Rating: 5 out of 5 stars
    5/5
    Among the best non-fiction i've read recently, up there with Shapiro's 'Contested Will'. Like Shapiro, he is at ease ranging over a huge range of stuff and bringing it to light. Sets the issues of data-handling, prediction and uncertainty in historical and scientific contexts. The core is Bayes' theory which is fairly mathematical for me to grasp fully but the surprise key seems to be: subjective sense of what is probable must be built into the overall projection; total objectivity is an illusion. RA Fisher comes in for a basting for advocating the elimination of all subjectivity; the fact that he was a smoking/cancer denier in the pay of tobacco companies while puffing away provides an extra coffin-nail! The fun part of the book is how Silver explores implications in many fields: - politics, weather and climate change, epidemiology, earthquakes, terrorism, flight safety and more - throwing anecdotal and analytical light as he goes. Weather forecasters do better when they look out of the window, not just at their charts, just as generals do better if they visit the front-line. Earthquakes remain highly impervious to prediction largely because the essential data is hidden below ground. "Big data", is nothing new;it's been expanding ever since the invention of writing. But now as it grows exponentially, there is a danger in expecting more data (noise) to provide better understanding. What we need are better tools for analysis and pattern discernment, that is, for detecting signal. And, Silver tells us, Bayes theorem is one of the most important.I was apprehensive of his background in baseball and poker (two fields that I abhor) and confess to skimming those chapters.
  • Rating: 5 out of 5 stars
    5/5
    Want to know what I do and/or how I think (generally)? Read this.
  • Rating: 5 out of 5 stars
    5/5
    A remarkably clear and comprehensive explanation and defense of Bayesian reasoning. Should be required reading for anyone who thinks there is a simple "system" to beat the market, stop a terrorist, win the World Series of Poker, or even predict the weather. Includes useful anecdotes from Silver's life experiences. I particularly liked the watered-down (but more accurate) version of the efficient market hypothesis.
  • Rating: 5 out of 5 stars
    5/5
    I first encountered Nate Silver at baseballprospectus.com where he developed player statistical projections for upcoming seasons. When he branched into politics with 538.com prior to the 2008 election I was ecstatic. Finally a push against the empty pomposity that makes up most political coverage in the U. S.! I wasn't disappointed. This is an excellent, readable approach to understanding probability and prediction. The 'gets' Silver's success at 538 garnered are tremendous. He discusses the housing bubble with Nobel-winnings economists, baseball with Bill James, Deep Blue's IBM creators, known unknowns and unknown unknowns with Dick Cheney, and concludes with a strong chapter on what to do about it. Fascinating.
  • Rating: 5 out of 5 stars
    5/5
    If only it were fashionable to state your initial belief and then set about learning things that will help you modify it in a meaningful way instead of only interacting with people and ideas that solidify it. His examples are fascinating, he actually considers other interpretations of data and he provides a plan for coming to conclusions that are certainly more considered than they might have been before applying his Bayesian approach. What an interesting and practical book—I read the news differently because of it.
  • Rating: 4 out of 5 stars
    4/5
    Excellent work on predictions, and forecasting. I'd have liked the author to spend a lot less time on the intricacies of sports, but otherwise it's well done.
  • Rating: 4 out of 5 stars
    4/5
    In which Nate Silver argues for better forecasting and lionises Baynesian method. Its all neatly set out and impossible to argue with - or rather to do so would be like arguing against the law of gravity. But I would have liked a broader range of examples; for example is dissection of the poor predictions and forecasting of the financial markets is will known by now, true though it is. His successes in the field of political prediction are also well known - and as he admits himself, competing methods had set a pretty low bar for him to beat. And as he admitted in a trip to Australia, he wouldn't be able to replicate his success there (at least not on a state by state basis) due to lack of data at the state levelHe spends some time discussing baseball - a field which is, as he accepts, unusually rich in historical data. This is interesting, and whilst not a baseball fan myself, I could see how baseball is unique. A team game yes, but one where individual clashes predominate. Its unlikely the same approach could be taken to cricket. He also spends time on poker, a topic that personally interests me not at allThe best chapters are those on weather forecasting - where forecasting and predictions are getting better and better , and climate science specifically global warming. And here he nails his argument. Noone disputes the greenhouse effect, few serious people dispute the heating effects of increased greenhouse gases. The problem of course is predicting what will happen; noisy data, patchy historical data, complicating factors galore (sunspots, the cooling effects of the Mt Pinatubo erruption, the cooling effects of sulpher, ENSO etc etc) make predictions difficult. And, as Mr Silver points out, what so called "sceptical" climate scientists disagree about are the predictive models - not the core scienceBut as Mr Silver points out throughout the book, the goal is not a perfect forecast, but better forecasting. As we get new data we can improve our forecasts. After all, better forecasting is at least an improvement (although in my view, when he tries to use Bayes method to predict terrorist activity, a forecast of the likelihood of an event taking place is not of a great deal of use without knowing in what direction to look for it)Still - its an excellent book. Should be read in schools. And in government
  • Rating: 5 out of 5 stars
    5/5
    The Signal and the Noise was a really great read. It's one of those books that lets you annoy your friends by tediously repeating facts, many of which they already have picked up from reading the book, reading reviews, or other tedious friends. Like when the Weather Channel says it is a 30% chance of rain it rains 30% of the time, but that when it says a 10% chance of rain it really means a 3% chance of rain (they would rather people be pleasantly surprised). Or that earthquakes are unpredictable. Or that minor league baseballs players are difficult to predict and give a good advantage to very good scouts, while in the majors it is different. Or that a particularly great professional sports better will win 56 percent of the time. Or other examples drawn from the areas Nate focuses on: political predictions, elections, the macroeconomy, financial markets, epidemics, earthquakes, terrorism, baseball, chess, and poker.

    There is a deeper and more important set of lessons in the book to anyone that has not been sufficiently exposed to Bayesian methods. None of that was new to me, but it is still interesting to read and should be mandatory reading for anyone who has not been exposed to it before.

    I take some issue with the presentation of economics. Nate is completely right that macroeconomic forecasting has a terrible track record, and does not even appreciate how terrible its own record is. But he doesn't seem to recognize that there is a lot more to economics than macroeconomic forecasts. And, at least as much of the fault with those forecasts lies with the people demanding and using them as with the people providing them.

    And I'm not quite as impressed with Nate's election forecasting--anyone relying just on public polls taken in the days before the election would have correctly picked 49 or 50 of the 50 states in both 2008 and 2012. Nate did not have any magic, but he did have much better perspective on the uncertainty in the forecasts and how to read/interpret the significance of movements well in advance of election day.

    But those are quibbles, this book really deserves wide readership, probably starting with all of those who rushed out to buy it after the election and still have it sitting on their shelves.
  • Rating: 4 out of 5 stars
    4/5
    After finishing this book, I found myself charmed by it and liking it a great deal more than I expected. Silver does not promote himself as a great predictor, but he does describe what his method is. He makes a strong argument for using his favored methods for making predictions (primarily using Bayes' Theorem).

    Silver does take a number of pot-shots at television pundits, however, since the book was written before the 2012 election and considering the ass-whupping Silver and his data-crunching colleagues delivered to the Corps of Punditry (tm), I'll have to invoke the Ali rule and declare: "It ain't bragging if it's true." Silver makes a very convincing argument that a forecaster using sound techniques and with access to adequate data sources will out-forecast any number of experienced experts using only their gut.

    In the end, however, Silver does not set up Bayesian forecasting as a replacement for expertise and personal wisdom. His goal is more accurate predictions, not winning a fight or a popularity contest. I believe Silver's wisdom comes from his background at making predictions. Both in baseball and in poker, the forecaster is wrong more often then right. Silver's approach is not to over-inflate his own reliability, rather to accurately represent how accurate his predictions are and to honestly work to make them more accurate in the future.

    Implicitly then, this is a book about rhetoric (or how to make arguments with forecasts) as it is about the act of forecasting itself.

    It is curious that for a book about crunching data (and big data) Silver makes his arguments with words rather than with numbers. Readers wanting to get their hands on some Bayesian forecasting will have to look elsewhere for the math.

    Overall: this is a very good book and I can recommend it to almost anyone.
  • Rating: 2 out of 5 stars
    2/5
    Disorganized a bit too heavy for my tastes.
  • Rating: 3 out of 5 stars
    3/5
    I bought "The Signal and the Noise" after I saw a recommendation (I think it was in "The Economist"). As everyone knows by now, Nate Silver became famous after correctly predicting the outcome of the US election, down to the last state. In this book, he strives to separate the chaff from the wheat when it comes to predictions about the future. In Silver's words, "the difference between what we know and what we think we know". He urges us to think in probabilistic terms, explaining Bayes Theorem of conditional probability and why he thinks it's relevant to almost any prediction: baseball scores, politics, stock market, etc. He calls for more modesty from TV pundits, and that is hardly a call one can argue with.
  • Rating: 4 out of 5 stars
    4/5
    Have you ever wondered how to make good predictions about difficult-to-predict phenomena? Can you handle occasionally dry descriptions of statistical models and predictive strategies? Read this book.

    That said, I agreed with this book more often than I enjoyed it. The topic of predictions and the information about them is great, but I found many of the specific examples uninteresting. Baseball, for instance, is meaningless and dull to me. Climate change, weather patterns, and earthquake predictions are not topics that I'd seek out detailed discussions about, but in this context they were alright. The sections dealing with predictions about politics, terrorism, and disease pandemics were much more interesting to me.

  • Rating: 4 out of 5 stars
    4/5
    A very good book on forecasting and Bayesian thinking. It's a bit heavy on the examples and a bit light on the details, but I suppose equations are off-putting and the author may want to keep his methods hidden. That's fine, but it rather undermines the book's title. Still it's enjoyable, intelligent, and good at explaining how to view the world in a probabilistic manner.
  • Rating: 4 out of 5 stars
    4/5
    Very enjoyable read with some fabulous chapters. I wanted to read it for the Pecota chapters but I loved most many of the politics, climate and health chapters even more.
  • Rating: 2 out of 5 stars
    2/5
    Full of BS about 9/11 . Waste ot time mostly.

  • Rating: 5 out of 5 stars
    5/5
    Comprehensive subject analysis. The author covers diverse areas to illustrate what works and what does not. Some areas may feel repetitive but hang on and go thru the various dimensions.
  • Rating: 5 out of 5 stars
    5/5
    The book clinically examined statistics, techniques, and modern practices. You will not find a book on FiveThirtyEight. Instead, you will find a greater understanding of the forecasting industry. The book's analysis on confident intervals, and why forecasters choose a single number, explain the "shock" of Brexit and the 2016 election.
  • Rating: 3 out of 5 stars
    3/5
    I'm a huge fan of Nate Silver's blog FiveThirtyEight.com. It's the most interesting and convincing and reliable blog for election poll analysis.

    This book is good-ish, but it's no FiveThirtyEight. I feel like the blog entries are sharper and more on point than the book.

    That said, I read the whole book, and I was rarely bored.
  • Rating: 5 out of 5 stars
    5/5
    In the movie 'The Graduate', Dustin Hoffman's character is given career advice in the form of one word, 'plastics'. If a remake of the movie were to be done in present times, that word might just be 'statistics'. Nate Silver embodies this worldview and through a hodgepodge of disparate but interrelated vignettes he explores how statistics and probability play out in such areas as sports, economics, chess, poker, climate, politics, and terrorism. He is an admitted bayesian, a follower of the probabilistic approach based on the works of the 18th century reverend and mathematician Thomas Bayes, and a proponent of thinking like a fox as opposed to thinking like a hedgehog. While hedgehogs have one defining ideology or model of the world, foxes have multiple competing models, and using Bayesian probability, constantly update their predictions when new data comes in. Bayesian probability is based on subjective predictions, not on objective statistical distributions like those found in the frequentist school founded by Fisher, a 195h century biologist. It remains to be seen whether the bayesian approach is the 'true' way, but it certainly does seem more effective in practice. Silver spends a lot of time delving into the nuances of each of the example fields he explores, with just enough detail to offer insightful commentary without overwhelming the lay reader with advanced mathematics. In an increasingly unpredictable and complex world, Silver gives us a glimpse of the skills necessary to wrangle the messy reality which we find ourselves living in.
  • Rating: 5 out of 5 stars
    5/5
    A worthwhile read, with lots of examples from his career as a poker player, pollster and sports fan. Could have been subtitled "Why Bayesian Thinking Matters." I bought this for work purposes but enjoyed it on a personal level too.
  • Rating: 4 out of 5 stars
    4/5
    Statistics are fun, it's why a lot of us are enamored with sports. While a player's OBP might be useful for your fantasy league, statistics and probability are far more useful in the science of prediction. This includes such subjects as political polling, poker, weather and earthquake forecasting. Sports statistician Nate Silver covers all of these and more in The Signal and the Noise. While entertaining us with stories about sports book, disease probability, and devastating hurricanes, Silver also teaches us about the science of prediction and how it differs from basic probability and other statistics-based sciences. We learn how sample size can skew results...just recently a sports statistician claimed the Blackhawks record-setting season start in hockey was a 1-in-750 year event. How can he say such a thing when hockey has been with us less than a century? Well, it seems predictions can be made based on available statistics...but we should also not be surprised when models based on limited data don't pan out as predicted.Silver teaches us how to evaluate such predictions -- whether to be skeptical because they are based on specious data or sample size, or when we should trust it because of sound modeling, such as present weather forecasting. This is an interesting look at the current state of prediction in many fields...and the gains in accuracy made in just the past 20-30 years. Still, no matter how good you are at crunching numbers, you are unlikely to consistently beat Wall Street or the local bookie. Those who make a living gambling do so with slim margins and a large bank roll. Silver has made a living both gambling and working for sports statistics companies. Hearing his take is interesting...and a wake up call for anyone who thinks they have a system to beat Vegas. I always enjoyed seeing Silver on TV (he's been on The Daily Show) and hearing him on radio. I hope he writes additional books on the topic. as a sports fan, I have a natural affinity for statistics and the meaning of.
  • Rating: 5 out of 5 stars
    5/5
    This is probably one of the best nonfiction books I've ever read. Predictions are everywhere in our lives: we choose mates, jobs, our clothes in the morning, a school for our children based on our possibly poorly informed, influenced yet usually overly confident forecasts. And of course, there are those who make a living out of predicting the weather or the evolution of the stock market (and then bet somebody else's money). Nate Silver very efficiently proves that nearly every prediction we make is flawed because we mistake noise for signal, dismiss low probabilities as "impossibilities" and tend to prefer listening to over-confident forecasters rather than cautious ones. It is a vital reality check: this book should be mandatory reading to every professional forecasters, especially economists and policy makers.It is a very dense text, though. The theory is very well explained and expertly illustrated, but there remains an awful lot of it. I also had trouble with the arcane baseball terminology that is used repeatedly to illustrate key concepts. Whereas all other examples are nicely introduced to help the reader understand, a prior knowledge of baseball seems to be a prerequisite for this book... I didn't like that, and it cost the book its otherwise well-deserved fifth star!
  • Rating: 5 out of 5 stars
    5/5
    The Signal and the Noise was a really great read. It's one of those books that lets you annoy your friends by tediously repeating facts, many of which they already have picked up from reading the book, reading reviews, or other tedious friends. Like when the Weather Channel says it is a 30% chance of rain it rains 30% of the time, but that when it says a 10% chance of rain it really means a 3% chance of rain (they would rather people be pleasantly surprised). Or that earthquakes are unpredictable. Or that minor league baseballs players are difficult to predict and give a good advantage to very good scouts, while in the majors it is different. Or that a particularly great professional sports better will win 56 percent of the time. Or other examples drawn from the areas Nate focuses on: political predictions, elections, the macroeconomy, financial markets, epidemics, earthquakes, terrorism, baseball, chess, and poker.There is a deeper and more important set of lessons in the book to anyone that has not been sufficiently exposed to Bayesian methods. None of that was new to me, but it is still interesting to read and should be mandatory reading for anyone who has not been exposed to it before.I take some issue with the presentation of economics. Nate is completely right that macroeconomic forecasting has a terrible track record, and does not even appreciate how terrible its own record is. But he doesn't seem to recognize that there is a lot more to economics than macroeconomic forecasts. And, at least as much of the fault with those forecasts lies with the people demanding and using them as with the people providing them.And I'm not quite as impressed with Nate's election forecasting--anyone relying just on public polls taken in the days before the election would have correctly picked 49 or 50 of the 50 states in both 2008 and 2012. Nate did not have any magic, but he did have much better perspective on the uncertainty in the forecasts and how to read/interpret the significance of movements well in advance of election day.But those are quibbles, this book really deserves wide readership, probably starting with all of those who rushed out to buy it after the election and still have it sitting on their shelves.
  • Rating: 4 out of 5 stars
    4/5
    Nate Silver is the wunderkind who accurately predicted how each state would vote in the 2012 presidential election. So he has more than a little authority when it comes to predictions. The Signal and the Noise is a fascinating read, with great examples from many fields in which predictions are the order of the day – from weather forecasting to baseball to poker to political punditry. I particularly liked his take on how the “smartest minds” on PBS’s The McLaughlin Group go so far astray in their predictions … and why. Although this book prompted to dust off my rusty algebra skills to figure out and use Bayes’s theorem, most of it was a pretty easy read. I was disappointed to find so many little errors in the book (the misspelling of the word “mammogram” is just one example). When a book relies heavily on accuracy, it’s a shame when the copyeditors don’t show up for work. Anyone who wants to understand how Mitt Romney’s pollsters got it so wrong (it’s likely they were “hedgehogs” and not “foxes”) needs only to read The Signal and the Noise.
  • Rating: 4 out of 5 stars
    4/5
    Silver understands how statistic works, and so he uses this hammer for any nail. It is not exactly the way but sometimes I had the feeling. Another strange part for a reader from Germany are the details of the football league and the poker play.Both not well known here in Germany. But I learned a lot, how to have a reality check, what has to be done, if new information drops in. The Bayesian seems to be very efficient in prediction problems, even if it doesn't look that clear in the first glance.Especially helpful was the chapter about market prediction, earth quakes and climate change. It gave me new insights, how people misunderstand signals and noise. And exactly this chapters, I wish, many more would read.
  • Rating: 4 out of 5 stars
    4/5
    I liked the book, but maybe not as much as I'd hoped to. My expectations were pretty high - maybe unrealistically so - given the author's success predicting the 2008 and 2012 elections (and his appearances on "The Daily Show with Jon Stewart"). He is at his best describing his own work with the elections, baseball, and poker - though I felt like he was really getting too far down in the weeds with his descriptions of poker. The portions of the book where he ventured into other people's work (in weather, earthquakes, and terrorism, for instance) were weaker. One central message that really resonated with me: estimate/quantify your uncertainty around your predictions. Another: think about the "priors" and how your predictions will/should change in the light of new evidence. All in all, recommended for any practitioner of data mining or predictive analytics.
  • Rating: 4 out of 5 stars
    4/5
    Like an artist's well composed album should enrich the listening experience of a hit single track, the longer form of a book should allow prominent bloggers to present their ideas in a more complex and deeper way. Alas, in reality, both albums and books are stuffed with filler material, dumbing down the message instead of deepening it and often pandering to the weird mindset of the current crop of American Republicans. Nate Silver's book includes very good parts. Usually, these are found whenever he talks about his core areas of experience - baseball, poker and polling. When he ventures far from his topics, the errors begin to pile up. Statistics is a powerful tool in helping to solve problems. Unfortunately, not every problem is solved with a proverbial statistical hammer. The 2008 economics crisis was not, as Silver implies, a prediction problem. Goldman Sachs knew very well that it was selling toxic sludge to its customers. Rating agency bosses sent their underlings back to work on their rating models until they matched their clients' expected AAA. This was not a failure of the models used but a lack of professionalism, regulation and ethics. Peddling his statistical tools as the underutilized wonder weapons is cheap. The worst part, however, is his accommodating discussion of the views of climate change deniers. There is simply no there there. Would a late 19th century Nate Silver found pro slavery activists worthy of discussing their viewpoints? It is sad that selling books in the United States requires so much pandering (and self-censorship) in order not to upset the small-minded.Barack Obama's win was not difficult to predict - given his sad mean lying toxic joke of an opponent. Nate Silver's compilation and cleaning up of polling data was a welcome corrective to the US media misinformation and entertainment complex which insisted on a horse race despite Romney jockeying a dead horse. Silver's public service of raising awareness of the elements and boundaries of polling are both necessary and timely. He unfortunately only lightly touches upon one of my pet peeves: Statistically significant results, which should be mostly used to not talk about not significant data relationships. Obtaining a certain level of statistical significance is a question of how many resources one wants to invest to attain it. The strength of the link and its explanation are much more important aspects for discussion. The highlight of the book is the introduction to Bayesian statistics which updates and improves upon prior estimates. This part should have been extended. Overall, an average read.
  • Rating: 3 out of 5 stars
    3/5
    Like many people I know, I was entranced by Silver’s election blog, and I love him for popularizing some important facts about survey data and probability. That said, this book was a bit of a letdown. Silver is a Bayesian, and he does a decent job explaining the basic principles, but there’s not nearly enough math. While I liked finding out that Gary Kasparov basically psyched himself out against Big Blue in the chess chapter, and I learned something about why earthquakes are so hard to predict, overall I found the treatments of his various topics (also including poker, climate change, terrorism, and the stock market, among others) too superficial to be really enjoyable. And his statement that, dire warnings notwithstanding, we shouldn’t worry too much that lower Manhattan would be covered by storm surge was amusing, in a bitter sort of way—though I should hurry to point out that the occurrence of an event deemed low-probability by a model doesn’t by itself mean that the model is wrong!
  • Rating: 5 out of 5 stars
    5/5
    I'm currently re-reading this book and am a fan of Silver's FiveThirtyEight blog on the WSJ website. I was introduced to Silver's work as the Republican primaries were tracking towards a candidate. I remember running across a reference to Silver's statistical models while working for a Fantasy Sports aggregator so the name had struck a cord, when mentioned by a friend to me. As the race towards the election started to heat up, I would visit the blog daily (just as I would all of the various media sites, both conservative and liberal). What I liked about FiveThirtyEight was the reliance on math as applied to a predictive model, which enticed me to purchase this book.The book does some interesting things from the very beginning, exploring the concept of Big Data and how that affects analysis and continuing to apply those concepts to real world scenarios - including a very good description of how the financial melt-down of 2008 happened and the events that lead to that point in history. It's fairly finance heavy but still understandable - actually I think I got more from those first couple of chapters than all the other reading I did during the collapse and now via historical perspective. Kudos to Silver for doing a good job in peeling that apart and making it approachable.I especially like the main premise, which is how to look and measure relevant data while being exposed to an over-abundance of cruft, then using that data to make strong predictive models. To understand the concept, think about doing simple key-word searches on Google – the number or return-hits can be astronomical. While the first few results may be relevant, as you page through the results the number of insignificant hits decreases proportional to a score, based on how the search engine interprets the number of times your words are used on the websites or documents returned. This actually only implies relevance – the problem is that with so much data available, the user still has to decide what to use as reference. To make matters worse, the engine relies on the frequency of the words and not of the contextual relevance of the sentence built by those words. And at the end of the even-less-relevant spectrum, many sites are created to intentionally misdirect the searcher with bad information. All of this would qualify as Noise as defined by Silver (the reference is old radio, how to get through all the static to the station you want to hear clearly) – so how to get the right Signal with all that going on?Going back to the election, with so much noise to penetrate, it’s easy to see how facts can be ignored while other facts are emphasized, to support your own views and/or argument. It also explains how the losing party could have been so convinced of victory, picking and choosing only those statistics that support the party while ignoring all else as “media bias” – Silver has proven that by first applying the judicious use of data, then an algorithm that adjusts daily poling numbers based on past performance, a relatively accurate predictive model is possible to produce. I’ll end the review here so there is something for others to discover in this work.