Reddit reviews The Signal and the Noise: Why So Many Predictions Fail-But Some Don't
We found 24 Reddit comments about The Signal and the Noise: Why So Many Predictions Fail-But Some Don't. Here are the top ones, ranked by their Reddit score.
Penguin Press
He's talking about Deep Blue's move 44 of game 1. To quote Nate Silver's excellent (and highly recommended fun read) The signal and the Noise:
> ... Deep Blue did something very strange, at least to Kasparov's eyes. On its forty-fourth turn, Deep Blue moved one of its rooks into white's first row rather into a more conventional position that would have placed Kasparov's king into check. The computer's move seemed completely pointless. At a moment when it was under assault from every direction, it had essentially passed its turn, allowing Kasparov to advance one of his pawns into black's second row, where it threatened to be promoted to a queen. Even more strangely, Deep Blue resigned the game just one turn later.
> What had the computer been thinking? Kasparov wondered. He was used to seeing Deep Blue commit strategic blunders--for example, accepting the bishop-rook exchange--in complex positions where it simply couldn't think deeply enough to recognize the implications. But this had been something different: a tactical error in a relatively simple position--exactly the sort of mistake that computers don't make.
> "How can a computer commit suicide like that?" Kasparov asked Fredric Friedel, a computer chess journalist who doubled as his friend and computer expert, when they studied the match back at the Plaza Hotel that night. There were some plausible explanations, none of which especially pleased Kasparov. Perhaps Deep Blue had indeed committed "suicide," figuring that since it was bound to lose anyway, it would rather not reveal any more to Kasparov about how it played. Or perhaps, Kasparov wondered, it was part of some kind of elaborate hustle? Maybe the programmers were sandbagging, hoping to make the hubristic Kasparov overconfident by throwing the first game.
> Kasparov did what came most naturally to him when he got anxious and began to pore through the data. With the assistance of Friedel and the computer program Fritz, he found that the conventional play--black moving its rook into the sixth column and checking white's king--wasn't such a good move for Deep Blue after all: it would ultimately lead to a checkmate for Kasparov, although it would still take more than twenty moves for him to complete it.
... As Friedel recalled:
>> Deep Blue had actually worked it all out, down to the very end and simply chosen the least obnoxious losing line. "It probably saw mates in 20 and more," said Garry, thankful that he had been on the right side of these awesome calculations.
> ...
> ... there were some bugs in Deep Blue's inventory: not many, but a few. Toward the end of my interview with him, Campbell somewhat mischievously refered to an incident that had occurred toward the end of the first game in their 1997 match with Kasparov.
> "A bug occurred in the game and it may have made Kasparov misunderstand the capabilities of Deep Blue," Campbell told me [Nate Silver]. "He didn't come up with the theory that the move that it played was a bug."
> The bug had arisen on the forty-fourth move of their first game against Kasparov; unable to select a move, the program had defaulted to a last-resort fail-safe in which it picked a play completely at random. The bug had been inconsequential, coming late in the game in a position that had already been lost; Campbell and team repaired it the next day. "We had seen it once before, in a test game earlier in 1997, and thought that it was fixed," he told me. "Unfortunately there was one case we had missed".
> In fact, the bug was anything but unfortunate for Deep Blue: it was likely what allowed the computer to beat Kasparov. In the popular recounting of Kasparov's match against Deep Blue, it was the second game in which his problems originated--when he had made the almost unprecedented error of forfeiting a position that he could probably have drawn. But what had inspired Kasparov to commit this mistake? His anxiety over Deep Blue's forty-fourth move in the first game--the move in which the computer had moved its rook for no apparent purpose. Kasparov had concluded that the counterintuitive play must be a sign of superior intelligence. He had never considered that it was simply a bug.
My thought is whoever made these graphs should read: http://www.amazon.com/The-Signal-Noise-Many-Predictions/dp/159420411X
Another study showed something similar, but from the other point of view. I believe I read it in Nate Silverman's The Signal and the Noise. It showed that people who made more money were seen as, for lack of a better word due to my own forgetfulness, more intelligent and deserving of their money. They would arbitrarily choose a subject in a group to be paid more, and that person was more likely to be listened to. I believe it also showed that a certain amount of respect is given for those who make or are perceived to make more money. I let someone borrow the book, so I don't currently have access to the passage/study.
The is no disagreement that climate change is real, there is however disagreement over the extent of climate change. If you get a chance, Nate Silver's book has a great chapter on this topic, a great book very worth reading. https://www.amazon.ca/Signal-Noise-Many-Predictions-Fail-but/dp/159420411X/ref=sr_1_1?ie=UTF8&qid=1463230626&sr=8-1&keywords=nate+silver
The truth on the extent of climate change is not fully agreed upon, and there is a lot of misinformation and agenda pushing even within the scientific community. There are cases of scientists using incorrect data to push their agenda because they felt that the agenda was more important than the science.
Predicting the extent of climate change is a messy thing. Saying climate change isn't real is clearly wrong, but it becomes problematic to say that we expect x carbon emission to cause y climate change over z time.
As usual, the devil is in the data.
I will admit to not being a seismologist either, but what I've read says that is just flat-out not true. It may very well be that the fundamental average is 140 years, and that we've had a string where they're about that long, but the next one might come in 140 years, or it might be 500 years. It's absolutely impossible to know ahead of time. Here's the technical information on a Poisson process: http://en.wikipedia.org/wiki/Poisson_process
The basic idea, as I said, is that in any given time interval the odds of a Poisson process occurring are exactly the same. The poster child of this is radioactive decay (which I actually am a specialist in). If you have a radioactive atom with a half-life of t, and you watch it for time T, there's some odds of it decaying. If it doesn't decay, and you watch it for time T again, then it has the exact same odds of decaying as the first time you watched it. In any given time window the odds are the same.
So, if earthquakes are a poisson process (and like I said I've read that they are, and while most of these links are PDFs they seem to back it up: https://www.google.com/search?q=earthquake%20poisson%20process), and we assume that 140 years is the half-life of the Hayward fault, what that means is that the odds of an earthquake happening in the next 140 years are 50%, and that's true regardless of how long it's been since the last big quake. That's what my original post meant, there is absolutely no sense in which a fault can be "due" for a quake.
If you want to read more about this, Nate Silver's book (http://www.amazon.com/The-Signal-Noise-Many-Predictions/dp/159420411X/) has a chapter about it that should be accessible to most anyone, and the rest of the book is awesome too.
Run from this project. Everyone else in this thread has already given you great reasons. I want to recommend a book, Nate Silver's Signal and Noise. It's a great read, and I think useful to any data scientist even though it's geared towards the general public.
Specifically, he introduces the reader to weaknesses in forecasting complex systems, as well as the efficient market hypothesis .
I've worked on a social media based market predictor before, and I definitely fell into many of the pitfalls Nate Silver describes.
I had to read Probability and Statistics for Engineers and Scientists for an engineering class I did way back in 2005. I learned counting theory, combinations, permutations, some crazy calculus to calculate ev and variance, and much more (hypothesis testing!?). I learned Bayes' theorem (conditional probabilities, super useful!)...
I also think everything should read Random Walk Down Wall Street. While it's primarily an investing book; it takes about markets (the poker player pool is a market) and behavioral science (overconfidence, bias, etc...).
A book I really liked was The Only Three Questions That Count: Investing by Knowing What Others Don't (also an investing book).
I got into poker a couple of year(s) after reading these books.
The math involved in ranges/equity calculations was much much much easier to learn after the knowledge I had gained in the course while the business books taught me to think about my personal errors, patterns, and try "thinking outside the box".
Recently, I picked up The Signal and the Noise: Why So Many Predictions Fail-but Some Don't. While it contains a chapter on poker (the author used to play professionally), the book made me think about how overconfidence in my game can often lead to (pretty big) failure and that I should be constantly trying to work on my ability to "make predictions" (e.g. assign a range to my opponent, picture how they will react in future streets, etc...).
I highly recommend Proofiness. It's a great book about how statistics are misused in the media, both intentionally and unknowingly.
Also, Steven Strogatz, who did a number of really well written articles about math for the New York Times a few years back compiled them with more into The Joy of X
Finally, I have Nate Silver's The Signal and the Noise right next to me, but I haven't actually started reading it so I can't vouch for it fully.
Love stuff like this and this.
Mostly Harmless Econometrics
and
The Signal and the Noise
are my recommendations for an introduction into more advanced topics in econometrics. If you want more of a textbook Th3Plot_inYou's suggestion is good (I still have mine from my class).
Edit: Signal and the Noise is more theoretical about forecasting in general.
Following :
Al FrankenVerified account
@alfranken
Al Franken, United States Senator from Minnesota
Follow
User Actions
Amy KlobucharVerified account
@amyklobuchar
U.S. Senator from Minnesota. Follows, Retweets, Replies ≠ endorsement
Follow
User Actions
Lottie Tomlinson
@lottietommo
[email protected] https://www.nailsinc.com/nail-polish/ranges/matchbox/ … snapchat / lottietommo1
Follow
User Actions
Hennepin CountyVerified account
@Hennepin
The official Twitter account for Hennepin County news and information. Tweets by staff in Communications. We're on Facebook: http://www.facebook.com/hennepin .
Follow
User Actions
MN State Patrol
@MnDPS_MSP
The official Twitter account for the Minnesota State Patrol. Like us on Facebook: https://www.facebook.com/MnStatePatrol
Follow
User Actions
MN Public Safety
@MnDPS_DPS
The official Twitter account for the Minnesota Department of Public Safety. Like us on Facebook: https://www.facebook.com/MnPublicSafety
Follow
User Actions
Governor Mark DaytonVerified account
@GovMarkDayton
Check here for updates from the Office of Governor Mark Dayton, Minnesota's 40th Governor.
Follow
User Actions
The Associated PressVerified account
@AP
News from The Associated Press, and a taste of the great journalism produced by AP members and customers. Managed 24/7 by these editors: …
Follow
User Actions
Better-Ed
@Better_Ed
Leading an Educational Renaissance A Project of Intellectual Takeout
Follow
User Actions
Gemma StylesVerified account
@GemmaAnneStyles
writing // eating // watching // sleeping ... 💌 [email protected] ... header image @demiiwhiffin
Follow
User Actions
Luke HemmingsVerified account
@Luke5SOS
Sounds Good Feels Good out now :-)
Follow
User Actions
Calum HoodVerified account
@Calum5SOS
Bassist in 5 Seconds of Summer.
Follow
User Actions
Ashton IrwinVerified account
@Ashton5SOS
// @5SOS since 2011 //
Follow
User Actions
michaelVerified account
@Michael5SOS
eh?
Follow
User Actions
LiamVerified account
@LiamPayne
the luckiest man in the world. #stripthatdown OUT NOW - http://liamp.co/stripthatdown
Follow
User Actions
Louis TomlinsonVerified account
@Louis_Tomlinson
1/4 of One Direction :) We would be nowhere without our incredible fans, we owe it all to you.
Follow
User Actions
zaynVerified account
@zaynmalik
pathetically aesthetic🍊 http://zayn.at/StillGotTimeVid
Follow
User Actions
Niall HoranVerified account
@NiallOfficial
My new single 'Slow Hands' is out now http://niall.to/SlowHandsTw
Follow
User Actions
Harry Styles.Verified account
@Harry_Styles
http://hstyles.co.uk/music
Follow
User Actions
mspmagVerified account
@mspmag
The Best of the Twin Cities. No, really. We got this.
Follow
User Actions
jimmy fallonVerified account
@jimmyfallon
astrophysicist
Follow
User Actions
Barack ObamaVerified account
@BarackObama
Dad, husband, President, citizen.
Follow
User Actions
Star TribuneVerified account
@StarTribune
Minnesota's top choice for news, sports and more. Follow our journalists at http://strib.mn/STtweets . For customer service, contact @startribsupport
Follow
User Actions
MinnPost
@MinnPost
Independent, member-supported journalism for Minnesota. Subscribe to MinnPost's e-mail newsletters: http://minnpo.st/ZPOiWb
Follow
User Actions
The CurrentVerified account
@TheCurrent
The Current, a non-commercial radio station in Minnesota, plays the best new music next to the music you already love, from local to legendary. 89.3…
Follow
User Actions
The AtlanticVerified account
@TheAtlantic
Politics, culture, business, science, technology, health, education, global affairs, more. Tweets by @CaitlinFrazier
Follow
User Actions
Ezra KleinVerified account
@ezraklein
Founder, http://Vox.com . Come work with us! http://bit.ly/1ToAmQ8
Follow
User Actions
Curiosity RoverVerified account
@MarsCuriosity
NASA's latest mission to explore the surface of Mars. Roving the Red Planet since Aug. 5, 2012 (PDT) (Aug 6 UTC).
Follow
User Actions
Star Tribune A&E
@entertain_mn
Everything you need to know about the Twin Cities entertainment scene – restaurants, music, art, movies and more, plus way too many tweets…
Follow
User Actions
KARE 11Verified account
@kare11
KARE 11 News, http://kare11.com and http://m.kare11.com are the places to go for all your news, weather, sports and traffic. Part of #teamTEGNA
Follow
User Actions
First AvenueVerified account
@FirstAvenue
The legendary concert venue in Minneapolis, MN. Your Downtown Danceteria Since 1970.
Follow
User Actions
MPR NewsVerified account
@MPRnews
Breaking News, and award-winning coverage from Minnesota Public Radio News. Tweets by @publicmic @n_yang @sarasporter @codyleenelson
Follow
User Actions
FiveThirtyEightVerified account
@FiveThirtyEight
The home of Nate Silver's FiveThirtyEight on Twitter. Politics, Economics, Science, Life, Sports.
Follow
User Actions
childish gambinoVerified account
@donaldglover
Follow
User Actions
R.T. Rybak
@R_T_Rybak
Not neutral about Minnesota;@Twins; Minneapolis beer-bikes-lakes-music-art; Obama; better schools n ESP: my family. (Personal acct doesn't reflect my…
Follow
User Actions
Johnny ManzielVerified account
@JManziel2
Gig 'Em
Follow
User Actions
Ellen DeGeneresVerified account
@TheEllenShow
Comedian, talk show host and ice road trucker. My tweets are real, and they’re spectacular. http://www.ellentv.com
Follow
User Actions
The Daily ShowVerified account
@TheDailyShow
Trevor Noah & The Best F#@king News Team. Weeknights 11/10c @ComedyCentral. Full episodes, videos, guest info. #DailyShow Tickets: …
Follow
User Actions
Minnesota TwinsVerified account
@Twins
The Official Twitter feed of the Minnesota Twins. Snapchat: Twins
Follow
User Actions
Nate SilverVerified account
@NateSilver538
Editor-in-Chief, @FiveThirtyEight. Author, The Signal and the Noise (http://amzn.to/QdyFYV ). Sports/politics/food geek.
Follow
User Actions
John OliverVerified account
@iamjohnoliver
Comedian. @LastWeekTonight, @TheDailyShow, The Bugle Podcast (@hellobuglers)
Follow
User Actions
Anna KendrickVerified account
@AnnaKendrick47
Pale, awkward and very very small. Form an orderly queue, gents.
Follow
User Actions
City of MinneapolisVerified account
@CityMinneapolis
Official news and information from the City of Minneapolis. For non-emergency requests, call 311.
Follow
User Actions
chris prattVerified account
@prattprattpratt
laughing with you, not at you.
Following
User Actions
Modern Seinfeld
@SeinfeldToday
What if Seinfeld were still on the air? (Tweets by @JackPMoore and @JoshGondelman.)
Following
User Actions
Neil deGrasse TysonVerified account
@neiltyson
Astrophysicist
Followers:
Politifact
@POLITIFvCT
completely fake political ratings just like @politifact #SethRich #MAGA
Follow
User Actions
Pek Alt Right
@PekAltR
Follow
User Actions
Beauregard
@beauzeph
I like technology politics philosophy. I politically identify as an agonist. If you want to collaborate funny bots or disruptive apps, let's chat..
Follow
User Actions
Kirk White
@kirkwhitetw
Following the revolution across the West. #MakeTheWorldGreatAgain
Follow
User Actions
Gigi Heath
@HeathGigi
passion#Proud alcohol nerd , Food maven ...
Follow
User Actions
The Books Story
@_BooksStory
Gathering and curating disparate pieces of content which together tell the #books story seen through my eyes.
Follow
User Actions
Kayla Trimble
@Love_laylapoo1
'18 🎓
Follow
User Actions
.
@OUTCASTIRWINS
Follow
User Actions
not active
@SahotaAneesha
Follow
User Actions
LOL ZOEY
@AishaJosaphat
Most are liars; There are ways to predict future player behavior, though. NateSilver is legendary for exploiting this ability, and he's published a guidebook on it: http://www.amazon.com/The-Signal-Noise-Many-Predictions/dp/159420411X
More generally, predictions hinge on other skills. You'll want to learn [[statistics]] and [[probability]], followed by domain-specific research into what exactly you want to predict (for predicting other players, you'll want [[psychology]], for example).
Nonfiction:
On the fiction front, I potentially want to read Where'd You Go, Bernadette? by Maria Semple. The thing is... with new fiction, I tend to wait for friends/colleagues to read the novel first.
Somebody read: The Signal and the Noise: Why So Many Predictions Fail - but Some Don't
Nate Silver has a book addressing this exactly.
http://www.amazon.com/gp/aw/d/159420411X/ref=redir_mdp_mobile
I am not Nate, just a fan of his work.
Generally speaking, it is from Bayesian probability
For more, Nate Silver's The Signal and the Noise is a good primer
Are you thinking of The Signal and The Noise?
It would be interesting. That reminds me of Nate Silver. He is a statistician that has done everything from baseball stats to presidential election stats.
His presidential election stats were very accurate, and they used a system you are describing. They took in all the predictions, then weighted those predictions based on prior accuracy of the predictor. Very interesting stuff described in his book The Signal and the Noise. I'm reading it now, and it is fascinating.
Finally, something I can answer on this sub!
I got a degree in cell biology, did the lab jobs, and didn't like it. I'm working on applying to post grad degrees in statistics and CS now. There are a lot of specialized data science degrees out there too.
Here's a book on broad ideas of data to get you started.
If you do happen to like dealing with big data, you must learn programming to get work at all. There's a lot of online programs built for this. I'm doing Udacity' data analyst nanodegree. I'd say your biology degree is mostly useless when applying for these jobs. Entry level into data science is mostly going to be about data wrangling which is pretty much all programming. You'll want to brush up on your statistics class, Python, SQL, and R.
The Signal and the Noise is a very light read as math(s) books go, but it's definitely interesting!
You need to know some programming to develop a model, if you learn web scraping that will be enough to gather data for a model. You should be able to learn how to do this online.
For books, I'd highly recommend reading these:
Fortune's Forumula - A great book about the Kelly Criterion but touches on a whole range of subjects, a fantastic read.
The Signal and the Noise - Very famous book about prediction in general.
Conquering Risk - Very good book about sports betting (relatively unknown)
Calculated Bets - About creating a model and automated betting system for a relatively unknown sport.
Who's #1? - A book about rankings systems, aimed at ranking sports teams but the authors previously wrote a book on ranking websites (like google search algorithm type stuff). The basis for my dota model came from this book.
I'd recommend everyone to read Fortune's Formula and The Signal and the Noise, even people not interested in modelling. They're both awesome reads.
Calculated bets is a pretty cool read if you're interested in modelling, the author has a really quirky writing style that's entertaining.
Conquering risk is basically about exploiting bookmakers, picking off mistakes. Not really about modelling but still pretty cool.
Who's #1 is a really good intro to making a model for predicting sports imo, there's some very simple ones that will get you started.
I thought it was general knowledge that the climate model forecasts have overestimated the rise in temperature. Nate Silver had some interesting things to say about it in his book.
What purpose is the figure with the smoothed data points supposed to serve other than to mislead?
In Poker, each player at the table is playing against all the other players, rather than against the dealer. The house takes a cut of each pot instead of fielding a hand. Players in Poker aren't subject to the standardized behavior rules that a dealer in Blackjack is, and each player has significantly more hidden information in Poker than in Blackjack which means significantly more possibilities to account for in each hand for a "skilled" player.
It also means that more of Poker is built around reading other players, trying to divine what hand they might have based on their betting behavior—and based on their prior performance, whether or not they're deliberately trying to throw people by betting like they have a better or worse hand than they actually have.
Also, decks in Poker are shuffled after each hand, which makes Blackjack-style counting impossible.
There ARE people who make a living playing Poker, but it's not easy. Nate Silver talks about his experience as a professional Poker player in his book, The Signal and the Noise. One of the big takeaways is that the long run is even longer in Poker than it is in Blackjack—even a talented pro who consistently plays well and ultimately has a profitable career can have years where they lose tens of thousands of dollars.
you can know your shit too!
http://www.amazon.com/The-Signal-Noise-Many-Predictions/dp/159420411X