I hadn’t come across Tetlock’s research on predictions.
He carried out a 20+ year project with people who made a living from predicting the future. He asked them to predict outcomes and the likelihood of outcomes. The “experts” were not successful:
We found that our experts’ predictions barely beat random guesses – the statistical equivalent of a dart-throwing chimp – and proved no better than predictions of reasonably well-read nonexperts. Ironically, the more famous the expert, the less accurate his or her predictions tended to be.
Worth reading the longer interview for his Yale vs rat story. PS I love the dart-throwing chimp, and want to get it into a conversation about something else.
Both pieces cover his distinction between hedghogs and foxes.
You know the famous line that [philosopher] Isaiah Berlin borrowed from a Greek poet, “The fox knows many things, but the hedgehog knows one big thing”? The better forecasters were like Berlin’s foxes: self-critical, eclectic thinkers who were willing to update their beliefs when faced with contrary evidence, were doubtful of grand schemes and were rather modest about their predictive ability. The less successful forecasters were like hedgehogs: They tended to have one big, beautiful idea that they loved to stretch, sometimes to the breaking point. They tended to be articulate and very persuasive as to why their idea explained everything. The media often love hedgehogs.
The New Yorker article goes into this in more depth.
Finally, you can hear some contemporary thinkers (experts?) discussing his ideas on Radio 4.