“The consumers of forecasting–governments, business, and the public–don’t demand evidence of accuracy. So there is no measurement. Which means no revision. And without revision, there can be no improvement.”
For the last few years, I’ve been a participant in a study. Each week, I log in to an online platform, and try to predict the future. I make predictions about geopolitical events – from the value of the American dollar to the likelihood of drone strikes in Iran or deaths in the South China Sea – and estimate how confident I am about my prediction. I do okay. I’m proud to say, though, that the team I’m a member of outperforms professional intelligence analysts with the U.S. military, who have access to classified information.
Predicting the future is a useful skill. It’s also one we’re awful at: previous work by Tetlock found that experts were, on average, slightly worse than chance at predicting the future. Monkeys with dartboards would be better. For the last few years, the American government has been funding research into using the wisdom of crowds to predict the future. The team I’m a member of is led and organized by Tetlock and his fellow researchers, and it has been very successful – so successful that after two years, halfway through the study, the other teams were all shut down.
Superforecasting is a book about that team, and about the top members of it (I’m nowhere close, if you’re interested). It’s interesting and well written – for me the addition of Dan Gardner, who has written some great other books, has made it much more readable than Tetlock’s previous writing. In the end, it ascribes much of the success of superforecasters to a method that can be learned: a way of analyzing problems and carefully testing our beliefs and biases to try to make sure we are as accurate as possible.
A useful skill, and an interesting book. We may not all become superforecasters, but we can all learn some skills to improve our ability to analyze events and predict outcomes.
Disclosure: I read Superforecasting as an advance reader copy. You can read more reviews, and pre-order a copy on Amazon for when it comes out on September 29th, here: Superforecasting.