Clay Graubard: So Andrew, we're coming up to the end of 2020 and we've talked about making a list of the top five books that we've read this year and I have a pretty good idea what's going to be at the top of the list. Any ideas on what that book might be?
Andrew Eaddy: Well Clay, I think I have an idea.
Clay Graubard: Thanks for sharing the idea.
Andrew Eaddy: …
Clay Graubard: …
Clay Graubard: Of course we are talking about Philip Tetlock and Dan Gardner’s book Superforecasting which came out in 2015.
In a nutshell Superforecasting posits that humans are able to make predictions about geopolitical events and, surprise surprise, are actually pretty damn good at it!
A lot of the time, it has been thought that statistics sort of replaced the human brain in making predictions and that there should be this blind trust in data. Superforecasting doesn't reject that premise, but what it says is that at the end of the day you can have all the statistics at your disposal but statistics are meant to be used as a tool: A means, not an end. The human brain is capable of breaking down extremely complex problems into these bite-size pieces that can be critically thought about and solved more granularity, whereas taken at the whole they're nearly impossible to answer.
What was most shocking about Tetlock’s Superforecasting isn’t that superforecasting works, but that of course it works.
Andrew Eaddy: Yeah I agree, I think it's a book that we both read before we even started Global Guessing and we both sort of fell in love with the book’s thesis and the way Tetlock structures the actual content in the book. It was also just really cool learning about this new area of statistics and analysis that we hadn't really explored before.
And you are right, it does feel kind of intuitive. What the book succeeds in doing is putting the science back into predictions that otherwise were devoid of it, which like sounds weird but when you consider that there are so many pundits on the news that will make predictions about the stock market and then those events either will or not transpire but those pundits aren't held to whether they were right or wrong, you realize just how unscientific the field can really be. And then you start to lose track of who's really trustworthy and what predictions you can have confidence in. Tetlock’s focus on trial and error, the scientific method, is crucial.
Clay Graubard: I also think what Tetlock introduces is the idea that the rules are there for a reason...but don’t be afraid to break them if you feel like they should.
One thing that comes to mind for me is thinking about FiveThirtyEight’s 2020 election forecast where on the eve of the election they gave Biden an 89% chance of winning, Trump a 10% chance. However, in retrospect, that prediction seems wrong. Was the methodology sound and was it the best-of-the-bunch? Of course! But Nate Silver found himself tied to his model. But what if he wasn’t?
What I’m thinking about Andrew, is a news story from Politico published on October 29, about how Republican turnout was over-performing both the 2020 Democrats and the 2016 Republican numbers in Miami. In a model like Silver’s, there is no place for such a report. However, in a superforecasting approach, one could consider the report to have a material impact and adjust accordingly.
Because I have no life, I did using FiveThirtyEight's "Choose Your Own Adventure" tool to help. And while it was crude and certainly subject to improvement and critique, it produced the result of: Biden with a 75.50% chance of winning; Trump, 23.25%. Odds that, based on the fact the election was within 70,000 votes of going in Trump’s favor and the polls were quite flawed, were probably more accurate than just the pure statistical analysis.
Andrew Eaddy: I’m sorry professor, but I thought we were having an accessible conversation...
Clay Graubard: That was the accessible explanation!
Andrew Eaddy: Okay. Moving on…
I think my final take away from Superforecasting is that it introduces new levels of quantitative analysis to qualitative data points and the forecasting process at large, which can in turn make a forecast more accurate.
Think of a machine learning program. Like most automated processes, its ‘trash in, trash out’. The machine learning program will only run well if you feed it the right information, and enough information. What superforecasting does is it's able to feed more information into a given model, which is more correct because the model isn’t omitting relevant data points, which in turn makes the outcome more reliable. What we’re describing is a ‘supervised learning’ environment, so not exactly deep learning, but an evolved approach to prediction-making.
And this doesn’t even include the difficult task of quantifying qualitative measures that we use when considering political events. Operationalizing those variables would take time, and a deep understanding of the existing theory. This is something that models like Nate Silver has a very tough time incorporating. But we won’t get into that right now.
Clay Graubard: Right. And unlike Silver, superforecasting says, "let's find the best tools and get to work." I like it.
Andrew Eaddy: Well you should, we did just put it at the top of our list.
2. The Better Angels of Our Nature
Clay Graubard: Moving ahead to the next book, Andrew, a fair warning. I could probably talk about this next book for hours on end, so I’ll do my best to keep it short.
The Better Angels of Our Nature by Harvard psychologist Steven Pinker is my favorite book of all time so you should probably read it too (although Brandon Sanderson’s Stormlight Archives series is a close contender).
The thesis of Pinker’s Better Angels is that indicators of societal disease–war, homicide, torture, capital punishment, slavery, racism, sexism, poverty, animal abuse, and more–have abated over time, particularly since the late seventeenth and early eighteenth centuries, and have done so because of gentle commerce, democracy, inter-governmental organizations, Enlightenment humanism, technological developments such as the printing press, a transformation of social systems, and a massive increase in IQ scores over the past 100 years.
Pinker backs his argument up with material and psychological evidence that is expansive, diverse, and sound, supplying at least a hundred pages of graphs and statistics, and many-more on theories, studies, and history (we’ll save the controversies and my rebuttals to those for another day). And although the book is long, it–along with his follow-up Enlightenment Now–provide a strong empirical, psychological, philosophical, and logical foundation to think and reason about the world and what has and has not made life better.
Andrew Eaddy: And you are recommending this book at the end of 2020? Have you even seen the news?
Clay Graubard: Haha. Very funny.
I’d actually argue that after the year we’ve all experienced, Pinker’s book is exactly the one that people should read. Pinker often gets described as an optimist–that things have gotten better so they will continue to do so. I don’t know who these people are, but they definitely haven’t read his book.
Pinker’s book does not say “hey, the issues of 2020 and the issues we are facing right now–whether they be climate change, growing income inequality, or persistent racial inequalit–are not serious and important. Because they are. But Pinker’s book is important for two reasons: 1) It provides context on the world we live in versus the world of past, and shows how rapid and transformative progress can be; and 2) It provides a roadmap for how we solve the issues of today.
Andrew Eaddy: But many would argue that things have been getting worse: Global tensions are rising; life-expectancy in the US has been decreasing; etc. etc. What does that say about Pinker’s argument.
Clay Graubard: Firstly, it is important to note that many of these developments have unfolded in too short a time-frame so they could either be a temporary dip or an indication of a long-term downward trend. It’s too soon to say. However, what I will also say is that if anything, those developments make Pinker’s argument more salient.
In the last 5 years, has democracy been cherished or pushed aside? Has gentle commerce continued or has protectionist trade re-emerged? Are countries more or less engaged with intergovernmental organizations? You can see where I’m heading with this...
Basically, we’ve turned away from the human institutions and systems that Pinker found to make life better. And now it got worse. So what does it say about Pinker’s argument?
Andrew Eaddy: Point taken.
3. The Signal and the Noise
Andrew Eaddy: So, the next book on my list was Nate Silver’s (of FiveThirtyEight) The Signal and The Noise.
Although The Signal and the Noise came out in 2012, it’s a book that I thought was and still is very important. First of all because in the world of forecasting and predictions, Nate Silver is the guy. He’s been doing this work for a long time, and doing a pretty-good job of it, so I thought reading his book would be valuable for the work we are doing at Global Guessing. But I also chose the book because forecasting is becoming a larger part of our world, and understanding the science behind predictions and how to be responsible and smart about predictions is really important.
Clay Graubard: So what does Silver talk about in his book?
Andrew Eaddy: The book is a part biography–walking you through Silver’s journey from being a baseball analyst to running FiveThirtyEight–part history and examination of predictions: How they’ve evolved over time, the mistakes people make when predicting, and how we can improve.
Clay Graubard: And what did you take from the book that was relevant to what we do at Global Guessing?
Andrew Eaddy: I think the biggest takeaway is actually in the title: the ‘signal’ and the ‘noise’.
Clay Graubard: Wait, you’re telling me the most valuable thing you took away was the title? Why even read the book?!
Andrew Eaddy: Okay, I deserved that after my Pinker question.
What I was going to say is that, when you're trying to predict political events, like we do at Global Guessing, it can be really easy to mistake noise for signals, aka mistaking something trivial as something important. With any situation you want to predict, there are so many variables, so many different aspects of problems that you could factor in when considering an outcome. Separating what is important (the signal) from the unimportant (the noise) is critical in trying to understand not what to think, but how to think and how to analyze certain problems.
Clay Graubard: So Nate Silver is most well-known as a statistician. After reading Superforecasting do you feel like he should have incorporated more of Tetlock’s philosophy into his own forecasts for the 2020 election like we discussed earlier?
Andrew Eaddy: At first glance I think these books seem to have friction. But I think the books sort of go hand-in-hand. I think Nate Silver does agree that the stage that forecasting is at right now is not adequate and that there's room to grow. It's tough when you're in the media space where you want to say something to but you don’t want to be wishy-washy either.
Clay Graubard: Two questions: 1) Would you say that in some ways Nate Silver is the more conservative of the two approaches? In that he wants to only include things that you know for certain, the signals, versus the superforecasting approach where you can include more uncertainty into your predictions and then just lower the confidence levels that you have in them so it's a more liberal approach to predictions.
And I’m wondering if superforecasting the future and Nate Silver the vanguard of the past but we exist in a period of transition. Bear with me for this analogy, but a comparison might be that Nate Silver is Johann Sebastian Bach, while Tetlock and superforecasting is Carl Philipp Emanuel Bach. J.S. Bach was the best of the baroque, a style that had been dormant for decades. But J.S. Bach wrote during a period of transformation into the classical style which his son C.P.E. Bach was a pioneer of a new classical style, even if it was not fully formed and Mozart was still years away.
So, to bring the analogy together and ask my second question, 2) in the future do you see the new (superforecasting) replacing the old (Silver-esque predictions) like classical replaced baroque?
Andrew Eaddy: I think that's a nice way of framing it. I think Nate Silver and FiveThirtyEight are hindered by certain things that you know Philip Tetlock and the Good Judgement Project are not. Nate Silver has sat under the New York Times, ESPN, these big companies where there's less room to experiment. By contrast I think a lot of the superforecasters that you'll read about in Superforecasting, they're much more able to trial and error. The stakes are lower. So I think Nate, in his head, is probably where Tetlock is, but in terms of what he’s actually able to produce, it's going to be maybe a couple steps behind.
As for whether or not superforecasting will overtake the more traditional methods being used, I think we will ultimately end up with a blended approach to predictions which gives credence to the methodologies promoted by superforecasting while applying them to traditional methods that have existed for some time.
4. After Tamerlane
Clay Graubard: So for my final book choice I chose John Darwin’s After Tamerlane, which covers just a small period of time: From the death of Taimur Tamerlane in 1405 until the year 2000.
The primary focus of Darwin’s Tamerlane is the rise and fall of the European colonial, Islamic (Ottoman, Iranian, Mughal), and Chinese Empires. What makes After Tamerlane such an important book is that it manages to contextualize the broad strokes of the last 600 years in 600 pages, and at the very least I walked away with a new perspective on the historical trends of empires.
For instance, up until the start of the 19th century, all three of those empire groups were relatively equal in power and influence. If anything, the European powers were laggard. It was not until the 19th century when there was a massive explosion in the disparities of power between empire groups, with Europe pulling far ahead. But from these disparities came the colonial domination of Africa and Asia, the collapse of the Islamic empires, and denial of Great Power status for China until the latter-half of the Cold War despite being a dominant power for most of human history.
Andrew Eaddy: As somebody that works in investment banking and also in the science of predictions, it's always nice to be able to zoom out and look at patterns and trends. Oftentimes historical trends can be strong indicators of future performance–not always, of course–but I think especially with something as significant and with as large of a sample size as the grouping of Empires Darwin references throughout his book, I think historical findings can be very telling.
I also think it's super relevant today when you look at the discussion people are having about the United States. In the same way that China was dominant until the industrial revolution arrived and they were unable to make the jump at the same pace as the European empires, you have many people saying that the United States is falling behind. I recently read A New Foreign Policy by Jeffrey Sachs, and he talks about a lot of ways the US is falling behind, and I was wondering what you learned from Tamerlane that might shed light on where the US is right now?
Clay Graubard: Although I would push-back on any definitive claims on where the US is headed, I would also say that Tamerlane is a cautionary tale for how rapidly fates can change.
As you were hinting at, one might consider the myriad of challenges the United States faces internally and externally to be similar to the ones China faced with industrialization, but instead with informationization. If you had only examined the Chinese experience of the 19th century, it would not be enough to explain the power disparity that emerged between China and Europe. What made the power disparity so large was that while China stalled (and took a few steps backwards), Europe underwent massive forward progress and development. And now we find ourselves in a period where the United States has been consumed by gridlock for years, is struggling to adapt to the information age, and is likely leaving the coronavirus pandemic with a GDP contraction in comparison to China’s GDP growth. What does that mean for the future? Nothing definitive, but viewed from the lens of After Tamerlane, well….happy thoughts. Right?
Andrew Eaddy: So I read Siddhartha by Hermann Hesse pretty early into quarantine. It’s a short book, about 152 pages–so if the 600+ page options are too much for you, this is the book for you!
Hesse's Siddhartha is about a child in India named Siddhartha who is really well-off with a really great life, but decides to leave his life of comfort because he's become troubled by deep questions of wisdom, death, and the purpose of life itself. He commits to an ascetic life, bound most-centrally by the five precepts of Buddhism, and sets off to find the answers he so desperately seeks. The book follows Siddhartha as he travels the world trying to find the answers to these pressing questions.
I think the book is one you can read and just enjoy as an amazing piece of literature, but I think it also has a lot of really interesting lessons that relate to the work that we’re doing at Global Guessing.
Clay Graubard: Right. So that plot sounds fascinating, don’t get me wrong, but...uh...it doesn’t seem all that related to geopolitical risk or forecasting. What’s the deal here? We getting book royalties or something?
Andrew Eaddy: -_-
OK. He maybe doesn’t use the same nomenclature as us, sure, but his goals in the novel are very similar to ours. On his journey, Siddartha tries to get rid of distractions such as friends, family, and material goods in pursuit of the truth. That’s similar to Nate Silver’s description of the signal and the noise. I also think in the same way that his pursuit for the truth is viewed as noble, so is the work that we, Nate Silver and Philip Tetlock, we’re all striving for.
Clay Graubard: That makes sense (except you better not get rid of me!). We’re ultimately trying to strip away everything that’s trivial because it keeps us from seeing what’s truly important. That seems like a worthwhile journey to embark on.
It seems like a lot of people went on their own mini Siddhartha journey over the coronavirus pandemic - really trying to understand what was frivolous in life and what was essential. That’s definitely a book that I should pick up and read. It seems like a good book for the mind.
A question for you: Do you think the novel in some way increased your perceptibility? Did your interest in what superforecasting change after you read Siddhartha, since you read it before Superforecasting?
Andrew Eaddy: Definitely. I think one of the biggest things that I got from Siddhartha that I think is super relevant to what we're doing is this sort of Emersonian-emphasis on self-realization.
When you’re doing something like what we’re doing at Global Guessing it can be a bit nerve-wracking, you know. We're really putting ourselves out there and from our very first article risking our reputation or ego basically. We have to rely on our own skills, confidence, and ability.
I think something that Siddhartha preaches that I really took to heart is just that you have to trust in yourself because that’s where the real truth lies.