Is it possible to predict the future of the war in Ukraine? Online forecasting communities think so

The day after Russia invaded Ukraine, someone wrote a message of gratitude on an internet forum. “Just want to say that I moved from Kyiv to Lviv on Feb 13 /entirely/ thanks to this prediction thread and the Metaculus estimates. (Still in Lviv but leaving Ukraine later today.) [...] Thank you, everyone.”

The message was left by a user called “availablegreen” on Metaculus, a community that is dedicated to forecasting the future by asking questions such as the one that prompted this user to leave his city. Will Russia invade Ukraine before 2023? In December, the forecasting community said that there was a 40% chance. And on February 13, when availablegreen left Kyiv, there was a 60% chance. At the time, the US intelligence services were already saying what was going to happen. Many people didn’t believe it, but availablegreen decided to go ahead anyway.

Prediction platforms are currently experiencing a boom. On Metaculus, Polymarket, Good Judgment and Insight, questions are asked about everything. About politics, for example: Will Emmanuel Macron win the French elections? Very probable (94%). About the pandemic: Will the World Health Organization add a new Covid variant “of concern” to the list in 2022? Probable (74%). Or about catastrophes: What risk is there for an alert citizen in London of dying next month due to a nuclear explosion? Some 24 micro deaths, 24 options in a million, according to the group of renowned forecasters of which Nuño Sempere, from Madrid, is a part.

Sempere, who writes a newsletter about these issues, explained how these platforms work. “Metaculus is a group of people who think that these questions are important, and that having models of the world that are able to make predictions is important. Imagine a collaborative community like Wikipedia or Reddit, which, rather than writing articles or selecting interesting content, generate investigations and a probability that sums them up.”

These predictions were popularized by three teachers from Pennsylvania, Philip Tetlock, Barbara Mellers and Don Moore, who in 2013 won a competition financed by US intelligence services. They showed that some people are better than others at doing this, whom they called “superforecasters,” and that by aggregating their predictions, they could equal or beat the successes of CIA experts.

But the community that now exists goes even further than that.

How do they do it?

Forecasters use open sources. They exploit the information that is on the internet, from scientific studies and press releases, to public data. As Sam Freedman said: “Anyone on Twitter can, if they filter information well, be better informed about the real-time course of the war than Eisenhower was about Korea or LBJ was about Vietnam.”

What’s more, they know the recipe for better forecasts. First, a certain kind of viewpoint is needed: quantitative, probabilistic, parsimonious, prepared to change their opinion. Second, the use of aggregation methods are better than a median one (for example, if two people with different information tell you that the likelihood of rain is 50%, you should bet that it will rain with more than 50% probability). And third, logically, but often forgotten: you have to have the real desire to get it right.

How much do they get right?

Within the community, they are not very satisfied with their successes during the war in Ukraine so far, although, in my opinion, the fact that they said in January that the invasion was somewhat probable has its merits.

I have also followed another forecast of theirs that began as a failure, but soon shifted. It was useful for me for organizing my journalistic coverage: Would Kyiv fall under Russian control before April 1? The second day of the invasion, in Metaculus they thought it was likely to fall (80%), as did the majority of observers, who were expecting a rapid advance of the Russians. But they soon corrected that view.

By the fifth day the likelihood of Kyiv falling under Russian control was down to 67%, on March 6 it was 37%, and on March 15, two weeks before the deadline, it was just 10%. What was even more interesting is that now they believe that the probability that the capital will resist another two months, until June 1, is 80 to 85%. Will they be right?

One of Metaculus’s successes has been its predictions during the Covid pandemic, as Juan Cambeiro explained from New York. I met him when he was leading the ranking of the best forecasters, and he now works for the platform. “Around December 2, Metaculus forecasters successfully predicted that omicron would quickly overtake delta.” And also that it had an “intrinsically greater transmissibility, which would erode the protection provided by vaccines and even that it would be less lethal than delta.”

Of course, the people from Metaculus are neither an oracle nor are they infallible. They almost never offer absolute predictions. But they have shown themselves to be correct in one key sense: they are very well-calibrated. The events that they class as having a 60% probability occur 60% of the time (more or less), and those with a 90% probability occur 90% of the time (more or less).

Why do they do it?

There are platforms where you can forecast to earn money or cryptocurrency, but the main motives, for now, appear to be as a hobby and out of dedication. If the internet has taught us anything, it’s that people can dedicate a lot of time to their interests.

For Sempere it’s not a game. “Sports betting is ridiculous to me and I can’t see the fun,” he says. “They are structured to create addiction.” He also forecasts in competitions and paid markets, but he finds advantages and disadvantages there. “They allow you to invest more effort,” he says. “But collaborating is more difficult.”

The key is that both Sempere and Cambeiro believe that their work is useful and has a lot of potential. All of us have to take quick decisions, often amid uncertainty. This is obvious for a mayor or an executive, but it is the same for the man who closed his bar during the pandemic or for the youngster who opts not to buy a house out of fear of a recession. These platforms, as Sempere says, can produce probabilities about “how much a quarantine will last, or who will be the next president.” They are not going to decide for you, but they can shed light on and inform your decisions. Cambeiro points out that this is already happening. “Many people have taken decisions about Covid based on our decisions,” he explains. “Many users and I were taking precautions before anyone else was.”

Could this be the case for availablegreen? I can’t guarantee that it is true, but I have spoken to him and it does indeed seem to be. He’s a young Belarusian who lived in Kyiv and in February, when he was already concerned, remembered that he had read about these platforms (“I went to see what the prediction markets were saying”). Trusting in them, and in what he could see in The New York Times and on Twitter, he took a decision: to abandon the city to go and live in Lviv, from where he later traveled to Warsaw.