"80% to Win": How to Read an Election Forecast

With US midterm elections on 3 November, forecasts will fill the news, and with them a familiar confusion. A candidate with an "80% chance" loses, and people declare the forecast wrong. Was it? Probability theory gives a clear answer, and it is not what most headlines imply.

What 80% means, and does not

An 80% forecast does not say a candidate will get 80% of the vote. Researchers found that many people mix up a probability of winning with a share of the vote, in an experiment described by political scientist Justin Grimmer (Grimmer, via Election Law Blog). It says that if elections like this one could be run many times, the favorite would win about eight in ten.

Twenty races, four surprises

Suppose a forecaster rates 20 independent races at 80% for the favorite in each. The expected number of upsets is 20 × 0.2 = 4. The binomial distribution calculator shows that zero upsets would happen only about 1.2% of the time, and six or more about 20% of the time. An evening with a handful of upsets is not evidence of a broken model.

An evening with none would actually be suspicious.

For 90% favorites, ten races give a 65% chance that at least one favorite loses. FiveThirtyEight made the same point about its Senate forecasts: if you rate many races as likely outcomes, you should expect to miss some of them (FiveThirtyEight).

What good calibration looks like

A forecaster is well calibrated if events given 80% happen about 80% of the time. FiveThirtyEight reported that across 5,589 events it rated at about 70%, 71% occurred, and of 55,853 events rated at about 5%, 4% happened (FiveThirtyEight). It also noted that with just ten events at 80%, chance alone could produce anything from five to ten hits.

The numbers show why. The chance of five or fewer hits out of ten at 80% is 3.3%, and the chance of all ten is 10.7%. Small samples cannot judge a forecaster. Even 72 correct calls out of 100 at 80% give p ≈ 0.06, borderline rather than damning. The one-sample proportion test does this check.

Why we cannot easily grade forecasts

Elections are rare, so evidence builds slowly. Grimmer and coauthors estimated that even in best-case scenarios, deciding which presidential forecast is better calibrated could take from 28 to 2,588 years. That does not make forecasts useless, but it suggests humility about claims that one model is clearly best.

Related tools

The Brier score grades probabilistic forecasts by squaring the gap between the stated probability and the outcome, so confident misses cost more than cautious ones. Our two-events probability calculator helps with combining races, and our piece on rain forecasts shows the same calibration logic with weather.

A caveat about independence

The binomial calculation assumes the races are independent. In reality, polling errors tend to be shared across races, so many favorites can lose together, or none at all. The number of upsets is more variable than a simple binomial suggests, which is why serious forecasters model the correlation between races.

The sensible approach: treat a forecast as a statement of odds, expect some surprises, and judge forecasters over many calls rather than one night. This article is about reading probabilities and takes no side in any election.