How did the political science forecasters fare?

TwitterredditpinterestMailDear readers: Next Wednesday morning we will be releasing a new installment in our Sabato Crystal web webinar series. It will be published on our YouTube channel UVACFP and we will also send the direct YouTube link in next Wednesday’s issue of the crystal ball. We’re going to talk about the Georgia Senate runoff, the ongoing impact of the election, the Senate map of 2022, and more.

If you have any questions that you would like us to answer during the webinar, please email us at goodpolitics@virginia.edu.

– The editors


– Before the election, several prominent political scientists predicted the election in PS: Political Science and Politics.

– Overall, the forecasts developed very well.

– However, several individual predictions missed the mark, and that choice showed the importance of questioning the assumptions of models amid an unusual choice.

Evaluation of political science forecasts for 2020

The October 2020 edition of PS: Political Science and Politics contained 10 forecasts of the national referendum and seven forecasts of the election vote of prominent political science forecasters. Some of these predictions were based on long-standing models while others were new. Some were based on national data, others on country level data. Some of the predictions were originally made a few months before the election, others much closer to the election. And some of the predictions turned out to be quite accurate, while others were far from the goal.

Table 1: Political science forecasts for the 2020 presidential election

Source: The predictions are summarized in Ruth Dassonneville and Charles Tien, “Introduction to the 2020 US Election Forecast,” PS: Political Science and Politics, Cambridge University Press, Oct.15, 2020: https://doi.org/10.1017/S104909652000147X . I’m excluding the PollyVote forecast which is based on an average of other forecasts.

Table 1 summarizes the forecasts for both the national referendum and the election vote. On average and as a group, projections came pretty close to actual election results, predicting that President Trump would receive 237 votes and 47.9% of the two-party referendum. (It now appears that Trump will actually get 232 electoral votes and nearly 47.8% of the two-party referendum.)

However, the fact that the averages of these popular and electoral predictions were fairly accurate hides the extremely large variation in the accuracy of each of the predictions.

Among the predictions of the votes, the predictions by Jerome et al., I myself and Enns & Lagodny came closest to the actual result and were missing with two, 13 and 16 votes, respectively. On the other hand, the forecasts by Murr & Lewis-Beck and Norpoth were far from the target and overestimated Trump’s total of 114 and 130 votes respectively. And Lewis-Beck & Tien’s forecast was even further off the mark in the opposite direction, underestimating Trump’s total of 164 votes. The two different Lewis Beck models managed to dramatically miss the mark in both directions.

Among the referendum predictions, it is more difficult to rate the accuracy of the forecasters because the exact final vote gap between two parties is unknown. Joe Biden’s current lead in the national referendum is four points, and this is likely to increase slightly when the final results are counted. The prognoses by Graefe and Jerome et al. were closest based on that estimate. On the other hand, two forecasters actually predicted that Trump would win the referendum – Murr & Lewis-Beck predicted a very tight Trump margin, while Lockerbie predicted a Trump margin of more than 10 points.

What caused some of the political scientists’ predictions to miss the mark so badly in these elections? In part, it may be the unusual circumstances of an election where the incumbent president was viewed by a large percentage of voters as not responsible for the severe economic downturn caused by the coronavirus pandemic. Models that incorporate traditional measures to measure economic conditions therefore overestimated the negative impact the recession had on President Trump’s support. The highly unusual nature of the 2020 economic downturn, especially the fact that it was intentionally brought about to fight the coronavirus pandemic, was the main reason I decided to change my own model and use the economy as a factor in forecasting the 2020 exclude result.

In addition, however, some models appear to be based on questionable assumptions about the behavior of American voters. For example, the Murr & Lewis-Beck model is based on the belief that citizens’ expectations of who will win an election can be used to predict the actual outcome. However, the fact that Trump was seen as a favorite to win a second term until just before the election date did not mean that voters actually intended to vote for him. It was more likely a reflection of the fact that voters generally expect an incumbent president to be re-elected.

Norpoth’s “primary model” caused one of the biggest mistakes this year. This model is based on the assumption that early primary results can provide an accurate measure of the ability of candidates to unite their party’s voters in the general election. That may be true at times, especially when an incumbent is faced with a difficult primary challenge, but this was clearly not the case in 2020. Despite his early stumbling blocks in the Democratic primary, Joe Biden was able to unite the Democratic voters behind his candidacy once he secured the nomination. Additionally, that fact was evident fairly early in the election year, as public polls consistently showed Biden as the leading Trump.

Perhaps the lesson from the 2020 election for future forecasting is that forecasting models sometimes need to take into account unusual circumstances that affect a particular election. When a model provides a prediction that is clearly inconsistent with reality, forecasters should consider changing or abandoning the model rather than duplicating it.

Alan I. Abramowitz is the Alben W. Barkley Professor of Political Science at Emory University and a senior columnist for Sabatos Crystal Ball. His latest book, The Great Alignment: Race, Party Transformation, and the Rise of Donald Trump, was published by Yale University Press in 2018.

Related Articles