Repository logo
 

How well did experts and laypeople forecast the size of the COVID-19 pandemic?

Published version
Peer-reviewed

Change log

Authors

Freeman, Alexandra L. J.  ORCID logo  https://orcid.org/0000-0002-4115-161X
Spiegelhalter, David 

Abstract

Throughout the COVID-19 pandemic, social and traditional media have disseminated predictions from experts and nonexperts about its expected magnitude. How accurate were the predictions of ‘experts’—individuals holding occupations or roles in subject-relevant fields, such as epidemiologists and statisticians—compared with those of the public? We conducted a survey in April 2020 of 140 UK experts and 2,086 UK laypersons; all were asked to make four quantitative predictions about the impact of COVID-19 by 31 Dec 2020. In addition to soliciting point estimates, we asked participants for lower and higher bounds of a range that they felt had a 75% chance of containing the true answer. Experts exhibited greater accuracy and calibration than laypersons, even when restricting the comparison to a subset of laypersons who scored in the top quartile on a numeracy test. Even so, experts substantially underestimated the ultimate extent of the pandemic, and the mean number of predictions for which the expert intervals contained the actual outcome was only 1.8 (out of 4), suggesting that experts should consider broadening the range of scenarios they consider plausible. Predictions of the public were even more inaccurate and poorly calibrated, suggesting that an important role remains for expert predictions as long as experts acknowledge their uncertainty.

Description

Funder: Winton Centre for Risk and Evidence Communication

Keywords

Research Article, Medicine and health sciences, Research and analysis methods, Physical sciences, Biology and life sciences, Social sciences

Journal Title

PLOS ONE

Conference Name

Journal ISSN

1932-6203

Volume Title

16

Publisher

Public Library of Science