TY - JOUR
T1 - Comparing human and model-based forecasts of COVID-19 in Germany and Poland
AU - Centre for the Mathematical Modelling of Infectious Diseases COVID-19 working group
AU - Bosse, Nikos I.
AU - Abbott, Sam
AU - Bracher, Johannes
AU - Hain, Habakuk
AU - Quilty, Billy J.
AU - Jit, Mark
AU - van Leeuwen, Edwin
AU - Cori, Anne
AU - Funk, Sebastian
N1 - Publisher Copyright:
© 2022 Bosse et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
PY - 2022/9
Y1 - 2022/9
N2 - Forecasts based on epidemiological modelling have played an important role in shaping public policy throughout the COVID-19 pandemic. This modelling combines knowledge about infectious disease dynamics with the subjective opinion of the researcher who develops and refines the model and often also adjusts model outputs. Developing a forecast model is difficult, resource- and time-consuming. It is therefore worth asking what modelling is able to add beyond the subjective opinion of the researcher alone. To investigate this, we analysed different real-time forecasts of cases of and deaths from COVID-19 in Germany and Poland over a 1-4 week horizon submitted to the German and Polish Forecast Hub. We compared crowd forecasts elicited from researchers and volunteers, against a) forecasts from two semi-mechanistic models based on common epidemiological assumptions and b) the ensemble of all other models submitted to the Forecast Hub. We found crowd forecasts, despite being overconfident, to outperform all other methods across all forecast horizons when forecasting cases (weighted interval score relative to the Hub ensemble 2 weeks ahead: 0.89). Forecasts based on computational models performed comparably better when predicting deaths (rel. WIS 1.26), suggesting that epidemiological modelling and human judgement can complement each other in important ways.
AB - Forecasts based on epidemiological modelling have played an important role in shaping public policy throughout the COVID-19 pandemic. This modelling combines knowledge about infectious disease dynamics with the subjective opinion of the researcher who develops and refines the model and often also adjusts model outputs. Developing a forecast model is difficult, resource- and time-consuming. It is therefore worth asking what modelling is able to add beyond the subjective opinion of the researcher alone. To investigate this, we analysed different real-time forecasts of cases of and deaths from COVID-19 in Germany and Poland over a 1-4 week horizon submitted to the German and Polish Forecast Hub. We compared crowd forecasts elicited from researchers and volunteers, against a) forecasts from two semi-mechanistic models based on common epidemiological assumptions and b) the ensemble of all other models submitted to the Forecast Hub. We found crowd forecasts, despite being overconfident, to outperform all other methods across all forecast horizons when forecasting cases (weighted interval score relative to the Hub ensemble 2 weeks ahead: 0.89). Forecasts based on computational models performed comparably better when predicting deaths (rel. WIS 1.26), suggesting that epidemiological modelling and human judgement can complement each other in important ways.
UR - http://www.scopus.com/inward/record.url?scp=85139572150&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85139572150&partnerID=8YFLogxK
U2 - 10.1371/JOURNAL.PCBI.1010405
DO - 10.1371/JOURNAL.PCBI.1010405
M3 - Article
C2 - 36121848
AN - SCOPUS:85139572150
SN - 1553-734X
VL - 18
JO - PLoS computational biology
JF - PLoS computational biology
IS - 9
M1 - e1010405
ER -