skip to main content

Explore Research Products in the NSF-PAR

Title: A pre-registered short-term forecasting study of COVID-19 in Germany and Poland during the second wave
Abstract Disease modelling has had considerable policy impact during the ongoing COVID-19 pandemic, and it is increasingly acknowledged that combining multiple models can improve the reliability of outputs. Here we report insights from ten weeks of collaborative short-term forecasting of COVID-19 in Germany and Poland (12 October–19 December 2020). The study period covers the onset of the second wave in both countries, with tightening non-pharmaceutical interventions (NPIs) and subsequently a decay (Poland) or plateau and renewed increase (Germany) in reported cases. Thirteen independent teams provided probabilistic real-time forecasts of COVID-19 cases and deaths. These were reported for lead times of one to four weeks, with evaluation focused on one- and two-week horizons, which are less affected by changing NPIs. Heterogeneity between forecasts was considerable both in terms of point predictions and forecast spread. Ensemble forecasts showed good relative performance, in particular in terms of coverage, but did not clearly dominate single-model predictions. The study was preregistered and will be followed up in future phases of the pandemic.  more » « less
Award ID(s):
2135784
NSF-PAR ID:
10338519
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; more » ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; « less
Date Published:
Journal Name:
Nature Communications
Volume:
12
Issue:
1
ISSN:
2041-1723
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ( , Scientific Reports)
    Abstract Public health intervention to contain the ongoing COVID-19 pandemic significantly differed by country since the SARS-CoV-2 spread varied regionally in time and in scale. Since vaccinations were not available until the end of 2020 non-pharmaceutical interventions (NPIs) remained the only strategies to mitigate the pandemic spread at that time. Belarus in Europe is one of a few countries with a high Human Development Index where no lockdowns have ever been implemented and only limited NPIs have taken place for a period of time. Therefore, the Belarusian case was evaluated and compared in terms of the mortality burden. Since the COVID-19 mortality was low, the excess overall mortality was studied for Belarus. Since no overall mortality data have been reported past June 2020 the analysis was complemented by the study of Google Trends funeral-related search queries up until August 2021. Depending on the model, the Belarusian mortality for June of 2020 was 29 to 39% higher than otherwise expected with the corresponding estimated excess death was from 2953 to 3690 while the reported COVID-19 mortality for June 2020 was only 157 cases. The Belarusian excess mortality for June 2020 was higher than for all neighboring countries with an excess of 5% for Poland, 5% for Ukraine, 8% for Russia, 11% for Lithuania and 11% for Latvia. The relationship between Google Trends and mortality time series was studied using Granger’s test and the results were statistically significant. The results for Google Trends searches did vary by key phrase with the largest excess of 138% for April 2020 and 148% for September 2020 was observed for a key phrase “coffin”, while the largest excess of 218% for January 2021 was observed for “funeral services”. In summary, there are indications of the excess overall mortality in Belarus, which is larger than the reported COVID-19-related mortality. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al ( , eLife)
    Background:

    Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022.

     Methods:

    We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance.

     Results:

    Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models.

     Conclusions:

    Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks.

     Funding:

    AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 (https://www.nfdi4health.de/task-force-covid-19-2) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).

     
    more » « less
  3. ; ; ; ; ; ( , Journal of Medical Internet Research)
    Background Worldwide, nonpharmacologic interventions (NPIs) have been the main tool used to mitigate the COVID-19 pandemic. This includes social distancing measures (closing businesses, closing schools, and quarantining symptomatic persons) and contacttracing (tracking and following exposed individuals). While preliminary research across the globe has shown these policies to be effective, there is currently a lack of information on the effectiveness of NPIs in the United States. Objective The purpose of this study was to create a granular NPI data set at the county level and then analyze the relationship between NPI policies and changes in reported COVID-19 cases. Methods Using a standardized crowdsourcing methodology, we collected time-series data on 7 key NPIs for 1320 US counties. Results This open-source data set is the largest and most comprehensive collection of county NPI policy data and meets the need for higher-resolution COVID-19 policy data. Our analysis revealed a wide variation in county-level policies both within and among states (P<.001). We identified a correlation between workplace closures and lower growth rates of COVID-19 cases (P=.004). We found weak correlations between shelter-in-place enforcement and measures of Democratic local voter proportion (R=0.21) and elected leadership (R=0.22). Conclusions This study is the first large-scale NPI analysis at the county level demonstrating a correlation between NPIs and decreased rates of COVID-19. Future work using this data set will explore the relationship between county-level policies and COVID-19 transmission to optimize real-time policy formulation. 
    more » « less
  4. ; ; ; ; ; ; ; ; ( , Frontiers in Sustainable Food Systems)
    Throughout history, urban agriculture practitioners have adapted to various challenges by continuing to provide food and social benefits. Urban gardens and farms have also responded to sudden political, economic, ecological, and social crises: wartime food shortages; urban disinvestment and property abandonment; earthquakes and floods; climate-change induced weather events; and global economic disruptions. This paper examines the effects on, and responses by, urban farms and gardens to the COVID-19 pandemic. The paper is based on data collected in the summer of 2020 at the onset of the pandemic when cities were struggling with appropriate responses to curb its spread. It builds on an international research project (FEW-meter) that developed a methodology to measure material and social benefits of urban agriculture (UA) in five countries (France, Germany, Poland, UK and USA) over two growing seasons, from a Food-Energy-Water nexus perspective. We surveyed project partners to ascertain the effects of COVID-19 on those gardens and farms and we interviewed policy stakeholders in each country to investigate the wider impacts of the pandemic on UA. We report the results with respect to five key areas: (1) garden accessibility and service provision during the pandemic; (2) adjustments to operational arrangements; (3) effects on production; (4) support for urban farms and gardens through the pandemic; and (5) thoughts about the future of urban agriculture in the recovery period and beyond. The paper shows that the pandemic resulted in multiple challenges to gardens and farms including the loss of ability to provide support services, lost income, and reductions in output because of reduced labor supply. But COVID-19 also created several opportunities: new markets to sell food locally; more time available to gardeners to work in their allotments; and increased community cohesion as neighboring gardeners looked out for one another. By illustrating the range of challenges faced by the pandemic, and strategies to address challenges used by different farms and gardens, the paper illustrates how gardens in this pandemic have adapted to become more resilient and suggests lessons for pandemic recovery and longer-term planning to enable UA to respond to future public health and other crises. 
    more » « less
  5. ; ; ; ; ; ; ( , Journal of Big Data)
    Abstract The early detection of the coronavirus disease 2019 (COVID-19) outbreak is important to save people’s lives and restart the economy quickly and safely. People’s social behavior, reflected in their mobility data, plays a major role in spreading the disease. Therefore, we used the daily mobility data aggregated at the county level beside COVID-19 statistics and demographic information for short-term forecasting of COVID-19 outbreaks in the United States. The daily data are fed to a deep learning model based on Long Short-Term Memory (LSTM) to predict the accumulated number of COVID-19 cases in the next two weeks. A significant average correlation was achieved ( r =0.83 ( p = 0.005 )) between the model predicted and actual accumulated cases in the interval from August 1, 2020 until January 22, 2021. The model predictions had r > 0.7 for 87% of the counties across the United States. A lower correlation was reported for the counties with total cases of <1000 during the test interval. The average mean absolute error (MAE) was 605.4 and decreased with a decrease in the total number of cases during the testing interval. The model was able to capture the effect of government responses on COVID-19 cases. Also, it was able to capture the effect of age demographics on the COVID-19 spread. It showed that the average daily cases decreased with a decrease in the retiree percentage and increased with an increase in the young percentage. Lessons learned from this study not only can help with managing the COVID-19 pandemic but also can help with early and effective management of possible future pandemics. The code used for this study was made publicly available on https://github.com/Murtadha44/covid-19-spread-risk. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email