skip to main content

Explore Research Products in the NSF-PAR

  • NSF Public Access
  • Search Results
  • Deep reinforcement learning framework for controlling infectious disease outbreaks in the context of multi-jurisdictions

This content will become publicly available on June 29, 2024

Title: Deep reinforcement learning framework for controlling infectious disease outbreaks in the context of multi-jurisdictions
In the absence of pharmaceutical interventions, social distancing and lockdown have been key options for controlling new or reemerging respiratory infectious disease outbreaks. The timely implementation of these interventions is vital for effectively controlling and safeguarding the economy.Motivated by the COVID-19 pandemic, we evaluated whether, when, and to what level lockdowns are necessary to minimize epidemic and economic burdens of new disease outbreaks. We formulated the question as a sequential decision-making Markov Decision Process and solved it using deep Q-network algorithm. We evaluated the question under two objective functions: a 2-objective function to minimize economic burden and hospital capacity violations, suitable for diseases with severe health risks but with minimal death, and a 3-objective function that additionally minimizes the number of deaths, suitable for diseases that have high risk of mortality.A key feature of the model is that we evaluated the above questions in the context of two-geographical jurisdictions that interact through travel but make autonomous and independent decisions, evaluating under cross-jurisdictional cooperation and non-cooperation. In the 2-objective function under cross-jurisdictional cooperation, the optimal policy was to aim for shutdowns at 50 and 25% per day. Though this policy avoided hospital capacity violations, the shutdowns extended until a large proportion of the population reached herd immunity. Delays in initiating this optimal policy or non-cooperation from an outside jurisdiction required shutdowns at a higher level of 75% per day, thus adding to economic burdens. In the 3-objective function, the optimal policy under cross-jurisdictional cooperation was to aim for shutdowns of up to 75% per day to prevent deaths by reducing infected cases. This optimal policy continued for the entire duration of the simulation, suggesting that, until pharmaceutical interventions such as treatment or vaccines become available, contact reductions through physical distancing would be necessary to minimize deaths. Deviating from this policy increased the number of shutdowns and led to several deaths.In summary, we present a decision-analytic methodology for identifying optimal lockdown strategy under the context of interactions between jurisdictions that make autonomous and independent decisions. The numerical analysis outcomes are intuitive and, as expected, serve as proof of the feasibility of such a model. Our sensitivity analysis demonstrates that the optimal policy exhibits robustness to minor alterations in the transmission rate, yet shows sensitivity to more substantial deviations. This finding underscores the dynamic nature of epidemic parameters, thereby emphasizing the necessity for models trained across a diverse range of values to ensure effective policy-making.  more » « less
Award ID(s):
1915481
NSF-PAR ID:
10478706
Author(s) / Creator(s):
;
Corporate Creator(s):
Publisher / Repository:
AIMS Press
Date Published:
Journal Name:
Mathematical Biosciences and Engineering
Volume:
20
Issue:
8
ISSN:
1551-0018
Page Range / eLocation ID:
7666 to 7684
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ; ; ; et al ( , PLOS Computational Biology)
    Flegg, Jennifer A. (Ed.)
    Stay-at-home orders and shutdowns of non-essential businesses are powerful, but socially costly, tools to control the pandemic spread of SARS-CoV-2. Mass testing strategies, which rely on widely administered frequent and rapid diagnostics to identify and isolate infected individuals, could be a potentially less disruptive management strategy, particularly where vaccine access is limited. In this paper, we assess the extent to which mass testing and isolation strategies can reduce reliance on socially costly non-pharmaceutical interventions, such as distancing and shutdowns. We develop a multi-compartmental model of SARS-CoV-2 transmission incorporating both preventative non-pharmaceutical interventions (NPIs) and testing and isolation to evaluate their combined effect on public health outcomes. Our model is designed to be a policy-guiding tool that captures important realities of the testing system, including constraints on test administration and non-random testing allocation. We show how strategic changes in the characteristics of the testing system, including test administration, test delays, and test sensitivity, can reduce reliance on preventative NPIs without compromising public health outcomes in the future. The lowest NPI levels are possible only when many tests are administered and test delays are short, given limited immunity in the population. Reducing reliance on NPIs is highly dependent on the ability of a testing program to identify and isolate unreported, asymptomatic infections. Changes in NPIs, including the intensity of lockdowns and stay at home orders, should be coordinated with increases in testing to ensure epidemic control; otherwise small additional lifting of these NPIs can lead to dramatic increases in infections, hospitalizations and deaths. Importantly, our results can be used to guide ramp-up of testing capacity in outbreak settings, allow for the flexible design of combined interventions based on social context, and inform future cost-benefit analyses to identify efficient pandemic management strategies. 
    more » « less
  2. ; ; ; ; ; ; ( , Mathematical Models and Methods in Applied Sciences)
    The outbreak of COVID-19 resulted in high death tolls all over the world. The aim of this paper is to show how a simple SEIR model was used to make quick predictions for New Jersey in early March 2020 and call for action based on data from China and Italy. A more refined model, which accounts for social distancing, testing, contact tracing and quarantining, is then proposed to identify containment measures to minimize the economic cost of the pandemic. The latter is obtained taking into account all the involved costs including reduced economic activities due to lockdown and quarantining as well as the cost for hospitalization and deaths. The proposed model allows one to find optimal strategies as combinations of implementing various non-pharmaceutical interventions and study different scenarios and likely initial conditions. 
    more » « less
  3. ; ; ; ( , Brookings papers on economic activity)
    Eberly, Jan ; Romer, David (Ed.)
    In the spring of 2020, the initial surge of COVID-19 infections and deaths was flattened using a combination of economic shutdowns and noneconomic non-pharmaceutical interventions (NPIs). The possibility of a second wave of infections and deaths raises the question of what interventions can be used to significantly reduce deaths while supporting, not preventing, economic recovery. We use a five-age epidemiological model combined with sixty-six-sector economic accounting to examine policies to avert and to respond to a second wave. We find that a second round of economic shutdowns alone are neither sufficient nor necessary to avert or quell a second wave. In contrast, noneconomic NPIs, such as wearing masks and personal distancing, increasing testing and quarantine, reintroducing restrictions on social and recreational gatherings, and enhancing protections for the elderly together can mitigate a second wave while leaving room for an economic recovery. 
    more » « less
  4. ; ; ; ; ; ; ; ; ; ; et al ( , Scientific Reports)
    Abstract This research measures the epidemiological and economic impact of COVID-19 spread in the US under different mitigation scenarios, comprising of non-pharmaceutical interventions. A detailed disease model of COVID-19 is combined with a model of the US economy to estimate the direct impact of labor supply shock to each sector arising from morbidity, mortality, and lockdown, as well as the indirect impact caused by the interdependencies between sectors. During a lockdown, estimates of jobs that are workable from home in each sector are used to modify the shock to labor supply. Results show trade-offs between economic losses, and lives saved and infections averted are non-linear in compliance to social distancing and the duration of the lockdown. Sectors that are worst hit are not the labor-intensive sectors such as the Agriculture sector and the Construction sector, but the ones with high valued jobs such as the Professional Services, even after the teleworkability of jobs is accounted for. Additionally, the findings show that a low compliance to interventions can be overcome by a longer shutdown period and vice versa to arrive at similar epidemiological impact but their net effect on economic loss depends on the interplay between the marginal gains from averting infections and deaths, versus the marginal loss from having healthy workers stay at home during the shutdown. 
    more » « less
  5. ; ; ; ; ; ; ; ; ; ; et al ( , medRxiv)
    This research measures the epidemiological and economic impact of COVID-19 spread in the US under different mitigation scenarios, comprising of non-pharmaceutical interventions. A detailed disease model of COVID-19 is combined with a model of the US economy to estimate the direct impact of labor supply shock to each sector arising from morbidity, mortality, and lockdown, as well as the indirect impact caused by the interdependencies between sectors. During a lockdown, estimates of jobs that are workable from home in each sector are used to modify the shock to labor supply. Results show trade-o s between economic losses, and lives saved and infections averted are non-linear in compliance to social distancing and the duration of lockdown. Sectors that are worst hit are not the labor-intensive sectors such as Agriculture and Construction, but the ones with high valued jobs such as Professional Services, even after the teleworkability of jobs is accounted for. Additionally, the findings show that a low compliance to interventions can be overcome by a longer shutdown period and vice versa to arrive at similar epidemiological impact but their net effect on economic loss depends on the interplay between the marginal gains from averting infections and deaths, versus the marginal loss from having healthy workers stay at home during the shutdown. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email