skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Spatiotemporal Forecasting of Opioid-Related Fatal Overdoses: Towards Best Practices for Modeling and Evaluation
To inform public health interventions, researchers have developed models to forecast opioid-related overdose mortality. These efforts often have limited overlap in the models and datasets employed, presenting challenges to assessing progress in this field. Furthermore, common error-based performance metrics, such as root mean squared error (RMSE), cannot directly assess a key modeling purpose: the identification of priority areas for interventions. We recommend a new intervention-aware performance metric, Percentage of Best Possible Reach (%BPR). We compare metrics for many published models across two distinct geographic settings, Cook County, Illinois and Massachusetts, assuming the budget to intervene in 100 census tracts out of 1000s in each setting. The top-performing models based on RMSE recommend areas that do not always reach the most possible overdose events. In Massachusetts, the top models preferred by %BPR could have reached 18 additional fatal overdoses per year in 2020-2021 compared to models favored by RMSE. In Cook County, the different metrics select similar top-performing models, yet other models with similar RMSE can have significant variation in %BPR. We further find that simple models often perform as well as recently published ones. We release open code and data for others to build upon.  more » « less
Award ID(s):
2338962 1908617 2149871
PAR ID:
10569557
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
Oxford University Press
Date Published:
Journal Name:
American Journal of Epidemiology
ISSN:
0002-9262
Subject(s) / Keyword(s):
predictive modeling, spatiotemporal forecasting, machine learning, opioid, overdose, public health practice
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, PNAS Nexus)
    Galea, Sandro (Ed.)
    Abstract The drug-overdose crisis in the United States continues to intensify. Fatalities have increased 5-fold since 1999 reaching a record high of 108,000 deaths in 2021. The epidemic has unfolded through distinct waves of different drug types, uniquely impacting various age, gender, race, and ethnic groups in specific geographical areas. One major challenge in designing interventions and efficiently delivering treatment is forecasting age-specific overdose patterns at the local level. To address this need, we develop a forecasting method that assimilates observational data obtained from the CDC WONDER database with an age-structured model of addiction and overdose mortality. We apply our method nationwide and to three select areas: Los Angeles County, Cook County, and the five boroughs of New York City, providing forecasts of drug-overdose mortality and estimates of relevant epidemiological quantities, such as mortality and age-specific addiction rates. 
    more » « less
  2. ; ; (, Journal of Urban Health)
    Abstract Overdose deaths involving fentanyl represent a major public health crisis in the USA. This study investigates the spatiotemporal dynamics of fentanyl-involved deaths before, during, and after the COVID-19 pandemic and examines how sociodemographic factors influence these deaths across geographic regions. Using a retrospective ecological approach, we analyzed data on ZIP code-level fentanyl-related deaths in Cook County, IL, between 2018 and 2023, obtained from the Medical Examiner’s Office and linked with sociodemographic data from the American Community Survey. We first mapped area-level death rates to assess their distribution and then conducted global and local clustering analyses to identify spatial autocorrelations and the locations of high- or low-death-rate areas. A geographically weighted Poisson regression (GWPR) model evaluated the associations between area-level fentanyl-related death rates and the area-level proportion of young adults, males, and individuals with at least a college degree, disability rate, and poverty rate. Spatial analyses found stronger spatial autocorrelations during (2020–2021) and after (2022–2023) the pandemic. Initially, high death rates were concentrated in the downtown area of Chicago, and they expanded to the surrounding areas during and after the pandemic. The GWPR model revealed that an increase in the area-level proportions of poverty, disability, and young adult residents increased the fentanyl-related death rates in most of the areas. Our findings highlight the urgent need to address the evolving dynamics of fentanyl-related overdoses through tailored public health interventions that account for the unique socioeconomic determinants of different regions. Importantly, a comprehensive approach to addressing differences in overdose death rates and their risk factors will be crucial to mitigating this public health crisis. 
    more » « less
  3. ; (, Water Resources Research)
    Abstract An inadequate characterization of hydrogeological properties can significantly decrease the trustworthiness of subsurface flow and transport model predictions. A variety of data assimilation methods have been proposed in order to estimate hydrogeological parameters from spatially scarce data by incorporating them into the governing physical models. In order to quantify the accuracy of the estimations, several metrics have been used such as Rank Histograms, root‐mean‐square error (RMSE), and Ensemble Spread. However, these commonly used metrics do not regard the spatial correlation of the aquifer's properties. This can cause permeability fields with very different spatial structures to have similar histograms or RMSE. In this paper, we propose an approach based on color coherence vectors (CCV) for evaluating the performance of these estimation methods. CCV is a histogram‐based technique for comparing images that incorporate spatial information. We represent estimated fields as digital three‐channel images and use CCV to compare and quantify the accuracy of estimations. The appealing feature of this technique is that it considers the spatial structure embedded in the estimated fields. The sensitivity of CCV to spatial information makes it a suitable metric for assessing the performance of data assimilation techniques. Under various factors, such as numbers of measurements and structural parameters of the log conductivity field, we compare the performance of CCV with the RMSE. 
    more » « less
  4. ; ; (, Earth Surface Processes and Landforms)
    Abstract Grid independence studies have emerged as essential methodological frameworks for comprehending the impact of domain resolution on simulating anisotropic turbulence at the river‐reach scale using large eddy simulation models. This study proposes a methodology to assess the loss of information in turbulent flow patterns when coarsening the computational domain, examined in a 1‐km transect of the Colorado River along Marble Canyon. Seven computational domain resolutions are explored to analyse the sensitivity of turbulent flow to spatial resolution changes, utilizing the turbulent kinetic energy (TKE) spectrum technique and spatiotemporal analysis of eddy structures via statistical metrics such as root mean square error (RMSE), Kullback‐Leibler (KL) divergence, Nash‐Sutcliffe model efficiency coefficient (NSE), wavelet power spectrum and grid convergence index (GCI). Based on physical principles and statistics, these metrics quantify information loss and assess domain resolutions. A computational fluid dynamic (CFD) model is developed by employing the detached eddy simulation (DES) technique, with boundary condition (BC) integrating the rough wall extension of the Spallart‐Allmaras model in cells near the bed. Evaluation of domain resolutions aims to identify grid cell sizes capturing flow behaviour and hydraulic characteristics, including primary and secondary flows, return currents, shear layers and primary and secondary eddies. The study observes an increase in data representation of the TKE spectrum with finer spatial domain resolution. Additionally, surface analysis, conducted via RMSE, KL and NSE metrics, identifies specific areas within the flow field showing high sensitivity to refining the grid cell sizes. 
    more » « less
  5. ; ; ; ; (, Scientific Reports)
    Abstract An intelligent sensing framework using Machine Learning (ML) and Deep Learning (DL) architectures to precisely quantify dielectrophoretic force invoked on microparticles in a textile electrode-based DEP sensing device is reported. The prediction accuracy and generalization ability of the framework was validated using experimental results. Images of pearl chain alignment at varying input voltages were used to build deep regression models using modified ML and CNN architectures that can correlate pearl chain alignment patterns of Saccharomyces cerevisiae(yeast) cells and polystyrene microbeads to DEP force. Various ML models such as K-Nearest Neighbor, Support Vector Machine, Random Forest, Neural Networks, and Linear Regression along with DL models such as Convolutional Neural Network (CNN) architectures of AlexNet, ResNet-50, MobileNetV2, and GoogLeNet have been analyzed in order to build an effective regression framework to estimate the force induced on yeast cells and microbeads. The efficiencies of the models were evaluated using Mean Absolute Error, Mean Absolute Relative, Mean Squared Error, R-squared, and Root Mean Square Error (RMSE) as evaluation metrics. ResNet-50 with RMSPROP gave the best performance, with a validation RMSE of 0.0918 on yeast cells while AlexNet with ADAM optimizer gave the best performance, with a validation RMSE of 0.1745 on microbeads. This provides a baseline for further studies in the application of deep learning in DEP aided Lab-on-Chip devices. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email