skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Cost-Effectiveness and Value-of-Information Analysis Using Machine Learning–Based Metamodeling: A Case of Hepatitis C Treatment
Title: Cost-Effectiveness and Value-of-Information Analysis Using Machine Learning–Based Metamodeling: A Case of Hepatitis C Treatment
BackgroundMetamodels can address some of the limitations of complex simulation models by formulating a mathematical relationship between input parameters and simulation model outcomes. Our objective was to develop and compare the performance of a machine learning (ML)–based metamodel against a conventional metamodeling approach in replicating the findings of a complex simulation model. MethodsWe constructed 3 ML-based metamodels using random forest, support vector regression, and artificial neural networks and a linear regression-based metamodel from a previously validated microsimulation model of the natural history hepatitis C virus (HCV) consisting of 40 input parameters. Outcomes of interest included societal costs and quality-adjusted life-years (QALYs), the incremental cost-effectiveness (ICER) of HCV treatment versus no treatment, cost-effectiveness analysis curve (CEAC), and expected value of perfect information (EVPI). We evaluated metamodel performance using root mean squared error (RMSE) and Pearson’s R2on the normalized data. ResultsThe R2values for the linear regression metamodel for QALYs without treatment, QALYs with treatment, societal cost without treatment, societal cost with treatment, and ICER were 0.92, 0.98, 0.85, 0.92, and 0.60, respectively. The corresponding R2values for our ML-based metamodels were 0.96, 0.97, 0.90, 0.95, and 0.49 for support vector regression; 0.99, 0.83, 0.99, 0.99, and 0.82 for artificial neural network; and 0.99, 0.99, 0.99, 0.99, and 0.98 for random forest. Similar trends were observed for RMSE. The CEAC and EVPI curves produced by the random forest metamodel matched the results of the simulation output more closely than the linear regression metamodel. ConclusionsML-based metamodels generally outperformed traditional linear regression metamodels at replicating results from complex simulation models, with random forest metamodels performing best. HighlightsDecision-analytic models are frequently used by policy makers and other stakeholders to assess the impact of new medical technologies and interventions. However, complex models can impose limitations on conducting probabilistic sensitivity analysis and value-of-information analysis, and may not be suitable for developing online decision-support tools. Metamodels, which accurately formulate a mathematical relationship between input parameters and model outcomes, can replicate complex simulation models and address the above limitation. The machine learning–based random forest model can outperform linear regression in replicating the findings of a complex simulation model. Such a metamodel can be used for conducting cost-effectiveness and value-of-information analyses or developing online decision support tools.  more » « less
Award ID(s):
1722665 1722614
PAR ID:
10371514
Author(s) / Creator(s):
 ;  ;  
Publisher / Repository:
SAGE Publications
Date Published:
Journal Name:
Medical Decision Making
Volume:
43
Issue:
1
ISSN:
0272-989X
Format(s):
Medium: X Size: p. 68-77
Size(s):
p. 68-77
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, Alimentary Pharmacology & Therapeutics)
    Summary BackgroundThe hepatitis C virus (HCV) care cascade has changed dramatically following the introduction of direct‐acting anti‐virals (DAAs). Up‐to‐date estimates of the cascade are needed to monitor progress, identify key gaps and inform policy. AimTo estimate the current and future HCV care cascade in the United States, nationally and in select subpopulations of interest. MethodsWe used a previously validated mathematical model to simulate the landscape of HCV in the United States from 2011 onwards, accounting for HCV screening policy updates, newer HCV treatments and rising HCV incidence. ResultsBy the end of 2018, of 4.29 million HCV persons alive, 2.71 million (63%) were actively viremic, 2.24 million (52%) aware and 1.58 million (37%) cured. By 2030, under the status quo, of 3.65 million HCV persons alive, 1.88 million (51%) would be viremic, 2.25 million (62%) aware and 1.77 million (49%) cured. The HCV care cascade in 2018 differed substantially by subpopulation: of 1.34 million incarcerated HCV persons, 96% were viremic, 36% aware and 4% cured; of 0.87 million HCV persons in Medicare, 31% were viremic, 72% aware and 69% cured; and of 0.37 million HCV persons in Medicaid, 49% were viremic, 54% aware and 51% cured. Implementing universal screening, providing unrestricted treatment and controlling HCV incidence were factors found to have the largest effect on improving the HCV care cascade. ConclusionsSince the launch of DAAs, the HCV care cascade has shifted towards higher awareness and treatment rates; however, additional interventions are needed to move towards HCV elimination. 
    more » « less
  2. ; ; ; (, Head & Neck)
    Abstract BackgroundNew patient referrals are often processed by practice coordinators with little‐to‐no medical background. Treatment delays due to incorrect referral processing, however, have detrimental consequences. Identifying variables that are associated with a higher likelihood of surgical oncological resection may improve patient referral processing and expedite the time to treatment. The study objective is to develop a supervised machine learning (ML) platform that identifies relevant variables associated with head and neck surgical resection. MethodsA retrospective cohort study was conducted on 64 222 patient datapoints from the SEER database. ResultsThe random forest ML model correctly classified patients who were offered head and neck surgery with an 81% accuracy rate. The sensitivity and specificity rates were 86% and 71%. The positive and negative predictive values were 85% and 73%. ConclusionsML modeling accurately predicts head and neck cancer surgery recommendations based on patient and cancer information from a large population‐based dataset. ML adjuncts for referral processing may decrease the time to treatment for patients with cancer. 
    more » « less
  3. ; ; ; ; (, Agronomy Journal)
    Abstract Preharvest yield estimates can be used for harvest planning, marketing, and prescribing in‐season fertilizer and pesticide applications. One approach that is being widely tested is the use of machine learning (ML) or artificial intelligence (AI) algorithms to estimate yields. However, one barrier to the adoption of this approach is that ML/AI algorithms behave as a black block. An alternative approach is to create an algorithm using Bayesian statistics. In Bayesian statistics, prior information is used to help create the algorithm. However, algorithms based on Bayesian statistics are not often computationally efficient. The objective of the current study was to compare the accuracy and computational efficiency of four Bayesian models that used different assumptions to reduce the execution time. In this paper, the Bayesian multiple linear regression (BLR), Bayesian spatial, Bayesian skewed spatial regression, and the Bayesian nearest neighbor Gaussian process (NNGP) models were compared with ML non‐Bayesian random forest model. In this analysis, soybean (Glycine max) yields were the response variable (y), and spaced‐based blue, green, red, and near‐infrared reflectance that was measured with the PlanetScope satellite were the predictor (x). Among the models tested, the Bayesian (NNGP;R2‐testing = 0.485) model, which captures the short‐range correlation, outperformed the (BLR;R2‐testing = 0.02), Bayesian spatial regression (SRM;R2‐testing = 0.087), and Bayesian skewed spatial regression (sSRM;R2‐testing = 0.236) models. However, associated with improved accuracy was an increase in run time from 534 s for the BLR model to 2047 s for the NNGP model. These data show that relatively accurate within‐field yield estimates can be obtained without sacrificing computational efficiency and that the coefficients have biological meaning. However, all Bayesian models had lowerR2values and higher execution times than the random forest model. 
    more » « less
  4. ; ; ; ; ; ; ; (, The Journal of Infectious Diseases)
    Abstract BackgroundPrior studies demonstrate that eliminating hepatitis C virus (HCV) in the United States (US) heavily depends on treating incarcerated persons. Knowing the scope of the carceral HCV epidemic by state will help guide national elimination efforts. MethodsBetween 2019 and 2023, all state prison systems received surveys requesting data on hepatitis C antibody and viremic prevalence. We supplemented survey information with publicly available HCV data to corroborate responses and fill in data gaps. ResultsWeighting HCV prevalence by state prison population size, we estimate that 15.2% of the US prison population is HCV seropositive and 8.7% is viremic; 54.9% of seropositive persons have detectable RNA. Applying prevalence estimates to the total prison population at year-end 2021, 91 090 persons with HCV infection resided in a state prison. ConclusionsWith updated and more complete HCV data from all 50 states, HCV prevalence in state prisons is nearly 9-fold higher than the US general population. The heterogeneity in HCV prevalence by state prison system may reflect variable exposure before arrest and/or differences in treatment availability during incarceration. Elimination of HCV in the country depends on addressing the carceral epidemic, and one of the first steps is understanding the size of the problem. 
    more » « less
  5. ; ; ; ; ; ; ; ; (, Scientific Reports)
    Abstract Hepatitis C virus (HCV) is 15 times more prevalent among persons in Spain’s prisons than in the community. Recently, Spain initiated a pilot program, JAILFREE-C, to treat HCV in prisons using direct-acting antivirals (DAAs). Our aim was to identify a cost-effective strategy to scale-up HCV treatment in all prisons. Using a validated agent-based model, we simulated the HCV landscape in Spain’s prisons considering disease transmission, screening, treatment, and prison-community dynamics. Costs and disease outcomes under status quo were compared with strategies to scale-up treatment in prisons considering prioritization (HCV fibrosis stage vs. HCV prevalence of prisons), treatment capacity (2,000/year vs. unlimited) and treatment initiation based on sentence lengths (>6 months vs. any). Scaling-up treatment by treating all incarcerated persons irrespective of their sentence length provided maximum health benefits–preventing 10,200 new cases of HCV, and 8,300 HCV-related deaths between 2019–2050; 90% deaths prevented would have occurred in the community. Compared with status quo, this strategy increased quality-adjusted life year (QALYs) by 69,700 and costs by €670 million, yielding an incremental cost-effectiveness ratio of €9,600/QALY. Scaling-up HCV treatment with DAAs for the entire Spanish prison population, irrespective of sentence length, is cost-effective and would reduce HCV burden. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email