More Like this

1. Inferring Which Medical TreatmentsWork from Reports of Clinical Trials

   Lehman, Eric; DeYoung, Jay; Barzilay, Regina; Wallace, Byron C. (January 2019, Annual Conference of the North American Chapter of the Association for Computational Linguistics)

   How do we know if a particular medical treatment actually works? Ideally one would consult all available evidence from relevant clinical trials. Unfortunately, such results are primarily disseminated in natural language scientific articles, imposing substantial burden on those trying to make sense of them. In this paper, we present a new task and corpus for making this unstructured evidence actionable. The task entails inferring reported findings from a full-text article describing a randomized controlled trial (RCT) with respect to a given intervention, comparator, and outcome of interest, e.g., inferring if an article provides evidence supporting the use of aspirin to reduce risk of stroke, as compared to placebo. We present a new corpus for this task comprising 10,000+ prompts coupled with fulltext articles describing RCTs. Results using a suite of models — ranging from heuristic (rule-based) approaches to attentive neural architectures — demonstrate the difficulty of the task, which we believe largely owes to the lengthy, technical input texts. To facilitate further work on this important, challenging problem we make the corpus, documentation, a website and leaderboard, and code for baselines and evaluation available at http: //evidence-inference.ebm-nlp.com/. 

   more » « less
2. Leveraging infectious disease models to interpret randomized controlled trials: Controlling enteric pathogen transmission through water, sanitation, and hygiene interventions

   https://doi.org/10.1371/journal.pcbi.1010748  

   Brouwer, Andrew F.; Eisenberg, Marisa C.; Bakker, Kevin M.; Boerger, Savannah N.; Zahid, Mondal H.; Freeman, Matthew C.; Eisenberg, Joseph N. (December 2022, PLOS Computational Biology)

   Lau, Eric HY (Ed.)

   Randomized controlled trials (RCTs) evaluate hypotheses in specific contexts and are often considered the gold standard of evidence for infectious disease interventions, but their results cannot immediately generalize to other contexts (e.g., different populations, interventions, or disease burdens). Mechanistic models are one approach to generalizing findings between contexts, but infectious disease transmission models (IDTMs) are not immediately suited for analyzing RCTs, since they often rely on time-series surveillance data. We developed an IDTM framework to explain relative risk outcomes of an infectious disease RCT and applied it to a water, sanitation, and hygiene (WASH) RCT. This model can generalize the RCT results to other contexts and conditions. We developed this compartmental IDTM framework to account for key WASH RCT factors: i) transmission across multiple environmental pathways, ii) multiple interventions applied individually and in combination, iii) adherence to interventions or preexisting conditions, and iv) the impact of individuals not enrolled in the study. We employed a hybrid sampling and estimation framework to obtain posterior estimates of mechanistic parameter sets consistent with empirical outcomes. We illustrated our model using WASH Benefits Bangladesh RCT data (n = 17,187). Our model reproduced reported diarrheal prevalence in this RCT. The baseline estimate of the basic reproduction number R 0 for the control arm (1.10, 95% CrI: 1.07, 1.16) corresponded to an endemic prevalence of 9.5% (95% CrI: 7.4, 13.7%) in the absence of interventions or preexisting WASH conditions. No single pathway was likely able to sustain transmission: pathway-specific R 0 s for water, fomites, and all other pathways were 0.42 (95% CrI: 0.03, 0.97), 0.20 (95% CrI: 0.02, 0.59), and 0.48 (95% CrI: 0.02, 0.94), respectively. An IDTM approach to evaluating RCTs can complement RCT analysis by providing a rigorous framework for generating data-driven hypotheses that explain trial findings, particularly unexpected null results, opening up existing data to deeper epidemiological understanding. 

   more » « less
3. Precise unbiased estimation in randomized experiments using auxiliary observational data

   https://doi.org/10.1515/jci-2022-0011  

   Gagnon-Bartsch, Johann A.; Sales, Adam C.; Wu, Edward; Botelho, Anthony F.; Erickson, John A.; Miratrix, Luke W.; Heffernan, Neil T. (January 2023, Journal of Causal Inference)

   Abstract Randomized controlled trials (RCTs) admit unconfounded design-based inference – randomization largely justifies the assumptions underlying statistical effect estimates – but often have limited sample sizes. However, researchers may have access to big observational data on covariates and outcomes from RCT nonparticipants. For example, data from A/B tests conducted within an educational technology platform exist alongside historical observational data drawn from student logs. We outline a design-based approach to using such observational data for variance reduction in RCTs. First, we use the observational data to train a machine learning algorithm predicting potential outcomes using covariates and then use that algorithm to generate predictions for RCT participants. Then, we use those predictions, perhaps alongside other covariates, to adjust causal effect estimates with a flexible, design-based covariate-adjustment routine. In this way, there is no danger of biases from the observational data leaking into the experimental estimates, which are guaranteed to be exactly unbiased regardless of whether the machine learning models are “correct” in any sense or whether the observational samples closely resemble RCT samples. We demonstrate the method in analyzing 33 randomized A/B tests and show that it decreases standard errors relative to other estimators, sometimes substantially. 

   more » « less
4. Network Analysis and Recommendation for Infectious Disease Clinical Trial Research

   Elkin, Magdalyn; Andrews, Whitney A.; Zhu, Xingquan (September 2019, Proc. of the 10th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (BCB-2019))

   Clinical trials are crucial for the advancement of treatment and knowledge within the medical community. Since 2007, US federal government took the initiative and requires organizations sponsoring clinical trials with at least one site in the United States to submit information on these clinical trials to the ClinicalTrials.gov database, resulting in a rich source of information for clinical trial research. Nevertheless, only a handful of analytic studies have been carried out to understand this valuable data source. In this study, we propose to use network analysis to understand infectious disease clinical trial research. Our goal is to answer two important questions: (1) what are the concentrations and characteristics of infectious disease clinical trail research? and (2) how to accurately predict what type of clinical trials a sponsor (or an investigator) is interested in? The answers to the first question provide effective ways to summarize clinical trial research related to particular disease(s), and the answers to the second question help match clinical trial sponsors and investigators for information recommendation. By using 4,228 clinical trails as the test bed, our study involves 4,864 sponsors and 1,879 research areas characterized by Medical Subject Heading (MeSH) keywords. We extract a set of network measures to show patterns of infectious disease clinical trials, and design a new community based link prediction approach to predict sponsors' interests, with significant improvement compared to baselines. This trans-formative study concludes that using network analysis can tremendously help the understanding of clinical trial research for effective summarization, characterization, and prediction. 

   more » « less
5. Building a specialized lexicon for breast cancer clinical trial subject eligibility analysis

   https://doi.org/10.1177/1460458221989392  

   Jung, Euisung; Jain, Hemant; Sinha, Atish P; Gaudioso, Carmelo (January 2021, Health Informatics Journal)

   null (Ed.)

   A natural language processing (NLP) application requires sophisticated lexical resources to support its processing goals. Different solutions, such as dictionary lookup and MetaMap, have been proposed in the healthcare informatics literature to identify disease terms with more than one word (multi-gram disease named entities). Although a lot of work has been done in the identification of protein- and gene-named entities in the biomedical field, not much research has been done on the recognition and resolution of terminologies in the clinical trial subject eligibility analysis. In this study, we develop a specialized lexicon for improving NLP and text mining analysis in the breast cancer domain, and evaluate it by comparing it with the Systematized Nomenclature of Medicine Clinical Terms (SNOMED CT). We use a hybrid methodology, which combines the knowledge of domain experts, terms from multiple online dictionaries, and the mining of text from sample clinical trials. Use of our methodology introduces 4243 unique lexicon items, which increase bigram entity match by 38.6% and trigram entity match by 41%. Our lexicon, which adds a significant number of new terms, is very useful for matching patients to clinical trials automatically based on eligibility matching. Beyond clinical trial matching, the specialized lexicon developed in this study could serve as a foundation for future healthcare text mining applications. 

   more » « less