skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Predicting inpatient pharmacy order interventions using provider action data
Abstract Objective The widespread deployment of electronic health records (EHRs) has introduced new sources of error and inefficiencies to the process of ordering medications in the hospital setting. Existing work identifies orders that require pharmacy intervention by comparing them to a patient’s medical records. In this work, we develop a machine learning model for identifying medication orders requiring intervention using only provider behavior and other contextual features that may reflect these new sources of inefficiencies. Materials and Methods Data on providers’ actions in the EHR system and pharmacy orders were collected over a 2-week period in a major metropolitan hospital system. A classification model was then built to identify orders requiring pharmacist intervention. We tune the model to the context in which it would be deployed and evaluate global and local feature importance. Results The resultant model had an area under the receiver-operator characteristic curve of 0.91 and an area under the precision-recall curve of 0.44. Conclusions Providers’ actions can serve as useful predictors in identifying medication orders that require pharmacy intervention. Careful model tuning for the clinical context in which the model is deployed can help to create an effective tool for improving health outcomes without using sensitive patient data.  more » « less
Award ID(s):
1928614
PAR ID:
10333159
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
JAMIA Open
Volume:
4
Issue:
3
ISSN:
2574-2531
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; (, PLOS Digital Health)
    Pollard, Tom J. (Ed.)
    Modern predictive models require large amounts of data for training and evaluation, absence of which may result in models that are specific to certain locations, populations in them and clinical practices. Yet, best practices for clinical risk prediction models have not yet considered such challenges to generalizability. Here we ask whether population- and group-level performance of mortality prediction models vary significantly when applied to hospitals or geographies different from the ones in which they are developed. Further, what characteristics of the datasets explain the performance variation? In this multi-center cross-sectional study, we analyzed electronic health records from 179 hospitals across the US with 70,126 hospitalizations from 2014 to 2015. Generalization gap, defined as difference between model performance metrics across hospitals, is computed for area under the receiver operating characteristic curve (AUC) and calibration slope. To assess model performance by the race variable, we report differences in false negative rates across groups. Data were also analyzed using a causal discovery algorithm “Fast Causal Inference” that infers paths of causal influence while identifying potential influences associated with unmeasured variables. When transferring models across hospitals, AUC at the test hospital ranged from 0.777 to 0.832 (1st-3rd quartile or IQR; median 0.801); calibration slope from 0.725 to 0.983 (IQR; median 0.853); and disparity in false negative rates from 0.046 to 0.168 (IQR; median 0.092). Distribution of all variable types (demography, vitals, and labs) differed significantly across hospitals and regions. The race variable also mediated differences in the relationship between clinical variables and mortality, by hospital/region. In conclusion, group-level performance should be assessed during generalizability checks to identify potential harms to the groups. Moreover, for developing methods to improve model performance in new environments, a better understanding and documentation of provenance of data and health processes are needed to identify and mitigate sources of variation. 
    more » « less
  2. ; ; (, IUI '21: 26th International Conference on Intelligent User Interfaces)
    null (Ed.)
    Nearly half of people prescribed medication to treat chronic or short-term conditions do not take their medicine as prescribed. This leads to worse treatment outcomes, higher hospital admission rates, increased healthcare costs, and increased morbidity and mortality rates. While some instances of medication non-adherence are a result of problems with the treatment plan or barriers caused by the health care provider, many are instances caused by patient-related factors such as forgetting, running out of medication, and not understanding the required dosages. This presents a clear need for patient-centered systems that can reliably increase medication adherence. To that end, in this work we describe an activity recognition system capable of recognizing when individuals take medication in an unconstrained, real-world environment. Our methodology uses a modified version of the Bagging ensemble method to suit unbalanced data and a classifier trained on the prediction probabilities of the Bagging classifier to identify when individuals took medication during a full-day study. Using this methodology we are able to recognize when individuals took medication with an F-measure of 0.77. Our system is a first step towards developing personal health interfaces that are capable of providing personalized medication adherence interventions. 
    more » « less
  3. ; ; ; ; (, MIS quarterly)
    null (Ed.)
    Hospitalization of patients with chronic diseases poses a significant burden on the healthcare system. Frequent hospitalization can be partially attributed to the failure of healthcare providers to engage effectively with their patients. Recently, patient portals have become popular as information technology (IT) platforms that provide patients with online access to their medical records and help them engage effectively with healthcare providers. Despite the popularity of these portals, there is a paucity of research on the impact of patient–provider engagement on patients’ health outcomes. Drawing on the theory of effective use, we examine the association between portal use and the incidence of subsequent patient hospitalizations, based on a unique, longitudinal dataset of patients’ portal use, across a 12-year period at a large academic medical center in North Texas. Our results indicate that portal use is associated with improvements in patient health outcomes along multiple dimensions, including the frequency of hospital and emergency visits, readmission risk, and length of stay. This is one of the first studies to conduct a large-scale, longitudinal analysis of a health IT system and its effect on individual level health outcomes. Our results highlight the need for technologies that can improve patient–provider engagement and improve overall health outcomes for chronic disease management. 
    more » « less
  4. ; ; ; ; ; ; ; ; ; (, Proceedings of the 10th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics)
    Unplanned intensive care units (ICU) readmissions and in-hospital mortality of patients are two important metrics for evaluating the quality of hospital care. Identifying patients with higher risk of readmission to ICU or of mortality can not only protect those patients from potential dangers, but also reduce the high costs of healthcare. In this work, we propose a new method to incorporate information from the Electronic Health Records (EHRs) of patients and utilize hyperbolic embeddings of a medical ontology (i.e., ICD-9) in the prediction model. The results prove the effectiveness of our method and show that hyperbolic embeddings of ontological concepts give promising performance. 
    more » « less
  5. ; ; ; ; ; ; ; ; ; (, JMIR Cancer)
    Abstract BackgroundCancer-care complexity heightens communication challenges between health care providers and patients, impacting their treatment adherence. This is especially evident upon hospital discharge in patients undergoing surgical procedures. Digital health tools offer potential solutions to address communication challenges seen in current discharge protocols. We aim to explore the usability and acceptability of an interactive health platform among discharged patients who underwent oncology-related procedures. MethodsA 4-week exploratory cohort study was conducted. Following hospital discharge, a tablet equipped with an integrated Personal Virtual Assistant (PVA) system was provided to patients who underwent oncology-related procedures. The PVA encompasses automated features that provide personalized care plans, developed through collaboration among clinicians, researchers, and engineers from various disciplines. These plans include guidance on daily specific assignments that were divided into 4 categories: medication intake, exercise,symptom surveys, and postprocedural specific tasks. The aim was to explore the acceptability of the PVA by quantification of dropout rate and assessing adherence to each care plan category throughout the study duration. The secondary aim assessed acceptability of the PVA through a technology acceptance model (TAM) questionnaire that examined ease of use, usefulness, attitude toward use, and privacy concerns. ResultsIn total, 17 patients were enrolled. However, 1 (5.8%) patient dropped out from the study after 3 days due to health deterioration, leaving 16/17 (94.2%) completing the study (mean age 54.5, SD 12.7, years; n=9, 52% Caucasian; n=14, 82% with a gynecological disease; n=3, 18% with a hepatobiliary disease). At the study end point, adherence to care plan categories were 78% (SD 25%) for medications, 81% (SD 24%) for exercises, 61% (SD 30%) for surveys, and 58% (SD 44%) for specific tasks such as following step-by step wound care instructions, managing drains, administering injectable medications independently, and performing pelvic baths as instructed. There was an 80% patient endorsement (strongly agree or agree) across all TAM categories. ConclusionThis study suggests the potential acceptability of the PVA among patients discharged after oncology-related procedures, with a dropout rate of less than 6% and fair-to-good adherence to tasks such as medication intake and exercise. However, these findings are preliminary due to the small sample size and highlight the need for further research with larger cohorts to validate and refine the system. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email