skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Use of Patient-Reported Symptom Data in Clinical Decision Rules for Predicting Influenza in a Telemedicine Setting
Introduction: Increased use of telemedicine could potentially streamline influenza diagnosis and reduce transmission. However, telemedicine diagnoses are dependent on accurate symptom reporting by patients. If patients disagree with clinicians on symptoms, previously derived diagnostic rules may be inaccurate. Methods: We performed a secondary data analysis of a prospective, nonrandomized cohort study at a university student health center. Patients who reported an upper respiratory complaint were required to report symptoms, and their clinician was required to report the same list of symptoms. We examined the performance of 5 previously developed clinical decision rules (CDRs) for influenza on both symptom reports. These predictions were compared against PCR diagnoses. We analyzed the agreement between symptom reports, and we built new predictive models using both sets of data. Results: CDR performance was always lower for the patient-reported symptom data, compared with clinician-reported symptom data. CDRs often resulted in different predictions for the same individual, driven by disagreement in symptom reporting. We were able to fit new models to the patient-reported data, which performed slightly worse than previously derived CDRs. These models and models built on clinician-reported data both suffered from calibration issues. Discussion: Patients and clinicians frequently disagree about symptom presence, which leads to reduced accuracy when CDRs built with clinician data are applied to patient-reported symptoms. Predictive models using patient-reported symptom data performed worse than models using clinicianreported data and prior results in the literature. However, the differences are minor, and developing new models with more data may be possible. ( J Am Board Fam Med 2023;00:000–000.)  more » « less
Award ID(s):
1659683
PAR ID:
10512075
Author(s) / Creator(s):
; ; ; ; ; ;
Publisher / Repository:
American Board of Family Medicine
Date Published:
Journal Name:
The Journal of the American Board of Family Medicine
Volume:
36
Issue:
5
ISSN:
1557-2625
Page Range / eLocation ID:
766 to 776
Subject(s) / Keyword(s):
Clinical Decision Rules, Cohort Studies, Infectious Diseases, Influenza, Prospective Studies, Respiratory Tract Diseases, Students, Telemedicine, Triage
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Patient-generated data, such as recorded Observations of Daily Living (ODL) and Patient-Reported Outcomes (PRO) data, are valued sources of information in oncology care. However, prior work largely focuses on capturing clinician-defined, patient-generated data in adult oncology care. Emerging research at the intersection of human–computer interaction and medical informatics suggests that visual narratives of patients’ observations of daily living (Visual ODLs) could better support multi-party review of patients’ everyday symptoms and quality of life, potentially improving patient–clinician communication. In this paper, we build on a prior study of Visual ODLs by describing a formative, two-phase study with 15 pediatric oncology clinicians. In Phase I, we analyzed data from ethnographic interviews in a pediatric oncology setting to capture the needs of nurses, nurse practitioners, and oncologists. In Phase II, we constructed two low-fidelity dashboard display prototypes, populated with Visual ODLs contributed by actual adolescent oncology patients, and we subsequently interviewed pediatric oncology clinicians who reviewed each dashboard design. Findings from our study contribute four key design objectives for presenting interactive Visual ODL dashboards in pediatric oncology, along with three use cases for using these dashboards for symptom tracking and communication. 
    more » « less
  2. null (Ed.)
    Abstract Heterogeneity in the clinical presentation of major depressive disorder and response to antidepressants limits clinicians’ ability to accurately predict a specific patient’s eventual response to therapy. Validated depressive symptom profiles may be an important tool for identifying poor outcomes early in the course of treatment. To derive these symptom profiles, we first examined data from 947 depressed subjects treated with selective serotonin reuptake inhibitors (SSRIs) to delineate the heterogeneity of antidepressant response using probabilistic graphical models (PGMs). We then used unsupervised machine learning to identify specific depressive symptoms and thresholds of improvement that were predictive of antidepressant response by 4 weeks for a patient to achieve remission, response, or nonresponse by 8 weeks. Four depressive symptoms (depressed mood, guilt feelings and delusion, work and activities and psychic anxiety) and specific thresholds of change in each at 4 weeks predicted eventual outcome at 8 weeks to SSRI therapy with an average accuracy of 77% ( p  = 5.5E-08). The same four symptoms and prognostic thresholds derived from patients treated with SSRIs correctly predicted outcomes in 72% ( p  = 1.25E-05) of 1996 patients treated with other antidepressants in both inpatient and outpatient settings in independent publicly-available datasets. These predictive accuracies were higher than the accuracy of 53% for predicting SSRI response achieved using approaches that (i) incorporated only baseline clinical and sociodemographic factors, or (ii) used 4-week nonresponse status to predict likely outcomes at 8 weeks. The present findings suggest that PGMs providing interpretable predictions have the potential to enhance clinical treatment of depression and reduce the time burden associated with trials of ineffective antidepressants. Prospective trials examining this approach are forthcoming. 
    more » « less
  3. Patient-Reported Outcomes (PRO) are collected directly from the patients using symptom questionnaires. In the case of head and neck cancer patients, PRO surveys are recorded every week during treatment with each patient’s visit to the clinic and at different follow-up times after the treatment has concluded. PRO surveys can be very informative regarding the patient’s status and the effect of treatment on the patient’s quality of life (QoL). Processing PRO data is challenging for several reasons. First, missing data is frequent as patients might skip a question or a questionnaire altogether. Second, PROs are patient-dependent, a rating of 5 for one patient might be a rating of 10 for another patient. Finally, most patients experience severe symptoms during treatment which usually subside over time. However, for some patients, late toxicities persist negatively affecting the patient’s QoL. These long-term severe symptoms are hard to predict and are the focus of this study. In this work, we model PRO data collected from head and neck cancer patients treated at the MD Anderson Cancer Center using the MD Anderson Symptom Inventory (MDASI) questionnaire as time series. We impute missing values with a combination of K nearest neighbor (KNN) and Long Short-Term Memory (LSTM) neural networks, and finally, apply LSTM to predict late symptom severity 12 months after treatment. We compare performance against clinical and ARIMA models. We show that the LSTM model combined with KNN imputation is effective in predicting late-stage symptom ratings for occurrence and severity under the AUC and F1 score metrics. 
    more » « less
  4. Abstract Schizophrenia is a severe and complex psychiatric disorder with heterogeneous and dynamic multi-dimensional symptoms. Behavioral rhythms, such as sleep rhythm, are usually disrupted in people with schizophrenia. As such, behavioral rhythm sensing with smartphones and machine learning can help better understand and predict their symptoms. Our goal is to predict fine-grained symptom changes with interpretable models. We computed rhythm-based features from 61 participants with 6,132 days of data and used multi-task learning to predict their ecological momentary assessment scores for 10 different symptom items. By taking into account both the similarities and differences between different participants and symptoms, our multi-task learning models perform statistically significantly better than the models trained with single-task learning for predicting patients’ individual symptom trajectories, such as feeling depressed, social, and calm and hearing voices. We also found different subtypes for each of the symptoms by applying unsupervised clustering to the feature weights in the models. Taken together, compared to the features used in the previous studies, our rhythm features not only improved models’ prediction accuracy but also provided better interpretability for how patients’ behavioral rhythms and the rhythms of their environments influence their symptom conditions. This will enable both the patients and clinicians to monitor how these factors affect a patient’s condition and how to mitigate the influence of these factors. As such, we envision that our solution allows early detection and early intervention before a patient’s condition starts deteriorating without requiring extra effort from patients and clinicians. 
    more » « less
  5. Personalized head and neck cancer therapeutics have greatly improved survival rates for patients, but are often leading to understudied long-lasting symptoms which affect quality of life. Sequential rule mining (SRM) is a promising unsupervised machine learning method for predicting longitudinal patterns in temporal data which, however, can output many repetitive patterns that are difficult to interpret without the assistance of visual analytics. We present a data-driven, human-machine analysis visual system developed in collaboration with SRM model builders in cancer symptom research, which facilitates mechanistic knowledge discovery in large scale, multivariate cohort symptom data. Our system supports multivariate predictive modeling of post-treatment symptoms based on during-treatment symptoms. It supports this goal through an SRM, clustering, and aggregation back end, and a custom front end to help develop and tune the predictive models. The system also explains the resulting predictions in the context of therapeutic decisions typical in personalized care delivery. We evaluate the resulting models and system with an interdisciplinary group of modelers and head and neck oncology researchers. The results demonstrate that our system effectively supports clinical and symptom research. 
    more » « less