skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Predictability and stability testing to assess clinical decision instrument performance for children after blunt torso trauma
Title: Predictability and stability testing to assess clinical decision instrument performance for children after blunt torso trauma
Objective The Pediatric Emergency Care Applied Research Network (PECARN) has developed a clinical-decision instrument (CDI) to identify children at very low risk of intra-abdominal injury. However, the CDI has not been externally validated. We sought to vet the PECARN CDI with the Predictability Computability Stability (PCS) data science framework, potentially increasing its chance of a successful external validation. Materials & methods We performed a secondary analysis of two prospectively collected datasets: PECARN (12,044 children from 20 emergency departments) and an independent external validation dataset from the Pediatric Surgical Research Collaborative (PedSRC; 2,188 children from 14 emergency departments). We used PCS to reanalyze the original PECARN CDI along with new interpretable PCS CDIs developed using the PECARN dataset. External validation was then measured on the PedSRC dataset. Results Three predictor variables (abdominal wall trauma, Glasgow Coma Scale Score <14, and abdominal tenderness) were found to be stable. A CDI using only these three variables would achieve lower sensitivity than the original PECARN CDI with seven variables on internal PECARN validation but achieve the same performance on external PedSRC validation (sensitivity 96.8% and specificity 44%). Using only these variables, we developed a PCS CDI which had a lower sensitivity than the original PECARN CDI on internal PECARN validation but performed the same on external PedSRC validation (sensitivity 96.8% and specificity 44%). Conclusion The PCS data science framework vetted the PECARN CDI and its constituent predictor variables prior to external validation. We found that the 3 stable predictor variables represented all of the PECARN CDI’s predictive performance on independent external validation. The PCS framework offers a less resource-intensive method than prospective validation to vet CDIs before external validation. We also found that the PECARN CDI will generalize well to new populations and should be prospectively externally validated. The PCS framework offers a potential strategy to increase the chance of a successful (costly) prospective validation.  more » « less
Award ID(s):
1741340
PAR ID:
10373935
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ;
Editor(s):
Li-Jessen, Nicole Yee-Key
Date Published:
Journal Name:
PLOS Digital Health
Volume:
1
Issue:
8
ISSN:
2767-3170
Page Range / eLocation ID:
e0000076
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; (, Journal of Medical Internet Research)
    null (Ed.)
    Background Conventional diagnosis of COVID-19 with reverse transcription polymerase chain reaction (RT-PCR) testing (hereafter, PCR) is associated with prolonged time to diagnosis and significant costs to run the test. The SARS-CoV-2 virus might lead to characteristic patterns in the results of widely available, routine blood tests that could be identified with machine learning methodologies. Machine learning modalities integrating findings from these common laboratory test results might accelerate ruling out COVID-19 in emergency department patients. Objective We sought to develop (ie, train and internally validate with cross-validation techniques) and externally validate a machine learning model to rule out COVID 19 using only routine blood tests among adults in emergency departments. Methods Using clinical data from emergency departments (EDs) from 66 US hospitals before the pandemic (before the end of December 2019) or during the pandemic (March-July 2020), we included patients aged ≥20 years in the study time frame. We excluded those with missing laboratory results. Model training used 2183 PCR-confirmed cases from 43 hospitals during the pandemic; negative controls were 10,000 prepandemic patients from the same hospitals. External validation used 23 hospitals with 1020 PCR-confirmed cases and 171,734 prepandemic negative controls. The main outcome was COVID 19 status predicted using same-day routine laboratory results. Model performance was assessed with area under the receiver operating characteristic (AUROC) curve as well as sensitivity, specificity, and negative predictive value (NPV). Results Of 192,779 patients included in the training, external validation, and sensitivity data sets (median age decile 50 [IQR 30-60] years, 40.5% male [78,249/192,779]), AUROC for training and external validation was 0.91 (95% CI 0.90-0.92). Using a risk score cutoff of 1.0 (out of 100) in the external validation data set, the model achieved sensitivity of 95.9% and specificity of 41.7%; with a cutoff of 2.0, sensitivity was 92.6% and specificity was 59.9%. At the cutoff of 2.0, the NPVs at a prevalence of 1%, 10%, and 20% were 99.9%, 98.6%, and 97%, respectively. Conclusions A machine learning model developed with multicenter clinical data integrating commonly collected ED laboratory data demonstrated high rule-out accuracy for COVID-19 status, and might inform selective use of PCR-based testing. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al (, npj Digital Medicine)
    Abstract The COVID-19 pandemic has challenged front-line clinical decision-making, leading to numerous published prognostic tools. However, few models have been prospectively validated and none report implementation in practice. Here, we use 3345 retrospective and 474 prospective hospitalizations to develop and validate a parsimonious model to identify patients with favorable outcomes within 96 h of a prediction, based on real-time lab values, vital signs, and oxygen support variables. In retrospective and prospective validation, the model achieves high average precision (88.6% 95% CI: [88.4–88.7] and 90.8% [90.8–90.8]) and discrimination (95.1% [95.1–95.2] and 86.8% [86.8–86.9]) respectively. We implemented and integrated the model into the EHR, achieving a positive predictive value of 93.3% with 41% sensitivity. Preliminary results suggest clinicians are adopting these scores into their clinical workflows. 
    more » « less
  3. ; ; ; ; ; ; ; ; ; ; et al (, Science Advances)
    The “red reflex test” is used to screen children for leukocoria (“white eye”) in a standard pediatric examination, but is ineffective at detecting many eye disorders. Leukocoria also presents in casual photographs. The clinical utility of screening photographs for leukocoria is unreported. Here, a free smartphone application (CRADLE: ComputeR-Assisted Detector of LEukocoria) was engineered to detect photographic leukocoria and is available for download under the name “White Eye Detector.” This study determined the sensitivity, specificity, and accuracy of CRADLE by retrospectively analyzing 52,982 longitudinal photographs of children, collected by parents before enrollment in this study. The cohort included 20 children with retinoblastoma, Coats’ disease, cataract, amblyopia, or hyperopia and 20 control children. For 80% of children with eye disorders, the application detected leukocoria in photographs taken before diagnosis by 1.3 years (95% confidence interval, 0.4 to 2.3 years). The CRADLE application allows parents to augment clinical leukocoria screening with photography. 
    more » « less
  4. ; ; ; (, The Journal of Organic Chemistry)
    N-Heterocyclic carbene-carbodiimide (NHC-CDI) adducts are versatile compounds that can be used as ligands and (pre)catalysts, but their systematic structure–property relationships are underexplored. Herein, we investigated how structural electronic variations on both the NHC and CDI affect the inherent kinetic and thermodynamic properties of the adducts. Using in situ carbene trapping and variable-temperature NMR spectroscopy, we measured the rates of dissociation and the equilibrium constants and then used Eyring and van’t Hoff analyses to calculate ΔG‡ and ΔG, respectively. Linear free-energy relationships indicate that changing the para position of the CDI substituents yields a similar effect to changing the NHC core. These CDI structural modifications affected the adducts’ thermodynamics (ΔG) more than the kinetics (ΔG‡) and were found to be influenced more by inductive, rather than resonance, factors. Preliminary results suggest a steric threshold beyond which steric effects dominate electronic effects in governing the strength of the adduct bond. This systematic investigation provides valuable insight into the design of NHC-CDIs for current and future applications. 
    more » « less
  5. ; ; ; ; ; ; (, Myoelectric Controls Symposium)
    Children with Unilateral Congenital Below-Elbow Deficiencies (born without a hand, UCBED) have a high rate of prosthetic abandonment, pointing to unresolved challenges that may be distinct from those faced by adults with limb loss. There is limited knowledge of the motor control these children have over their affected muscles, a highly relevant question for effective dextrous prosthetic control. Our research aims to measure the extent of volitional muscle activation that exists in the residuum when children attempt moving their missing hand, with the goal of creating highly functional pediatric-specific prosthetic devices. In this work, we recruited 28 pediatric UCBED patients across four Shriners Hospital locations. We measured muscle activity using ultrasound imaging and surface electromyography while children attempted 10 missing-hand movements, then used machine learning to analyze the patterns of the affected and unaffected sides. Our algorithms predicted hand movements from residual muscle activity at over 80% accuracy in most cases, and well above chance in all participants. This indicates inherent muscular control which may be leveraged to develop more functional prosthetic devices tailored towards pediatric UCBED patients. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email