An official website of the United States government
Classification of clinical alarms is at the heart of prioritization, suppression, integration, postponement, and other methods of mitigating alarm fatigue. Since these methods directly affect clinical care, alarm classifiers, such as intelligent suppression systems, need to be evaluated in terms of their sensitivity and specificity, which is typically calculated on a labeled dataset of alarms. Unfortunately, the collection and particularly labeling of such datasets requires substantial effort and time, thus deterring hospitals from investigating mitigations of alarm fatigue. This article develops a lightweight method for evaluating alarm classifiers without perfect alarm labels. The method relies on probabilistic labels obtained from data programming—a labeling paradigm based on combining noisy and cheap-to-obtain labeling heuristics. Based on these labels, the method produces confidence bounds for the sensitivity/specificity values from a hypothetical evaluation with manual labeling. Our experiments on five alarm datasets collected at Children’s Hospital of Philadelphia show that the proposed method provides accurate bounds on the classifier’s sensitivity/specificity, appropriately reflecting the uncertainty from noisy labeling and limited sample sizes.
more »
« less
High-Confidence Data Programming for Evaluating Suppression of Physiological Alarms
False alarms generated by physiological monitors can overwhelm clinical caretakers with a variety of alarms. The resulting alarm fatigue can be mitigated with alarm suppression. Before being deployed, such suppression mechanisms need to be evaluated through a costly observational study, which would determine and label the truly suppressible alarms. This paper proposes a lightweight method for evaluating alarm suppression without access to the true alarm labels. The method is based on the data programming paradigm, which combines noisy and cheap-to-obtain labeling heuristics into probabilistic labels. Based on these labels, the method estimates the sensitivity/specificity of a suppression mechanism and describes the likely outcomes of an observational study in the form of confidence bounds. We evaluate the proposed method in a case study of low SpO2 alarms using a dataset collected at Children's Hospital of Philadelphia and show that our method provides tight and accurate bounds that significantly outperform the naive comparative method.
more »
« less
- Award ID(s):
- 1915063
- PAR ID:
- 10343417
- Date Published:
- Journal Name:
- 2021 IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE)
- Page Range / eLocation ID:
- 70 to 81
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
High false alarm rate in intensive care units (ICUs) has been identified as one of the most critical medical challenges in recent years. This often results in overwhelming the clinical staff by numerous false or unurgent alarms and decreasing the quality of care through enhancing the probability of missing true alarms as well as causing delirium, stress, sleep deprivation and depressed immune systems for patients. One major cause of false alarms in clinical practice is that the collected signals from different devices are processed individually to trigger an alarm, while there exists a considerable chance that the signal collected from one device is corrupted by noise or motion artifacts. In this paper, we propose a low-computational complexity yet accurate game-theoretic feature selection method which is based on a genetic algorithm that identifies the most informative biomarkers across the signals collected from various monitoring devices and can considerably reduce the rate of false alarms.more » « less
-
State of Science in Alarm System Safety: Implications for Researchers, Vendors, and Clinical LeadersAbstract Alarm fatigue is a complex phenomenon that needs to be assessed within the context of the clinical setting. Considering that complexity, the available information on how to address alarm fatigue and improve alarm system safety is relatively scarce. This article summarizes the state of science in alarm system safety based on the eight dimensions of a sociotechnical model for studying health information technology in complex adaptive healthcare systems. The summary and recommendations were guided by available systematic reviews on the topic, interventional studies published between January 2019 and February 2022, and recommendations and evidence-based practice interventions published by professional organizations. The current article suggests implications to help researchers respond to the gap in science related to alarm safety, help vendors design safe monitoring systems, and help clinical leaders apply evidence-based strategies to improve alarm safety in their settings. Physiologic monitors in intensive care units—the devices most commonly used in complex care environments and associated with the highest number of alarms and deaths—are the focus of the current work.more » « less
-
GPUs are progressively being integrated into modern society, playing a pivotal role in Artificial Intelligence and High-Performance Computing. Programmers need a deep understanding of the GPU programming model to avoid subtle data-races in their codes. Static verification that is sound and incomplete can guarantee data-race freedom, but the alarms it raises may be spurious and need to be validated. In this paper, we establish a True Positive Theorem for a static data-race detector for GPU programs, i.e., a result that identifies a class of programs for which our technique only raises true alarms. Our work builds on the formalism of memory access protocols, that models the concurrency operations of CUDA programs. The crux of our approach is an approximation analysis that can correctly identify true alarms, and pinpoint the conditions that make an alarm imprecise. Our approximation analysis detects when the reported locations are reachable (control independence, or CI), and when the reported locations are precise (data independence, or DI), as well identify inexact values in an alarm. In addition to a True Positive result for programs that are CI and DI, we establish the root causes of spurious alarms depending on whether CI or DI are present. We apply our theory to introduce FaialAA, the first sound and partially complete data-race detector. We evaluate FaialAA in three experiments. First, in a comparative study with the state-of-the-art tools, we show that FaialAA confirms more DRF programs than others while emitting 1.9× fewer potential alarms. Importantly, the approximation analysis of FaialAA detects 10 undocumented data-races. Second, in an experiment studying 6 commits of data-race fixes in open source projects OpenMM and Nvidia’s MegaTron, FaialAA confirmed the buggy and fixed versions of 5 commits, while others were only able to confirm 2. Third, we show that 59.5% of 2,770 programs are CI and DI, quantifying when the approximation analysis of FaialAA is complete. This paper is accompanied by the mechanized proofs of the theoretical results presented therein and a tool (FaialAA) implementing of our theory.more » « less
-
The Open ROADM ecosystem enables greater flexibility in deploying optical networks through centralized SDN management and intelligent service orchestration. However, since the maintenance signaling is yet to be standardized and implemented within each device, it is difficult to identify the root cause of issues and manage the network effectively when faults propagate in the network. To tackle this problem, this paper presents a proof-of-concept implementation of an alarm correlation and ticketing system for the multi-vendor Open ROADM ecosystem. The proposed system uses a graph-based method to identify the root cause of alarms and generate tickets for network operations teams. The results of our laboratory tests demonstrate the effectiveness of the proposed system in managing alarms in the multi-vendor Open ROADM ecosystem.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/CHASE52844.2021.00015
