skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Autonomous Detection of Disruptions in the Intensive Care Unit Using Deep Mask R-CNN
Patients staying in the Intensive Care Unit (ICU) have a severely disrupted circadian rhythm. Due to patients' critical medical condition, ICU physicians and nurses have to provide round-the-clock clinical care, further disrupting patients' circadian rhythm. Mistimed family visits during rest-time can also disrupt patients' circadian rhythm. Currently, such effects are only reported based on hospital visitation policies rather than the actual number of visitors and care providers in the room. To quantify visitation disruptions, we used a deep Mask R-CNN model, a deep learning framework for object instance segmentation to detect and quantify the number of individuals in the ICU unit. This study represents the first effort to automatically quantify visitations in an ICU room, which could have implications in terms of policy adjustment, as well as circadian rhythm investigation. Our model achieved precision of 0.97 and recall of 0.67, with F1 score of 0.79 for detecting disruptions in the ICU units.  more » « less
Award ID(s):
1750192
PAR ID:
10088546
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Page Range / eLocation ID:
1944 to 19442
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Sleep has been shown to be an indispensable and important component of patients' recovery process. Nonetheless, the sleep quality of patients in the Intensive Care Unit (ICU) is often low, due to factors such as noise, pain, and frequent nursing care activities. Frequent sleep disruptions by the medical staff and/or visitors at certain times might lead to disruption of the patient's sleep-wake cycle and can also impact the severity of pain. Examining the association between sleep quality and frequent visitation has been difficult, due to the lack of automated methods for visitation detection. In this study, we recruited 38 patients to automatically assess visitation frequency from captured video frames. We used the DensePose R-CNN (ResNet-101) model to calculate the number of people in the room in a video frame. We examined when patients are interrupted the most, and we examined the association between frequent disruptions and patient outcomes on pain and length of stay. 
    more » « less
  2. Accurate prediction and monitoring of patient health in the intensive care unit can inform shared decisions regarding appropriateness of care delivery, risk-reduction strategies, and intensive care resource use. Traditionally, algorithmic solutions for patient outcome prediction rely solely on data available from electronic health records (EHR). In this pilot study, we explore the benefits of augmenting existing EHR data with novel measurements from wrist-worn activity sensors as part of a clinical environment known as the Intelligent ICU. We implemented temporal deep learning models based on two distinct sources of patient data: (1) routinely measured vital signs from electronic health records, and (2) activity data collected from wearable sensors. As a proxy for illness severity, our models predicted whether patients leaving the intensive care unit would be successfully or unsuccessfully discharged from the hospital. We overcome the challenge of small sample size in our prospective cohort by applying deep transfer learning using EHR data from a much larger cohort of traditional ICU patients. Our experiments quantify added utility of non-traditional measurements for predicting patient health, especially when applying a transfer learning procedure to small novel Intelligent ICU cohorts of critically ill patients. 
    more » « less
  3. Currently, many critical care indices are repetitively assessed and recorded by overburdened nurses, e.g. physical function or facial pain expressions of nonverbal patients. In addition, many essential information on patients and their environment are not captured at all, or are captured in a non-granular manner, e.g. sleep disturbance factors such as bright light, loud background noise, or excessive visitations. In this pilot study, we examined the feasibility of using pervasive sensing technology and artificial intelligence for autonomous and granular monitoring of critically ill patients and their environment in the Intensive Care Unit (ICU). As an exemplar prevalent condition, we also characterized delirious and non-delirious patients and their environment. We used wearable sensors, light and sound sensors, and a high-resolution camera to collected data on patients and their environment. We analyzed collected data using deep learning and statistical analysis. Our system performed face detection, face recognition, facial action unit detection, head pose detection, facial expression recognition, posture recognition, actigraphy analysis, sound pressure and light level detection, and visitation frequency detection. We were able to detect patient's face (Mean average precision (mAP)=0.94), recognize patient's face (mAP=0.80), and their postures (F1=0.94). We also found that all facial expressions, 11 activity features, visitation frequency during the day, visitation frequency during the night, light levels, and sound pressure levels during the night were significantly different between delirious and non-delirious patients (p-value<0.05). In summary, we showed that granular and autonomous monitoring of critically ill patients and their environment is feasible and can be used for characterizing critical care conditions and related environment factors. 
    more » « less
  4. Abstract Traditional methods for assessing illness severity and predicting in-hospital mortality among critically ill patients require time-consuming, error-prone calculations using static variable thresholds. These methods do not capitalize on the emerging availability of streaming electronic health record data or capture time-sensitive individual physiological patterns, a critical task in the intensive care unit. We propose a novel acuity score framework (DeepSOFA) that leverages temporal measurements and interpretable deep learning models to assess illness severity at any point during an ICU stay. We compare DeepSOFA with SOFA (Sequential Organ Failure Assessment) baseline models using the same model inputs and find that at any point during an ICU admission, DeepSOFA yields significantly more accurate predictions of in-hospital mortality. A DeepSOFA model developed in a public database and validated in a single institutional cohort had a mean AUC for the entire ICU stay of 0.90 (95% CI 0.90–0.91) compared with baseline SOFA models with mean AUC 0.79 (95% CI 0.79–0.80) and 0.85 (95% CI 0.85–0.86). Deep models are well-suited to identify ICU patients in need of life-saving interventions prior to the occurrence of an unexpected adverse event and inform shared decision-making processes among patients, providers, and families regarding goals of care and optimal resource utilization. 
    more » « less
  5. The purpose of the DREAMS project (DREAMS = Digital Rehabilitation Environment-Augmenting Medical System) is to research the feasibility and clinical potential of a virtual reality (VR) system for reducing the occurrence of delirium among patients in the intensive care unit (ICU). Preliminary results of this ongoing study show VR produces minimal clinical effects but are strongly enjoyed by patients and easy to administer. We discuss important lessons learned from applying VR in the ICU. 
    more » « less