An official website of the United States government
Camera-based heart rate measurement is becoming an attractive option as a non-contact modality for continuous remote health and engagement monitoring. However, reliable heart rate extraction from camera-based measurement is challenging in realistic scenarios, especially when the subject is moving. In this work, we develop a motion-robust algorithm, labeled RobustPPG, for extracting photoplethysmography signals (PPG) from face video and estimating the heart rate. Our key innovation is to explicitly model and generate motion distortions due to the movements of the person’s face. We use inverse rendering to obtain the 3D shape and albedo of the face and environment lighting from video frames and then render the human face for each frame. The rendered face is similar to the original face but does not contain the heart rate signal; facial movements alone cause pixel intensity variation in the generated video frames. Finally, we use the generated motion distortion to filter the motion-induced measurements. We demonstrate that our approach performs better than the state-of-the-art methods in extracting a clean blood volume signal with over 2 dB signal quality improvement and 30% improvement in RMSE of estimated heart rate in intense motion scenarios.
more »
« less
StressNet: Detecting Stress in Thermal Videos
Precise measurement of physiological signals is critical for the effective monitoring of human vital signs. Recent developments in computer vision have demonstrated that signals such as pulse rate and respiration rate can be extracted from digital video of humans, increasing the possibility of contact-less monitoring. This paper presents a novel approach to obtaining physiological signals and classifying stress states from thermal video. The proposed network–”StressNet”–features a hybrid emission representation model that models the direct emission and absorption of heat by the skin and underlying blood vessels. This results in an information-rich feature representation of the face, which is used by spatio-temporal network for reconstructing the ISTI ( Initial Systolic Time Interval : a measure of change in cardiac sympathetic activity that is considered to be a quantitative index of stress in humans). The reconstructed ISTI signal is fed into a stress-detection model to detect and classify the individual’s stress state (i.e. stress or no stress). A detailed evaluation demonstrates that StressNet achieves estimated the ISTI signal with 95% accuracy and detect stress with average precision of 0.842.
more »
« less
- Award ID(s):
- 1664172
- PAR ID:
- 10296049
- Date Published:
- Journal Name:
- 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
- Page Range / eLocation ID:
- 998 to 1008
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Longitudinal fetal health monitoring is essential for high-risk pregnancies. Heart rate and heart rate variability are prime indicators of fetal health. In this work, we implemented two neural network architectures for heartbeat detection on a set of fetal phonocardiogram signals captured using fetal Doppler and a digital stethoscope. We test the efficacy of these networks using the raw signals and the hand-crafted energy from the signal. The results show a Convolutional Neural Network is the most efficient at identifying the S1 waveforms in a heartbeat, and its performance is improved when using the energy of the Doppler signals. We further discuss issues, such as low Signal-to-Noise Ratios (SNR), present in the training of a model based on the stethoscope signals. Finally, we show that we can improve the SNR, and subsequently the performance of the stethoscope, by matching the energy from the stethoscope to that of the Doppler signal.more » « less
-
Prior research has validated Photoplethysmography (PPG) as a promising biomarker for assessing stress factors in construction workers, including physical fatigue, mental stress, and heat stress. However, the reliability of PPG as a stress biomarker in construction workers is hindered by motion artifacts (MA) - distortions in blood volume pulse measurements caused by sensor movement. This paper develops a deep convolutional autoencoder-based framework, trained to detect and reduce MA in MA-contaminated PPG signals. The framework's performance is evaluated using PPG signals acquired from individuals engaged in specific construction tasks. The results demonstrate the framework has effectiveness in both detecting and reducing MA in PPG signals with a detection accuracy of 93% and improvement in signal-to-noise ratio by over 88%. This research contributes to a more reliable and error-reduced usage of PPG signals for health monitoring in the construction industry.more » « less
-
Heating, ventilation and air-conditioning (HVAC) systems have been adopted to create comfortable, healthy and safe indoor environments. In the control loop, the technical feature of the human demand-oriented supply can help operate HVAC effectively. Among many technical options, real time monitoring based on feedback signals from end users has been frequently reported as a critical technology to confirm optimizing building performance. Recent studies have incorporated human thermal physiology signals and thermal comfort/discomfort status as real-time feedback signals. A series of human subject experiments used to be conducted by primarily adopting subjective questionnaire surveys in a lab-setting study, which is limited in the application for reality. With the help of advanced technologies, physiological signals have been detected, measured and processed by using multiple technical formats, such as wearable sensors. Nevertheless, they mostly require physical contacts with the skin surface in spite of the small physical dimension and compatibility with other wearable accessories, such as goggles, and intelligent bracelets. Most recently, a low cost small infrared camera has been adopted for monitoring human facial images, which could detect the facial skin temperature and blood perfusion in a contactless way. Also, according to latest pilot studies, a conventional digital camera can generate infrared images with the help of new methods, such as the Euler video magnification technology. Human thermal comfort/discomfort poses can also be detected by video methods without contacting human bodies and be analyzed by the skeleton keypoints model. In this review, new sensing technologies were summarized, their cons and pros were discussed, and extended applications for the demand-oriented ventilation were also reviewed as potential development and applications.more » « less
-
Physiological and kinematic signals from humans are often used for monitoring health. Several processes of interest (e.g., cardiac and respiratory processes, and locomotion) demonstrate periodicity. Training models for inference on these signals (e.g., detection of anomalies, and extraction of biomarkers) require large amounts of data to capture their variability, which are not readily available. This hinders the performance of complex inference models. In this work, we introduce a methodology for improving inference on such signals by incorporating phase-based interpretability and other inference tasks into a multi-task framework applied to a generative model. For this purpose, we utilize phase information as a regularization term and as an input to the model and introduce an interpretable unit in a neural network, which imposes an interpretable structure on the model. This imposition helps us in the smooth generation of periodic signals that can aid in data augmentation tasks. We demonstrate the impact of our framework on improving the overall inference performance on ECG signals and inertial signals from gait locomotion.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/WACV48630.2021.00104
