An official website of the United States government
Basic vital signs such as heart and respiratory rates (HR and RR) are essential bio-indicators. Their longitudinal in-home collection enables prediction and detection of disease onset and change, providing for earlier health intervention. In this paper, we propose a robust, non-touch vital signs monitoring system using a pair of co-located Ultra-Wide Band (UWB) and depth sensors. By extensive manual examination, we identify four typical temporal and spectral signal patterns and their suitable vital signs estimators. We devise a probabilistic weighted framework (PWF) that quantifies evidence of these patterns to update the weighted combination of estimator output to track the vital signs robustly. We also design a “heatmap” based signal quality detector to exclude the disturbed signal from inadvertent motions. To monitor multiple co-habiting subjects in-home, we build a two-branch long short-term memory (LSTM) neural network to distinguish between individuals and their activities, providing activity context crucial to disambiguating critical from normal vital sign variability. To achieve reliable context annotation, we carefully devise the feature set of the consecutive skeletal poses from the depth data, and develop a probabilistic tracking model to tackle non-line-of-sight (NLOS) cases. Our experimental results demonstrate the robustness and superior performance of the individual modules as well as the end-to-end system for passive and context-aware vital signs monitoring.
more »
« less
DeepVS: A Deep Learning Approach For RF-based Vital Signs Sensing
Vital signs (e.g., heart and respiratory rate) are indicative for health status assessment. Efforts have been made to extract vital signs using radio frequency (RF) techniques (e.g., Wi-Fi, FMCW, UWB), which offer a non-touch solution for continuous and ubiquitous monitoring without users’ cooperative efforts. While RF-based vital signs monitoring is user-friendly, its robustness faces two challenges. On the one hand, the RF signal is modulated by the periodic chest wall displacement due to heartbeat and breathing in a nonlinear manner. It is inherently hard to identify the fundamental heart and respiratory rates (HR and RR) in the presence of higher order harmonics of them and intermodulation between HR and RR, especially when they have overlapping frequency bands. On the other hand, the inadvertent body movements may disturb and distort the RF signal, overwhelming the vital signals, thus inhibiting the parameter estimation of the physiological movement (i.e., heartbeat and breathing). In this paper, we propose DeepVS, a deep learning approach that addresses the aforementioned challenges from the non-linearity and inadvertent movements for robust RF-based vital signs sensing in a unified manner. DeepVS combines 1D CNN and attention models to exploit local features and temporal correlations. Moreover, it leverages a two-stream scheme to integrate features from both time and frequency domains. Additionally, DeepVS unifies the estimation of HR and RR with a multi-head structure, which only adds limited extra overhead (<1%) to the existing model, compared to doubling the overhead using two separate models for HR and RR respectively. Our experiments demonstrate that DeepVS achieves 80-percentile HR/RR errors of 7.4/4.9 beat/breaths per minute (bpm) on a challenging dataset, as compared to 11.8/7.3 bpm of a non-learning solution. Besides, an ablation study has been conducted to quantify the effectiveness of DeepVS.
more »
« less
- Award ID(s):
- 2119299
- PAR ID:
- 10356921
- Date Published:
- Journal Name:
- In 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics (BCB ’22)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Monitoring respiratory rate (RR) is essential for early identification of respiratory and metabolic abnormalities. However, the limitations of contact-based sensors and the lack of reliability in many contactless methods make continuous and accurate monitoring difficult in non-clinical settings. To address these challenges, we introduce RespFormer, an edgeoptimized, motion-guided temporal-frequency multimodal fusion transformer framework for real-time, contactless RR estimation and breathing pattern classification. RespFormer integrates dense optical flow analysis with temporal, statistical, and frequencydomain features derived from video sequences and enhances them through a multi-stage signal processing pipeline. These features are modeled using an ensemble of three time series transformer architectures (ETSformer, Temporal Fusion Transformer, and Informer) to capture distinct aspects of temporal dynamics. A shared attention-based refinement module enhances the feature representations, and final predictions are fused using a stackingbased meta-learner. We validate RespFormer on a multimodal dataset comprising synchronized RGB, NIR, and IR video data, including a custom in-house dataset captured under various conditions. Experimental results demonstrate that RespFormer achieves a mean absolute error (MAE) ≈ 0.98 bpm, improving prediction accuracy by ≈ 11% and reducing memory usage by ≈ 26%, while maintaining real-time inference (≈ 1.22 seconds) on resource-constrained devices. Furthermore, RespFormer accurately classifies breathing patterns (normal, bradypnea, tachypnea, and apnea) with 95% accuracy, underscoring it’s potential for practical application in telemedicine, clinical screening, and low-resource healthcare settings.more » « less
-
Baba, Justin S; Coté, Gerard L (Ed.)In this research, we examine the potential of measuring physiological variables, including heart rate (HR) and respiration rate (RR) on the upper arm using a wireless multimodal sensing system consisting of an accelerometer, a gyroscope, a three-wavelength photoplethysmography (PPG), single-sided electrocardiography (SS-ECG), and bioimpedance (BioZ). The study included collecting HR data when the subject was at rest and typing, and RR data when the subject was at rest. The data from three wavelengths of PPG and BioZ were collected and compared to the SS-ECG as the standard. The accelerometer and gyro signals were used to exclude data with excessive noise due to motion. The results showed that when the subject remained sedentary, the mean absolute error (MAE) for the HR calculation for all three wavelengths of the PPG modality was less than two bpm, while the BioZ was 3.5 bpm compared with SS-ECG HR. The MAE for typing increased for both modalities and was less than three bpm for all three wavelengths of the PPG but increased to 7.5 bpm for the BioZ. Regarding RR, both modalities resulted in RR within one breath per minute of the SS-ECG modality for the one breathing rate. Overall, all modalities on this upper arm wearable worked well when the subject was sedentary. Still, the SS-ECG and PPG showed less variability for the HR signal in the presence of motion during micro-motions such as typing.more » « less
-
Non-contact monitoring videos capture subtle respiratory-induced motions, yet existing methods primarily focus on estimating respiratory rate (RR), neglecting the extraction of respiratory waveforms -- a vital parameter that provides critical health information. We formulate video-based RR estimation as a Tracking All Points (TAP) problem and propose a coarse-to-fine, multi-frame Persistent Independent Particle (RRPIPs) framework for robust, multi-modal (RGB, NIR, IR) RR waveform estimation. Addressing the challenge of tracking minute, non-rigid pixel displacements caused by respiratory motions, our top-down approach magnifies respiratory motion using phase-based video magnification tuned to the respiratory frequency range and employs a pretrained RAFT optical flow model for initial region identification via a two-frame analysis. Coarsescale tracking is performed using the RRPIPs model, while a Signal Quality Index (SQI) block evaluates the SNR of trajectories to refine high-respiratory-activity regions. These regions are upsampled, and fine-scale tracking is applied to extract precise waveforms. We curated a large-scale multimodal dataset for respiratory point tracking, combining in-house collected (MPSC-RR) and public datasets, with dense annotations of non-rigid pixel movements across multiple scales in key respiratory regions. Experimental results demonstrate that our framework achieves state-of-the-art accuracy (∼1 MAE) and interpretability in respiratory waveform extraction across RGB, NIR, and IR modalities, effectively addressing multi-scale tracking and low-SNR challenges. Thorough ablation studies validate the contributions of each framework component, and we open-source our codes and dataset to support further research.more » « less
-
Non-contact vital signs monitoring (NCVSM) with radio frequency (RF) is attracting increasing attention due to its non-invasive nature. Recent advances in COTS radar technologies accelerate the development of RF-based solutions. While researchers have implemented and demonstrated the feasibility of NCVSM with diverse radar hardware, most efforts have been focused on devising algorithms to extract vital signs, with limited understanding about the effects of radar configurations. The deficiency in such understanding hinders the design of software defined radar (SDR) optimally customized for NCVSM. In this work, we first hypothesize the effects of FMCW radar configurations using signal-to-interference-plus-noise ratio (SINR) based signal modeling, then we conduct extensive experiments with a COTS FMCW radar, TinyRad, to understand how various parameters impact NCVSM performance compared to a medical device. We find that a larger bandwidth or higher transmitting power in general improves vital sign estimation accuracy; however, coherent processing of consecutive chirps (time diversity) or multiple receiving antennas (space diversity) does not improve the performance. Observations on the baseband (BB) signal show that coherent processing contributes to a higher amplitude but similar phase patterns, whose periodic changes are the key in extracting vital signs.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- https://doi.org/10.1145/3535508.3545554
