More Like this

1. Unsupervised Domain Adaptation for Action Recognition via Self-Ensembling and Conditional Embedding Alignment

   https://doi.org/10.1109/ICDM59182.2024.00079  

   Ghosh, Indrajeet; Chugh, Garvit; Md_Faridee, Abu Zaher; Roy, Nirmalya (December 2024, IEEE)

   Recent advancements in deep learning-based wearable human action recognition (wHAR) have improved the capture and classification of complex motions, but adoption remains limited due to the lack of expert annotations and domain discrepancies from user variations. Limited annotations hinder the model's ability to generalize to out-of-distribution samples. While data augmentation can improve generalizability, unsupervised augmentation techniques must be applied carefully to avoid introducing noise. Unsupervised domain adaptation (UDA) addresses domain discrepancies by aligning conditional distributions with labeled target samples, but vanilla pseudo-labeling can lead to error propagation. To address these challenges, we propose μDAR, a novel joint optimization architecture comprised of three functions: (i) consistency regularizer between augmented samples to improve model classification generalizability, (ii) temporal ensemble for robust pseudo-label generation and (iii) conditional distribution alignment to improve domain generalizability. The temporal ensemble works by aggregating predictions from past epochs to smooth out noisy pseudo-label predictions, which are then used in the conditional distribution alignment module to minimize kernel-based class-wise conditional maximum mean discrepancy (kCMMD) between the source and target feature space to learn a domain invariant embedding. The consistency-regularized augmentations ensure that multiple augmentations of the same sample share the same labels; this results in (a) strong generalization with limited source domain samples and (b) consistent pseudo-label generation in target samples. The novel integration of these three modules in μDAR results in a range of ~ 4-12% average macro-F1 score improvement over six state-of-the-art UDA methods in four benchmark wHAR datasets. 

   more » « less
2. Maternal Social Loneliness Detection Using Passive Sensing Through Continuous Monitoring in Everyday Settings: Longitudinal Study

   https://doi.org/10.2196/47950  

   Sarhaddi, Fatemeh; Azimi, Iman; Niela-Vilen, Hannakaisa; Axelin, Anna; Liljeberg, Pasi; Rahmani, Amir M (January 2023, JMIR Formative Research)

   BackgroundMaternal loneliness is associated with adverse physical and mental health outcomes for both the mother and her child. Detecting maternal loneliness noninvasively through wearable devices and passive sensing provides opportunities to prevent or reduce the impact of loneliness on the health and well-being of the mother and her child. ObjectiveThe aim of this study is to use objective health data collected passively by a wearable device to predict maternal (social) loneliness during pregnancy and the postpartum period and identify the important objective physiological parameters in loneliness detection. MethodsWe conducted a longitudinal study using smartwatches to continuously collect physiological data from 31 women during pregnancy and the postpartum period. The participants completed the University of California, Los Angeles (UCLA) loneliness questionnaire in gestational week 36 and again at 12 weeks post partum. Responses to this questionnaire and background information of the participants were collected through our customized cross-platform mobile app. We leveraged participants’ smartwatch data from the 7 days before and the day of their completion of the UCLA questionnaire for loneliness prediction. We categorized the loneliness scores from the UCLA questionnaire as loneliness (scores≥12) and nonloneliness (scores<12). We developed decision tree and gradient-boosting models to predict loneliness. We evaluated the models by using leave-one-participant-out cross-validation. Moreover, we discussed the importance of extracted health parameters in our models for loneliness prediction. ResultsThe gradient boosting and decision tree models predicted maternal social loneliness with weighted F1-scores of 0.897 and 0.872, respectively. Our results also show that loneliness is highly associated with activity intensity and activity distribution during the day. In addition, resting heart rate (HR) and resting HR variability (HRV) were correlated with loneliness. ConclusionsOur results show the potential benefit and feasibility of using passive sensing with a smartwatch to predict maternal loneliness. Our developed machine learning models achieved a high F1-score for loneliness prediction. We also show that intensity of activity, activity pattern, and resting HR and HRV are good predictors of loneliness. These results indicate the intervention opportunities made available by wearable devices and predictive models to improve maternal well-being through early detection of loneliness. 

   more » « less
3. UnTran: Recognizing Unseen Activities with Unlabeled Data Using Transfer Learning

   https://doi.org/10.1109/IoTDI.2018.00014  

   Khan, Md Abdullah; Roy, Nirmalya (April 2018, 2018 IEEE/ACM Third International Conference on Internet-of-Things Design and Implementation (IoTDI))

   The success and impact of activity recognition algorithms largely depends on the availability of the labeled training samples and adaptability of activity recognition models across various domains. In a new environment, the pre-trained activity recognition models face challenges in presence of sensing bias- ness, device heterogeneities, and inherent variabilities in human behaviors and activities. Activity Recognition (AR) system built in one environment does not scale well in another environment, if it has to learn new activities and the annotated activity samples are scarce. Indeed building a new activity recognition model and training the model with large annotated samples often help overcome this challenging problem. However, collecting annotated samples is cost-sensitive and learning activity model at wild is computationally expensive. In this work, we propose an activity recognition framework, UnTran that utilizes source domains' pre-trained autoencoder enabled activity model that transfers two layers of this network to generate a common feature space for both source and target domain activities. We postulate a hybrid AR framework that helps fuse the decisions from a trained model in source domain and two activity models (raw and deep-feature based activity model) in target domain reducing the demand of annotated activity samples to help recognize unseen activities. We evaluated our framework with three real-world data traces consisting of 41 users and 26 activities in total. Our proposed UnTran AR framework achieves ≈ 75% F1 score in recognizing unseen new activities using only 10% labeled activity data in the target domain. UnTran attains ≈ 98% F1 score while recognizing seen activities in presence of only 2-3% of labeled activity samples. 

   more » « less
4. Real-time Context-Aware Multimodal Network for Activity and Activity-Stage Recognition from Team Communication in Dynamic Clinical Settings

   https://doi.org/10.1145/3580798  

   Gao, Chenyang; Marsic, Ivan; Sarcevic, Aleksandra; Gestrich-Thompson, Waverly; Burd, Randall S. (March 2022, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies)

   In clinical settings, most automatic recognition systems use visual or sensory data to recognize activities. These systems cannot recognize activities that rely on verbal assessment, lack visual cues, or do not use medical devices. We examined speech-based activity and activity-stage recognition in a clinical domain, making the following contributions. (1) We collected a high-quality dataset representing common activities and activity stages during actual trauma resuscitation events-the initial evaluation and treatment of critically injured patients. (2) We introduced a novel multimodal network based on audio signal and a set of keywords that does not require a high-performing automatic speech recognition (ASR) engine. (3) We designed novel contextual modules to capture dynamic dependencies in team conversations about activities and stages during a complex workflow. (4) We introduced a data augmentation method, which simulates team communication by combining selected utterances and their audio clips, and showed that this method contributed to performance improvement in our data-limited scenario. In offline experiments, our proposed context-aware multimodal model achieved F1-scores of 73.2±0.8% and 78.1±1.1% for activity and activity-stage recognition, respectively. In online experiments, the performance declined about 10% for both recognition types when using utterance-level segmentation of the ASR output. The performance declined about 15% when we omitted the utterance-level segmentation. Our experiments showed the feasibility of speech-based activity and activity-stage recognition during dynamic clinical events. 

   more » « less
5. Classifying Residual Stroke Severity Using Robotics-Assisted Stroke Rehabilitation: Machine Learning Approach

   https://doi.org/10.2196/56980  

   Jeter, Russell; Greenfield, Raymond; Housley, Stephen N; Belykh, Igor (January 2024, JMIR Biomedical Engineering)

   BackgroundStroke therapy is essential to reduce impairments and improve motor movements by engaging autogenous neuroplasticity. Traditionally, stroke rehabilitation occurs in inpatient and outpatient rehabilitation facilities. However, recent literature increasingly explores moving the recovery process into the home and integrating technology-based interventions. This study advances this goal by promoting in-home, autonomous recovery for patients who experienced a stroke through robotics-assisted rehabilitation and classifying stroke residual severity using machine learning methods. ObjectiveOur main objective is to use kinematics data collected during in-home, self-guided therapy sessions to develop supervised machine learning methods, to address a clinician’s autonomous classification of stroke residual severity–labeled data toward improving in-home, robotics-assisted stroke rehabilitation. MethodsIn total, 33 patients who experienced a stroke participated in in-home therapy sessions using Motus Nova robotics rehabilitation technology to capture upper and lower body motion. During each therapy session, the Motus Hand and Motus Foot devices collected movement data, assistance data, and activity-specific data. We then synthesized, processed, and summarized these data. Next, the therapy session data were paired with clinician-informed, discrete stroke residual severity labels: “no range of motion (ROM),” “low ROM,” and “high ROM.” Afterward, an 80%:20% split was performed to divide the dataset into a training set and a holdout test set. We used 4 machine learning algorithms to classify stroke residual severity: light gradient boosting (LGB), extra trees classifier, deep feed-forward neural network, and classical logistic regression. We selected models based on 10-fold cross-validation and measured their performance on a holdout test dataset using F1-score to identify which model maximizes stroke residual severity classification accuracy. ResultsWe demonstrated that the LGB method provides the most reliable autonomous detection of stroke severity. The trained model is a consensus model that consists of 139 decision trees with up to 115 leaves each. This LGB model boasts a 96.70% F1-score compared to logistic regression (55.82%), extra trees classifier (94.81%), and deep feed-forward neural network (70.11%). ConclusionsWe showed how objectively measured rehabilitation training paired with machine learning methods can be used to identify the residual stroke severity class, with efforts to enhance in-home self-guided, individualized stroke rehabilitation. The model we trained relies only on session summary statistics, meaning it can potentially be integrated into similar settings for real-time classification, such as outpatient rehabilitation facilities. 

   more » « less