More Like this

1. Estimating human joint moments unifies exoskeleton control, reducing user effort

   https://doi.org/10.1126/scirobotics.adi8852  

   Molinaro, Dean D; Kang, Inseung; Young, Aaron J (March 2024, Science Robotics)

   Robotic lower-limb exoskeletons can augment human mobility, but current systems require extensive, context-specific considerations, limiting their real-world viability. Here, we present a unified exoskeleton control framework that autonomously adapts assistance on the basis of instantaneous user joint moment estimates from a temporal convolutional network (TCN). When deployed on our hip exoskeleton, the TCN achieved an average root mean square error of 0.142 newton-meters per kilogram across 35 ambulatory conditions without any user-specific calibration. Further, the unified controller significantly reduced user metabolic cost and lower-limb positive work during level-ground and incline walking compared with walking without wearing the exoskeleton. This advancement bridges the gap between in-lab exoskeleton technology and real-world human ambulation, making exoskeleton control technology viable for a broad community. 

   more » « less
2. My Pillow Knows My Sleep: Sleep Monitoring with Computational Fabrics in the Pillowcase

   https://doi.org/10.1145/3749550  

   Shao, Qijia; Chen, Junxiao; Leung, Ho_Man Colman; Zhao, Meiqi; Xu, Ruoyu; Liu, Jiting; García, Lisa_Maria DiSalvo; Jiang, Xiaofan; St-Onge, Marie-Pierre; Zhou, Xia (September 2025, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies)

   Sleep is a vital physiological state that significantly impacts overall health. Continuous monitoring of sleep posture, heart rate, respiratory rate, and body movement is crucial for diagnosing and managing sleep disorders. Current monitoring solutions often disrupt natural sleep due to discomfort or raise privacy and instrumentation concerns. We introduce PillowSense, a fabric-based sleep monitoring system seamlessly integrated into a pillowcase. PillowSense utilizes a dual-layer fabric design. The top layer comprises conductive fabrics for sensing electrocardiogram (ECG) and surface electromyogram (sEMG), while the bottom layer features pressure-sensitive fabrics to monitor sleep location and movement. The system processes ECG and sEMG signals sequentially to infer multiple sleep variables and incorporates an adversarial neural network to enhance posture classification accuracy. We fabricate prototypes using off-the-shelf hardware and conduct both lab-based and in-the-wild longitudinal user studies to evaluate the system's effectiveness. Across 151 nights and 912.2 hours of real-world sleep data, the system achieves an F1 score of 88% for classifying seven sleep postures, and clinically-acceptable accuracy in vital sign monitoring. PillowSense's comfort, washability, and robustness in multi-user scenarios underscore its potential for unobtrusive, large-scale sleep monitoring. 

   more » « less
3. In-the-Wild Affect Analysis of Children with ASD Using Heart Rate

   https://doi.org/10.3390/s23146572  

   Ali, Kamran; Shah, Sachin; Hughes, Charles E. (July 2023, Sensors)

   Recognizing the affective state of children with autism spectrum disorder (ASD) in real-world settings poses challenges due to the varying head poses, illumination levels, occlusion and a lack of datasets annotated with emotions in in-the-wild scenarios. Understanding the emotional state of children with ASD is crucial for providing personalized interventions and support. Existing methods often rely on controlled lab environments, limiting their applicability to real-world scenarios. Hence, a framework that enables the recognition of affective states in children with ASD in uncontrolled settings is needed. This paper presents a framework for recognizing the affective state of children with ASD in an in-the-wild setting using heart rate (HR) information. More specifically, an algorithm is developed that can classify a participant’s emotion as positive, negative, or neutral by analyzing the heart rate signal acquired from a smartwatch. The heart rate data are obtained in real time using a smartwatch application while the child learns to code a robot and interacts with an avatar. The avatar assists the child in developing communication skills and programming the robot. In this paper, we also present a semi-automated annotation technique based on facial expression recognition for the heart rate data. The HR signal is analyzed to extract features that capture the emotional state of the child. Additionally, in this paper, the performance of a raw HR-signal-based emotion classification algorithm is compared with a classification approach based on features extracted from HR signals using discrete wavelet transform (DWT). The experimental results demonstrate that the proposed method achieves comparable performance to state-of-the-art HR-based emotion recognition techniques, despite being conducted in an uncontrolled setting rather than a controlled lab environment. The framework presented in this paper contributes to the real-world affect analysis of children with ASD using HR information. By enabling emotion recognition in uncontrolled settings, this approach has the potential to improve the monitoring and understanding of the emotional well-being of children with ASD in their daily lives. 

   more » « less
4. Ring-a-Pose: A Ring for Continuous Hand Pose Tracking

   https://doi.org/10.1145/3699741  

   Yu, Tianhong Catherine; Hu, Guilin; Zhang, Ruidong; Lim, Hyunchul; Mahmud, Saif; Lee, Chi-Jung; Li, Ke; Agarwal, Devansh; Nie, Shuyang; Oh, Jinseok; et al (November 2024, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies)

   We present Ring-a-Pose, a single untethered ring that tracks continuous 3D hand poses. Located in the center of the hand, the ring emits an inaudible acoustic signal that each hand pose reflects differently. Ring-a-Pose imposes minimal obtrusions on the hand, unlike multi-ring or glove systems. It is not affected by the choice of clothing that may cover wrist-worn systems. In a series of three user studies with a total of 36 participants, we evaluate Ring-a-Pose's performance on pose tracking and micro-finger gesture recognition. Without collecting any training data from a user, Ring-a-Pose tracks continuous hand poses with a joint error of 14.1mm. The joint error decreases to 10.3mm for fine-tuned user-dependent models. Ring-a-Pose recognizes 7-class micro-gestures with a 90.60% and 99.27% accuracy for user-independent and user-dependent models, respectively. Furthermore, the ring exhibits promising performance when worn on any finger. Ring-a-Pose enables the future of smart rings to track and recognize hand poses using relatively low-power acoustic sensing. 

   more » « less
5. Holistic 3D Human and Scene Mesh Estimation from Single View Images.

   Weng, Zhenzhen; Yeung, Serena (January 2021, IEEE Xplore digital library)

   null (Ed.)

   The 3D world limits the human body pose and the hu- man body pose conveys information about the surrounding objects. Indeed, from a single image of a person placed in an indoor scene, we as humans are adept at resolving am- biguities of the human pose and room layout through our knowledge of the physical laws and prior perception of the plausible object and human poses. However, few computer vision models fully leverage this fact. In this work, we pro- pose a holistically trainable model that perceives the 3D scene from a single RGB image, estimates the camera pose and the room layout, and reconstructs both human body and object meshes. By imposing a set of comprehensive and sophisticated losses on all aspects of the estimations, we show that our model outperforms existing human body mesh methods and indoor scene reconstruction methods. To the best of our knowledge, this is the first model that outputs both object and human predictions at the mesh level, and performs joint optimization on the scene and human poses. 

   more » « less