More Like this

1. Automated Movement Assessment in Stroke Rehabilitation

   https://doi.org/10.3389/fneur.2021.720650  

   Ahmed, Tamim ; Thopalli, Kowshik ; Rikakis, Thanassis ; Turaga, Pavan ; Kelliher, Aisling ; Huang, Jia-Bin ; Wolf, Steven L. ( August 2021 , Frontiers in Neurology)

   We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents). 

   more » « less
2. Flexion and Extension Capable Motor Tendon Actuated Exosuit Glove with Open Palm

   Patterson, G. ; Zell, T. ; Parker, D. ; Grieve, J. ; Watson, S. ; Vikas, V. ( August 2019 , Proceedings of the ASME 2019 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference)

   Patients suffering from medical conditions resulting in hand impairment experience difficulty in performing simple daily tasks, like getting dressed or using a pencil, resulting in a poorer quality of life. Rehabilitation attempts to help such individuals regain a sense of control and normalcy. In this context, recent advances in robotics have manifested in multiple designs of hand exoskeletons and exosuit gloves for assistance and rehabilitation. These designs are typically actuated using pneumatic, shape memory alloys and motor-tendon actuators. The proposed Motor Tendon Actuated Exosuit Glove (MTAEG) with an open palm is a soft material glove capable of both flexion and extension of all four fingers of the human hand. Its minimally invasive design maintains an open palm to facilitate haptic and tactile interaction with the environment. The MTAEG achieves flexion-extension motion with joint angles of 45° at the metacarpal joint which is 57% of the desired motion; 90° at the proximal interphalangeal joint which is 100% of the desired motion; and 50° at the distal interphalangeal joint which is 96% of the desired motion. The paper discusses the challenges in achieving the desired motion without the ability to directly model human tendons, and the inability to actuate joints individually. 

   more » « less
3. Gamification in a Physical Rehabilitation Setting: Developing a Proprioceptive Training Exercise for a Wrist Robot

   https://doi.org/10.1145/3355056.3364572  

   Curry, Christopher ; Elangovan, Naveen ; Reid, Reuben Gardos ; Xu, Jiapeng ; Konczak, Jürgen ( November 2019 , SIGGRAPH Asia 2019 Posters)

   Proprioception or body awareness is an essential sense that aids in the neural control of movement. Proprioceptive impairments are commonly found in people with neurological conditions such as stroke and Parkinson’s disease. Such impairments are known to impact the patient’s quality of life. Robot-aided proprioceptive training has been proposed and tested to improve sensorimotor performance. However, such robot-aided exercises are implemented similar to many physical rehabilitation exercises, requiring task-specific and repetitive movements from patients. Monotonous nature of such repetitive exercises can result in reduced patient motivation, thereby, impacting treatment adherence and therapy gains. Gamification of exercises can make physical rehabilitation more engaging and rewarding. In this work, we discuss our ongoing efforts to develop a game that can accompany a robot-aided wrist proprioceptive training exercise. 

   more » « less
4. mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors

   An, Sizhe An ; Li, Yin ; Ogras, Umit ( November 2022 , Advances in Neural Information Processing Systems 35 (NeurIPS 2022))

   The ability to estimate 3D human body pose and movement, also known as human pose estimation (HPE), enables many applications for home-based health monitoring, such as remote rehabilitation training. Several possible solutions have emerged using sensors ranging from RGB cameras, depth sensors, millimeter-Wave (mmWave) radars, and wearable inertial sensors. Despite previous efforts on datasets and benchmarks for HPE, few dataset exploits multiple modalities and focuses on home-based health monitoring. To bridge this gap, we present mRI1, a multi-modal 3D human pose estimation dataset with mmWave, RGB-D, and Inertial Sensors. Our dataset consists of over 160k synchronized frames from 20 subjects performing rehabilitation exercises and supports the benchmarks of HPE and action detection. We perform extensive experiments using our dataset and delineate the strength of each modality. We hope that the release of mRI can catalyze the research in pose estimation, multi-modal learning, and action understanding, and more importantly facilitate the applications of home-based health monitoring. 

   more » « less
5. Fast and scalable human pose estimation using mmWave point cloud

   https://doi.org/10.1145/3489517.3530522  

   An, Sizhe ; Ogras, Umit Y. ( July 2022 , DAC '22: Proceedings of the 59th ACM/IEEE Design Automation Conference)

   Millimeter-Wave (mmWave) radar can enable high-resolution human pose estimation with low cost and computational requirements. However, mmWave data point cloud, the primary input to processing algorithms, is highly sparse and carries significantly less information than other alternatives such as video frames. Furthermore, the scarce labeled mmWave data impedes the development of machine learning (ML) models that can generalize to unseen scenarios. We propose a fast and scalable human pose estimation (FUSE) framework that combines multi-frame representation and meta-learning to address these challenges. Experimental evaluations show that FUSE adapts to the unseen scenarios 4× faster than current supervised learning approaches and estimates human joint coordinates with about 7 cm mean absolute error. 

   more » « less