We are developing a system for long term Semi-Automated Rehabilitation At the Home (SARAH) that relies on low-cost and unobtrusive video-based sensing. We present a cyber-human methodology used by the SARAH system for automated assessment of upper extremity stroke rehabilitation at the home. We propose a hierarchical model for automatically segmenting stroke survivor's movements and generating training task performance assessment scores during rehabilitation. The hierarchical model fuses expert therapist knowledge-based approaches with data-driven techniques. The expert knowledge is more observable in the higher layers of the hierarchy (task and segment) and therefore more accessible to algorithms incorporating high level constraints relating to activity structure (i.e., type and order of segments per task). We utilize an HMM and a Decision Tree model to connect these high level priors to data driven analysis. The lower layers (RGB images and raw kinematics) need to be addressed primarily through data driven techniques. We use a transformer based architecture operating on low-level action features (tracking of individual body joints and objects) and a Multi-Stage Temporal Convolutional Network(MS-TCN) operating on raw RGB images. We develop a sequence combining these complimentary algorithms effectively, thus encoding the information from different layers of the movement hierarchy. Through this combination, we produce a robust segmentation and task assessment results on noisy, variable and limited data, which is characteristic of low cost video capture of rehabilitation at the home. Our proposed approach achieves 85% accuracy in per-frame labeling, 99% accuracy in segment classification and 93% accuracy in task completion assessment. Although the methodology proposed in this paper applies to upper extremity rehabilitation using the SARAH system, it can potentially be used, with minor alterations, to assist automation in many other movement rehabilitation contexts (i.e., lower extremity training for neurological accidents).
more »
« less
This content will become publicly available on August 17, 2024
aBnormal motION capture In aCute Stroke (BIONICS): A Low-Cost Tele-Evaluation Tool for Automated Assessment of Upper Extremity Function in Stroke Patients
The incidence of stroke and stroke-related hemiparesis has been steadily increasing and is projected to become a serious social, financial, and physical burden on the aging population. Limited access to outpatient rehabilitation for these stroke survivors further deepens the healthcare issue and estranges the stroke patient demographic in rural areas. However, new advances in motion detection deep learning enable the use of handheld smartphone cameras for body tracking, offering unparalleled levels of accessibility.
In this study we want to develop an automated method for evaluation of a shortened variant of the Fugl-Meyer assessment, the standard stroke rehabilitation scale describing upper extremity motor function. We pair this technology with a series of machine learning models, including different neural network structures and an eXtreme Gradient Boosting model, to score 16 of 33 (49%) Fugl-Meyer item activities.
In this observational study, 45 acute stroke patients completed at least 1 recorded Fugl-Meyer assessment for the training of the auto-scorers, which yielded average accuracies ranging from 78.1% to 82.7% item-wise.
In this study, an automated method was developed for the evaluation of a shortened variant of the Fugl-Meyer assessment, the standard stroke rehabilitation scale describing upper extremity motor function. This novel method is demonstrated with potential to conduct telehealth rehabilitation evaluations and assessments with accuracy and availability.
- NSF-PAR ID:
- 10442842
- Publisher / Repository:
- SAGE Publications
- Date Published:
- Journal Name:
- Neurorehabilitation and Neural Repair
- Volume:
- 37
- Issue:
- 9
- ISSN:
- 1545-9683
- Page Range / eLocation ID:
- p. 591-602
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Background Individuals with hemiparesis post-stroke often have difficulty with tasks requiring upper extremity (UE) intra- and interlimb use, yet methods to quantify both are limited. Objective To develop a quantitative yet sensitive method to identify distinct features of UE intra- and interlimb use during task performance. Methods Twenty adults post-stroke and 20 controls wore five inertial sensors (wrists, upper arms, sternum) during 12 seated UE tasks. Three sensor modalities (acceleration, angular rate of change, orientation) were examined for three metrics (peak to peak amplitude, time, and frequency). To allow for comparison between sensor data, the resultant values were combined into one motion parameter, per sensor pair, using a novel algorithm. This motion parameter was compared in a group-by-task analysis of variance as a similarity score (0–1) between key sensor pairs: sternum to wrist, wrist to wrist, and wrist to upper arm. A use ratio (paretic/non-paretic arm) was calculated in persons post-stroke from wrist sensor data for each modality and compared to scores from the Adult Assisting Hand Assessment (Ad-AHA Stroke) and UE Fugl-Meyer (UEFM). Results A significant group × task interaction in the similarity score was found for all key sensor pairs. Post-hoc tests between task type revealed significant differences in similarity for sensor pairs in 8/9 comparisons for controls and 3/9 comparisons for persons post stroke. The use ratio was significantly predictive of the Ad-AHA Stroke and UEFM scores for each modality. Conclusions Our algorithm and sensor data analyses distinguished task type within and between groups and were predictive of clinical scores. Future work will assess reliability and validity of this novel metric to allow development of an easy-to-use app for clinicians.more » « less
-
more » « less
Abstract Background Vestibular deficits can impair an individual’s ability to maintain postural and/or gaze stability. Characterizing gait abnormalities among individuals affected by vestibular deficits could help identify patients at high risk of falling and inform rehabilitation programs. Commonly used gait assessment tools rely on simple measures such as timing and visual observations of path deviations by clinicians. These simple measures may not capture subtle changes in gait kinematics. Therefore, we investigated the use of wearable inertial measurement units (IMUs) and machine learning (ML) approaches to automatically discriminate between gait patterns of individuals with vestibular deficits and age-matched controls. The goal of this study was to examine the effects of IMU placement and gait task selection on the performance of automatic vestibular gait classifiers.
Methods Thirty study participants (15 with vestibular deficits and 15 age-matched controls) participated in a single-session gait study during which they performed seven gait tasks while donning a full-body set of IMUs. Classification performance was reported in terms of area under the receiver operating characteristic curve (AUROC) scores for Random Forest models trained on data from each IMU placement for each gait task.
Results Several models were able to classify vestibular gait better than random (AUROC > 0.5), but their performance varied according to IMU placement and gait task selection. Results indicated that a single IMU placed on the left arm when walking with eyes closed resulted in the highest AUROC score for a single IMU (AUROC = 0.88 [0.84, 0.89]). Feature permutation results indicated that participants with vestibular deficits reduced their arm swing compared to age-matched controls while they walked with eyes closed.
Conclusions These findings highlighted differences in upper extremity kinematics during walking with eyes closed that were characteristic of vestibular deficits and showed evidence of the discriminative ability of IMU-based automated screening for vestibular deficits. Further research should explore the mechanisms driving arm swing differences in the vestibular population.
-
Chua Chin Heng, Matthew (Ed.)Stroke rehabilitation seeks to accelerate motor recovery by training functional activities, but may have minimal impact because of insufficient training doses. In animals, training hundreds of functional motions in the first weeks after stroke can substantially boost upper extremity recovery. The optimal quantity of functional motions to boost recovery in humans is currently unknown, however, because no practical tools exist to measure them during rehabilitation training. Here, we present PrimSeq, a pipeline to classify and count functional motions trained in stroke rehabilitation. Our approach integrates wearable sensors to capture upper-body motion, a deep learning model to predict motion sequences, and an algorithm to tally motions. The trained model accurately decomposes rehabilitation activities into elemental functional motions, outperforming competitive machine learning methods. PrimSeq furthermore quantifies these motions at a fraction of the time and labor costs of human experts. We demonstrate the capabilities of PrimSeq in previously unseen stroke patients with a range of upper extremity motor impairment. We expect that our methodological advances will support the rigorous measurement required for quantitative dosing trials in stroke rehabilitation.more » « less
-
more » « less
Abstract Background Individualized, targeted, and intense training is the hallmark of successful gait rehabilitation in people post-stroke. Specifically, increasing use of the impaired ankle to increase propulsion during the stance phase of gait has been linked to higher walking speeds and symmetry. Conventional progressive resistance training is one method used for individualized and intense rehabilitation, but often fails to target paretic ankle plantarflexion during walking. Wearable assistive robots have successfully
assisted ankle-specific mechanisms to increase paretic propulsion in people post-stroke, suggesting their potential to provide targetedresistance to increase propulsion, but this application remains underexamined in this population. This work investigates the effects of targeted stance-phase plantarflexion resistance training with a soft ankle exosuit on propulsion mechanics in people post-stroke.Methods We conducted this study in nine individuals with chronic stroke and tested the effects of three resistive force magnitudes on peak paretic propulsion, ankle torque, and ankle power while participants walked on a treadmill at their comfortable walking speeds. For each force magnitude, participants walked for 1 min while the exosuit was inactive, 2 min with active resistance, and 1 min with the exosuit inactive, in sequence. We evaluated changes in gait biomechanics during the active resistance and post-resistance sections relative to the initial inactive section.
Results Walking with active resistance increased paretic propulsion by more than the minimal detectable change of 0.8 %body weight at all tested force magnitudes, with an average increase of 1.29 ± 0.37 %body weight at the highest force magnitude. This improvement corresponded to changes of 0.13 ± 0.03 N m kg− 1in peak biological ankle torque and 0.26 ± 0.04 W kg− 1in peak biological ankle power. Upon removal of resistance, propulsion changes persisted for 30 seconds with an improvement of 1.49 ± 0.58 %body weight after the highest resistance level and without compensatory involvement of the unresisted joints or limb.
Conclusions Targeted exosuit-applied functional resistance of paretic ankle plantarflexors can elicit the latent propulsion reserve in people post-stroke. After-effects observed in propulsion highlight the potential for learning and restoration of propulsion mechanics. Thus, this exosuit-based resistive approach may offer new opportunities for individualized and progressive gait rehabilitation.
