skip to main content

Explore Research Products in the NSF-PAR

  • NSF Public Access
  • Search Results
  • Ambulation Mode Classification of Individuals with Transfemoral Amputation through A-Mode Sonomyography and Convolutional Neural Networks
Title: Ambulation Mode Classification of Individuals with Transfemoral Amputation through A-Mode Sonomyography and Convolutional Neural Networks
Many people struggle with mobility impairments due to lower limb amputations. To participate in society, they need to be able to walk on a wide variety of terrains, such as stairs, ramps, and level ground. Current lower limb powered prostheses require different control strategies for varying ambulation modes, and use data from mechanical sensors within the prosthesis to determine which ambulation mode the user is in. However, it can be challenging to distinguish between ambulation modes. Efforts have been made to improve classification accuracy by adding electromyography information, but this requires a large number of sensors, has a low signal-to-noise ratio, and cannot distinguish between superficial and deep muscle activations. An alternative sensing modality, A-mode ultrasound, can detect and distinguish between changes in superficial and deep muscles. It has also shown promising results in upper limb gesture classification. Despite these advantages, A-mode ultrasound has yet to be employed for lower limb activity classification. Here we show that A- mode ultrasound can classify ambulation mode with comparable, and in some cases, superior accuracy to mechanical sensing. In this study, seven transfemoral amputee subjects walked on an ambulation circuit while wearing A-mode ultrasound transducers, IMU sensors, and their passive prosthesis. The circuit consisted of sitting, standing, level-ground walking, ramp ascent, ramp descent, stair ascent, and stair descent, and a spatial–temporal convolutional network was trained to continuously classify these seven activities. Offline continuous classification with A-mode ultrasound alone was able to achieve an accuracy of 91.8±3.4%, compared with 93.8±3.0%, when using kinematic data alone. Combined kinematic and ultrasound produced 95.8±2.3% accuracy. This suggests that A-mode ultrasound provides additional useful information about the user’s gait beyond what is provided by mechanical sensors, and that it may be able to improve ambulation mode classification. By incorporating these sensors into powered prostheses, users may enjoy higher reliability for their prostheses, and more seamless transitions between ambulation modes.  more » « less
Award ID(s):
1925371
NSF-PAR ID:
10463767
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Sensors
Volume:
22
Issue:
23
ISSN:
1424-8220
Page Range / eLocation ID:
9350
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ( , IEEE transactions on biomedical engineering)
    null (Ed.)
    Clinical translation of “intelligent” lower-limb assistive technologies relies on robust control interfaces capable of accurately detecting user intent. To date, mechanical sensors and surface electromyography (EMG) have been the primary sensing modalities used to classify ambulation. Ultrasound (US) imaging can be used to detect user-intent by characterizing structural changes of muscle. Our study evaluates wearable US imaging as a new sensing modality for continuous classification of five discrete ambulation modes: level, incline, decline, stair ascent, and stair descent ambulation, and benchmarks performance relative to EMG sensing. Ten able-bodied subjects were equipped with a wearable US scanner and eight unilateral EMG sensors. Time-intensity features were recorded from US images of three thigh muscles. Features from sliding windows of EMG signals were analyzed in two configurations: one including 5 EMG sensors on muscles around the thigh, and another with 3 additional sensors placed on the shank. Linear discriminate analysis was implemented to continuously classify these phase-dependent features of each sensing modality as one of five ambulation modes. US-based sensing statistically improved mean classification accuracy to 99.8% (99.5-100% CI) compared to 8-EMG sensors (85.8%; 84.0-87.6% CI) and 5-EMG sensors (75.3%; 74.5-76.1% CI). Further, separability analyses show the importance of superficial and deep US information for stair classification relative to other modes. These results are the first to demonstrate the ability of US-based sensing to classify discrete ambulation modes, highlighting the potential for improved assistive device control using less widespread, less superficial and higher resolution sensing of skeletal muscle. 
    more » « less
  2. ; ( , Frontiers in Robotics and AI)

    Research on robotic lower-limb assistive devices over the past decade has generated autonomous, multiple degree-of-freedom devices to augment human performance during a variety of scenarios. However, the increase in capabilities of these devices is met with an increase in the complexity of the overall control problem and requirement for an accurate and robust sensing modality for intent recognition. Due to its ability to precede changes in motion, surface electromyography (EMG) is widely studied as a peripheral sensing modality for capturing features of muscle activity as an input for control of powered assistive devices. In order to capture features that contribute to muscle contraction and joint motion beyond muscle activity of superficial muscles, researchers have introduced sonomyography, or real-time dynamic ultrasound imaging of skeletal muscle. However, the ability of these sonomyography features to continuously predict multiple lower-limb joint kinematics during widely varying ambulation tasks, and their potential as an input for powered multiple degree-of-freedom lower-limb assistive devices is unknown. The objective of this research is to evaluate surface EMG and sonomyography, as well as the fusion of features from both sensing modalities, as inputs to Gaussian process regression models for the continuous estimation of hip, knee and ankle angle and velocity during level walking, stair ascent/descent and ramp ascent/descent ambulation. Gaussian process regression is a Bayesian nonlinear regression model that has been introduced as an alternative to musculoskeletal model-based techniques. In this study, time-intensity features of sonomyography on both the anterior and posterior thigh along with time-domain features of surface EMG from eight muscles on the lower-limb were used to train and test subject-dependent and task-invariant Gaussian process regression models for the continuous estimation of hip, knee and ankle motion. Overall, anterior sonomyography sensor fusion with surface EMG significantly improved estimation of hip, knee and ankle motion for all ambulation tasks (level ground, stair and ramp ambulation) in comparison to surface EMG alone. Additionally, anterior sonomyography alone significantly improved errors at the hip and knee for most tasks compared to surface EMG. These findings help inform the implementation and integration of volitional control strategies for robotic assistive technologies.

     
    more » « less
  3. ; ; ; ; ( , IEEE Transactions on Biomedical Engineering)
    Autonomous lower-limb exoskeletons must modulate assistance based on locomotion mode (e.g., ramp or stair ascent) to adapt to the corresponding changes in human biological joint dynamics. However, current mode classification strategies for exoskeletons often require user-specific tuning, have a slow update rate, and rely on additional sensors outside of the exoskeleton sensor suite. In this study, we introduce a deep convolutional neural network-based locomotion mode classifier for hip exoskeleton applications using an open-source gait biomechanics dataset with various wearable sensors. Our approach removed the limitations of previous systems as it is 1) subject-independent (i.e., no user-specific data), 2) capable of continuously classifying for smooth and seamless mode transitions, and 3) only utilizes minimal wearable sensors native to a conventional hip exoskeleton. We optimized our model, based on several important factors contributing to overall performance, such as transition label timing, model architecture, and sensor placement, which provides a holistic understanding of mode classifier design. Our optimized DL model showed a 3.13% classification error (steady-state: 0.80 ± 0.38% and transitional: 6.49 ± 1.42%), outperforming other machine learning-based benchmarks commonly practiced in the field (p<0.05). Furthermore, our multi-modal analysis indicated that our model can maintain high performance in different settings such as unseen slopes on stairs or ramps. Thus, our study presents a novel locomotion mode framework, capable of advancing robotic exoskeleton applications toward assisting community ambulation. 
    more » « less
  4. ; ; ( , IEEE Transactions on Medical Robotics and Bionics)
    Powered knee-ankle prostheses can offer benefits over conventional passive devices during stair locomotion by providing biomimetic net-positive work and active control of joint angles. However, many modern control approaches for stair ascent and descent are often limited by time-consuming hand-tuning of user/task-specific parameters, predefined trajectories that remove user volition, or heuristic approaches that cannot be applied to both stair ascent and descent. This work presents a phase-based hybrid kinematic and impedance controller (HKIC) that allows for semi-volitional, biomimetic stair ascent and descent at a variety of step heights. We define a unified phase variable for both stair ascent and descent that utilizes lower-limb geometry to adjust to different users and step heights. We extend our prior data-driven impedance model for variable-incline walking, modifying the cost function and constraints to create a continuously-varying impedance parameter model for stair ascent and descent over a continuum of step heights. Experiments with above-knee amputee participants (N=2) validate that our HKIC controller produces biomimetic ascent and descent joint kinematics, kinetics, and work across four step height configurations. We also show improved kinematic performance with our HKIC controller in comparison to a passive microprocessor-controlled device during stair locomotion. 
    more » « less
  5. ; ; ; ( , IEEE International Conference on Robotics and Automation)
    Passive prostheses cannot provide the net positive work required at the knee and ankle for step-over stair ascent. Powered prostheses can provide this net positive work, but user synchronization of joint motion and power input are critical to enabling natural stair ascent gaits. In this work, we build on previous phase variable-based control methods for walking and propose a stair ascent controller driven by the motion of the user's residual thigh. We use reference kinematics from an able-bodied dataset to produce knee and ankle joint trajectories parameterized by gait phase. We redefine the gait cycle to begin at the point of maximum hip flexion instead of heel strike to improve the phase estimate. Able-bodied bypass adapter experiments demonstrate that the phase variable controller replicates normative able-bodied kinematic trajectories with a root mean squared error of 12.66 deg and 2.64 deg for the knee and ankle, respectively. The knee and ankle joints provided on average 0.387J/kg and 0.212J/kg per stride, compared to the normative averages of 0.335J/kg and 0.207J/kg, respectively. Thus, this controller allows powered knee-ankle prostheses to perform net positive mechanical work to assist stair ascent. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email