An official website of the United States government
Abstract BackgroundImproving the prediction ability of a human-machine interface (HMI) is critical to accomplish a bio-inspired or model-based control strategy for rehabilitation interventions, which are of increased interest to assist limb function post neurological injuries. A fundamental role of the HMI is to accurately predict human intent by mapping signals from a mechanical sensor or surface electromyography (sEMG) sensor. These sensors are limited to measuring the resulting limb force or movement or the neural signal evoking the force. As the intermediate mapping in the HMI also depends on muscle contractility, a motivation exists to include architectural features of the muscle as surrogates of dynamic muscle movement, thus further improving the HMI’s prediction accuracy. ObjectiveThe purpose of this study is to investigate a non-invasive sEMG and ultrasound (US) imaging-driven Hill-type neuromuscular model (HNM) for net ankle joint plantarflexion moment prediction. We hypothesize that the fusion of signals from sEMG and US imaging results in a more accurate net plantarflexion moment prediction than sole sEMG or US imaging. MethodsTen young non-disabled participants walked on a treadmill at speeds of 0.50, 0.75, 1.00, 1.25, and 1.50 m/s. The proposed HNM consists of two muscle-tendon units. The muscle activation for each unit was calculated as a weighted summation of the normalized sEMG signal and normalized muscle thickness signal from US imaging. The HNM calibration was performed under both single-speed mode and inter-speed mode, and then the calibrated HNM was validated across all walking speeds. ResultsOn average, the normalized moment prediction root mean square error was reduced by 14.58 % ($$p=0.012$$ ) and 36.79 % ($$p<0.001$$ ) with the proposed HNM when compared to sEMG-driven and US imaging-driven HNMs, respectively. Also, the calibrated models with data from the inter-speed mode were more robust than those from single-speed modes for the moment prediction. ConclusionsThe proposed sEMG-US imaging-driven HNM can significantly improve the net plantarflexion moment prediction accuracy across multiple walking speeds. The findings imply that the proposed HNM can be potentially used in bio-inspired control strategies for rehabilitative devices due to its superior prediction.
more »
« less
A deep learning method to predict ankle joint moment during walking at different speeds with ultrasound imaging: A framework for assistive devices control
Abstract Robotic assistive or rehabilitative devices are promising aids for people with neurological disorders as they help regain normative functions for both upper and lower limbs. However, it remains challenging to accurately estimate human intent or residual efforts non-invasively when using these robotic devices. In this article, we propose a deep learning approach that uses a brightness mode, that is, B-mode, of ultrasound (US) imaging from skeletal muscles to predict the ankle joint net plantarflexion moment while walking. The designed structure of customized deep convolutional neural networks (CNNs) guarantees the convergence and robustness of the deep learning approach. We investigated the influence of the US imaging’s region of interest (ROI) on the net plantarflexion moment prediction performance. We also compared the CNN-based moment prediction performance utilizing B-mode US and sEMG spectrum imaging with the same ROI size. Experimental results from eight young participants walking on a treadmill at multiple speeds verified an improved accuracy by using the proposed US imaging + deep learning approach for net joint moment prediction. With the same CNN structure, compared to the prediction performance by using sEMG spectrum imaging, US imaging significantly reduced the normalized prediction root mean square error by 37.55% ( $ p $ < .001) and increased the prediction coefficient of determination by 20.13% ( $ p $ < .001). The findings show that the US imaging + deep learning approach personalizes the assessment of human joint voluntary effort, which can be incorporated with assistive or rehabilitative devices to improve clinical performance based on the assist-as-needed control strategy.
more »
« less
- Award ID(s):
- 2002261
- PAR ID:
- 10414442
- Date Published:
- Journal Name:
- Wearable Technologies
- Volume:
- 3
- ISSN:
- 2631-7176
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Predicting future walking joint kinematics is crucial for assistive device control, especially in variable walking environments. Traditional optical motion capture systems provide kinematics data but require laborious post-processing, whereas IMU based systems provide direct calculations but add delays due to data collection and algorithmic processes. Predicting future kinematics helps to compensate for these delays, enabling the system real-time. Furthermore, these predicted kinematics could serve as target trajectories for assistive devices such as exoskeletal robots and lower limb prostheses. However, given the complexity of human mobility and environmental factors, this prediction remains to be challenging. To address this challenge, we propose the Dual-ED-Attention-FAM-Net, a deep learning model utilizing two encoders, two decoders, a temporal attention module, and a feature attention module. Our model outperforms the state-of-the-art LSTM model. Specifically, for Dataset A, using IMUs and a combination of IMUs and videos, RMSE values decrease from 4.45° to 4.22° and from 4.52° to 4.15°, respectively. For Dataset B, IMUs and IMUs combined with pressure insoles result in RMSE reductions from 7.09° to 6.66° and from 7.20° to 6.77°, respectively. Additionally, incorporating other modalities alongside IMUs helps improve the performance of the model.more » « less
-
Robotic prostheses and powered exoskeletons are novel assistive robotic devices for modern medicine. Muscle activity sensing plays an important role in controlling assistive robotics devices. Most devices measure the surface electromyography (sEMG) signal for myoelectric control. However, sEMG is an integrated signal from muscle activities. It is difficult to sense muscle movements in specific small regions, particularly at different depths. Alternatively, traditional ultrasound imaging has recently been proposed to monitor muscle activity due to its ability to directly visualize superficial and at-depth muscles. Despite their advantages, traditional ultrasound probes lack wearability. In this paper, a wearable ultrasound (US) transducer, based on lead zirconate titanate (PZT) and a polyimide substrate, was developed for a muscle activity sensing demonstration. The fabricated PZT-5A elements were arranged into a 4 × 4 array and then packaged in polydimethylsiloxane (PDMS). In vitro porcine tissue experiments were carried out by generating the muscle activities artificially, and the muscle movements were detected by the proposed wearable US transducer via muscle movement imaging. Experimental results showed that all 16 elements had very similar acoustic behaviors: the averaged central frequency, −6 dB bandwidth, and electrical impedance in water were 10.59 MHz, 37.69%, and 78.41 Ω, respectively. The in vitro study successfully demonstrated the capability of monitoring local muscle activity using the prototyped wearable transducer. The findings indicate that ultrasonic sensing may be an alternative to standardize myoelectric control for assistive robotics applications.more » « less
-
This journal review article focuses on the use of assistive and rehabilitative exoskeletons as a new opportunity for individuals with diminished mobility. The article aims to identify gaps and inconsistencies in state-of-the-art assistive and rehabilitative devices, with the overall goal of promoting innovation and improvement in this field. The literature review explores the mechanisms, actuators, and sensing procedures employed in each application, specifically focusing on passive shoulder supports and active soft robotic actuator gloves. Passive shoulder supports are an excellent option for bearing heavy loads, as they enable the load to be evenly distributed across the shoulder joint. This, in turn, reduces stress and strain around the surrounding muscles. On the other hand, the active soft robotic actuator glove is well suited for providing support and assistance by mimicking the characteristics of human muscle. This review reveals that these devices improve the overall standard of living for those who experience various impairments but also encounter limitations requiring redress. Overall, this article serves as a valuable resource for individuals working in the field of assistive and rehabilitative exoskeletons, providing insight into the state of the art and potential areas for improvement.more » « less
-
null (Ed.)This paper investigates an ultrasound (US) imaging-based methodology to assess the contraction levels of plantar flexors quantitatively. Echogenicity derived from US imaging at different anatomical depths, including both lateral gastrocnemius (LGS) and soleus (SOL) muscles, is used for the prediction of the volitional isometric plantar flexion moment. Synchronous measurements, including a plantar flexion torque signal, a surface electromyography (sEMG) signal, and US imaging of both LGS and SOL muscles, are collected. Four feature sets, including sole sEMG, sole LGS echogenicity, sole SOL echogenicity, and their fusion, are used to train a Gaussian process regression (GPR) model and predict plantar flexion torque. The experimental results on four non-disabled participants show that the torque prediction accuracy is improved significantly by using the LGS or SOL echogenicity signal than using the sEMG signal. However, there is no significant improvement by using the fused feature compared to sole LGS or SOL echogenicity. The findings imply that using US imaging-derived signals improves the accuracy of predicting volitional effort on human plantar flexors. Potentially, US imaging can be used as a new sensing modality to measure or predict human lower limb motion intent in clinical rehabilitation devices.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1017/wtc.2022.18
