An official website of the United States government
Objective: Real-time measurement of biological joint moment could enhance clinical assessments and generalize exoskeleton control. Accessing joint moments outside clinical and laboratory settings requires harnessing non-invasive wearable sensor data for indirect estimation. Previous approaches have been primarily validated during cyclic tasks, such as walking, but these methods are likely limited when translating to non-cyclic tasks where the mapping from kinematics to moments is not unique. Methods: We trained deep learning models to estimate hip and knee joint moments from kinematic sensors, electromyography (EMG), and simulated pressure insoles from a dataset including 10 cyclic and 18 non-cyclic activities. We assessed estimation error on combinations of sensor modalities during both activity types. Results: Compared to the kinematics-only baseline, adding EMG reduced RMSE by 16.9% at the hip and 30.4% at the knee (p<0.05) and adding insoles reduced RMSE by 21.7% at the hip and 33.9% at the knee (p<0.05). Adding both modalities reduced RMSE by 32.5% at the hip and 41.2% at the knee (p<0.05) which was significantly higher than either modality individually (p<0.05). All sensor additions improved model performance on non-cyclic tasks more than cyclic tasks (p<0.05). Conclusion: These results demonstrate that adding kinetic sensor information through EMG or insoles improves joint moment estimation both individually and jointly. These additional modalities are most important during non-cyclic tasks, tasks that reflect the variable and sporadic nature of the real-world. Significance: Improved joint moment estimation and task generalization is pivotal to developing wearable robotic systems capable of enhancing mobility in everyday life.
more »
« less
This content will become publicly available on June 5, 2026
Is EMG Information Necessary for Deep Learning Estimation of Joint and Muscle Level States?
Objective: Accurate, non-invasive methods for estimating joint and muscle physiological states have the potential to greatly enhance control of wearable devices during real-world ambulation. Traditional modeling approaches and current estimation methods used to predict muscle dynamics often rely on complex equipment or computationally intensive simulations and have difficulty estimating across a broad spectrum of tasks or subjects. Methods: Our approach used deep learning (DL) models trained on kinematic inputs to estimate internal physiological states at the knee, including moment, power, velocity, and force. We assessed each model's performance against ground truth labels from both a commonly used, standard OpenSim musculoskeletal model without EMG (static optimization) and an EMG-informed method (CEINMS), across 28 different cyclic and noncyclic tasks. Results: EMG provided no benefit for joint moment/power estimation (e.g., biological moment), but was critical for estimating muscle states. Models trained with EMG-informed labels but without EMG as an input to the DL system significantly outperformed models trained without EMG (e.g., 33.7% improvement for muscle moment estimation) (p < 0.05). Models that included EMG-informed labels and EMG as a model input demonstrated even higher performance (49.7% improvement for muscle moment estimation) (p < 0.05), but require the availability of EMG during model deployment, which may be impractical. Conclusion/Significance: While EMG information is not necessary for estimating joint level states, there is a clear benefit during muscle level state estimation. Our results demonstrate excellent tracking of these states with EMG included only during training, highlighting the practicality of real-time deployment of this approach.
more »
« less
- Award ID(s):
- 2233164
- PAR ID:
- 10613611
- Publisher / Repository:
- IEEE
- Date Published:
- Journal Name:
- IEEE Transactions on Biomedical Engineering
- ISSN:
- 0018-9294
- Page Range / eLocation ID:
- 1 to 12
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract Reinforcement learning (RL) has potential to provide innovative solutions to existing challenges in estimating joint moments in motion analysis, such as kinematic or electromyography (EMG) noise and unknown model parameters. Here, we explore feasibility of RL to assist joint moment estimation for biomechanical applications. Forearm and hand kinematics and forearm EMGs from four muscles during free finger and wrist movement were collected from six healthy subjects. Using the proximal policy optimization approach, we trained two types of RL agents that estimated joint moment based on measured kinematics or measured EMGs, respectively. To quantify the performance of trained RL agents, the estimated joint moment was used to drive a forward dynamic model for estimating kinematics, which was then compared with measured kinematics using Pearson correlation coefficient. The results demonstrated that both trained RL agents are feasible to estimate joint moment for wrist and metacarpophalangeal (MCP) joint motion prediction. The correlation coefficients between predicted and measured kinematics, derived from the kinematics-driven agent and subject-specific EMG-driven agents, were 98% ± 1% and 94% ± 3% for the wrist, respectively, and were 95% ± 2% and 84% ± 6% for the metacarpophalangeal joint, respectively. In addition, a biomechanically reasonable joint moment-angle-EMG relationship (i.e., dependence of joint moment on joint angle and EMG) was predicted using only 15 s of collected data. In conclusion, this study illustrates that an RL approach can be an alternative technique to conventional inverse dynamic analysis in human biomechanics study and EMG-driven human-machine interfacing applications.more » « less
-
Research on robotic lower-limb assistive devices over the past decade has generated autonomous, multiple degree-of-freedom devices to augment human performance during a variety of scenarios. However, the increase in capabilities of these devices is met with an increase in the complexity of the overall control problem and requirement for an accurate and robust sensing modality for intent recognition. Due to its ability to precede changes in motion, surface electromyography (EMG) is widely studied as a peripheral sensing modality for capturing features of muscle activity as an input for control of powered assistive devices. In order to capture features that contribute to muscle contraction and joint motion beyond muscle activity of superficial muscles, researchers have introduced sonomyography, or real-time dynamic ultrasound imaging of skeletal muscle. However, the ability of these sonomyography features to continuously predict multiple lower-limb joint kinematics during widely varying ambulation tasks, and their potential as an input for powered multiple degree-of-freedom lower-limb assistive devices is unknown. The objective of this research is to evaluate surface EMG and sonomyography, as well as the fusion of features from both sensing modalities, as inputs to Gaussian process regression models for the continuous estimation of hip, knee and ankle angle and velocity during level walking, stair ascent/descent and ramp ascent/descent ambulation. Gaussian process regression is a Bayesian nonlinear regression model that has been introduced as an alternative to musculoskeletal model-based techniques. In this study, time-intensity features of sonomyography on both the anterior and posterior thigh along with time-domain features of surface EMG from eight muscles on the lower-limb were used to train and test subject-dependent and task-invariant Gaussian process regression models for the continuous estimation of hip, knee and ankle motion. Overall, anterior sonomyography sensor fusion with surface EMG significantly improved estimation of hip, knee and ankle motion for all ambulation tasks (level ground, stair and ramp ambulation) in comparison to surface EMG alone. Additionally, anterior sonomyography alone significantly improved errors at the hip and knee for most tasks compared to surface EMG. These findings help inform the implementation and integration of volitional control strategies for robotic assistive technologies.more » « less
-
Abstract Cerebrovascular accidents like a stroke can affect the lower limb as well as upper extremity joints (i.e., shoulder, elbow, or wrist) and hinder the ability to produce necessary torque for activities of daily living. In such cases, muscles’ ability to generate forces reduces, thus affecting the joint’s torque production. Understanding how muscles generate forces is a key element to injury detection. Researchers have developed several computational methods to obtain muscle forces and joint torques. Electromyography (EMG) driven modeling is one of the approaches to estimate muscle forces and obtain joint torques from muscle activity measurements. Musculoskeletal models and EMG-driven models require necessary muscle-specific parameters for the calculation. The focus of this study is to investigate the EMG-driven approach along with an upper extremity musculoskeletal model to determine muscle forces of two major muscle groups, biceps brachii and triceps brachii, consisting of seven muscle-tendon units. Estimated muscle forces are used to determine the elbow joint torque. Experimental EMG signals and motion capture data are collected for a healthy subject. The musculoskeletal model is scaled to match the geometric parameters of the subject. Then, the approach calculates muscle forces and joint moment for two tasks: simple elbow flexion extension and triceps kickback. Individual muscle forces and net joint torques for both tasks are estimated. The study also has compared the effect of muscle-tendon parameters (optimal fiber length and tendon slack length) on the estimated results.more » « less
-
Abstract Patients with neuromuscular disease fail to produce necessary muscle force and have trouble maintaining joint moment required to perform activities of daily living. Measuring muscle force values in patients with neuromuscular disease is important but challenging. Electromyography (EMG) can be used to obtain muscle activation values, which can be converted to muscle forces and joint torques. Surface electrodes can measure activations of superficial muscles, but fine-wire electrodes are needed for deep muscles, although it is invasive and require skilled personnel and preparation time. EMG-driven modeling with surface electrodes alone could underestimate the net torque. In this research, authors propose a methodology to predict muscle activations from deeper muscles of the upper extremity. This method finds missing muscle activation one at a time by combining an EMG-driven musculoskeletal model and muscle synergies. This method tracks inverse dynamics joint moments to determine synergy vector weights and predict muscle activation of selected shoulder and elbow muscles of a healthy subject. In addition, muscle-tendon parameter values (optimal fiber length, tendon slack length, and maximum isometric force) have been personalized to the experimental subject. The methodology is tested for a wide range of rehabilitation tasks of the upper extremity across multiple healthy subjects. Results show this methodology can determine single unmeasured muscle activation up to Pearson's correlation coefficient (R) of 0.99 (root mean squared error, RMSE = 0.001) and 0.92 (RMSE = 0.13) for the elbow and shoulder muscles, respectively, for one degree-of-freedom (DoF) tasks. For more complicated five DoF tasks, activation prediction accuracy can reach up to R = 0.71 (RMSE = 0.29).more » « less
