skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Using Reinforcement Learning to Estimate Human Joint Moments From Electromyography or Joint Kinematics: An Alternative Solution to Musculoskeletal-Based Biomechanics
Abstract Reinforcement learning (RL) has potential to provide innovative solutions to existing challenges in estimating joint moments in motion analysis, such as kinematic or electromyography (EMG) noise and unknown model parameters. Here, we explore feasibility of RL to assist joint moment estimation for biomechanical applications. Forearm and hand kinematics and forearm EMGs from four muscles during free finger and wrist movement were collected from six healthy subjects. Using the proximal policy optimization approach, we trained two types of RL agents that estimated joint moment based on measured kinematics or measured EMGs, respectively. To quantify the performance of trained RL agents, the estimated joint moment was used to drive a forward dynamic model for estimating kinematics, which was then compared with measured kinematics using Pearson correlation coefficient. The results demonstrated that both trained RL agents are feasible to estimate joint moment for wrist and metacarpophalangeal (MCP) joint motion prediction. The correlation coefficients between predicted and measured kinematics, derived from the kinematics-driven agent and subject-specific EMG-driven agents, were 98% ± 1% and 94% ± 3% for the wrist, respectively, and were 95% ± 2% and 84% ± 6% for the metacarpophalangeal joint, respectively. In addition, a biomechanically reasonable joint moment-angle-EMG relationship (i.e., dependence of joint moment on joint angle and EMG) was predicted using only 15 s of collected data. In conclusion, this study illustrates that an RL approach can be an alternative technique to conventional inverse dynamic analysis in human biomechanics study and EMG-driven human-machine interfacing applications.  more » « less
Award ID(s):
1856441
PAR ID:
10296979
Author(s) / Creator(s):
; ;  
Date Published:
Journal Name:
Journal of Biomechanical Engineering
Volume:
143
Issue:
4
ISSN:
0148-0731
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Cerebrovascular accidents like a stroke can affect the lower limb as well as upper extremity joints (i.e., shoulder, elbow, or wrist) and hinder the ability to produce necessary torque for activities of daily living. In such cases, muscles’ ability to generate forces reduces, thus affecting the joint’s torque production. Understanding how muscles generate forces is a key element to injury detection. Researchers have developed several computational methods to obtain muscle forces and joint torques. Electromyography (EMG) driven modeling is one of the approaches to estimate muscle forces and obtain joint torques from muscle activity measurements. Musculoskeletal models and EMG-driven models require necessary muscle-specific parameters for the calculation. The focus of this study is to investigate the EMG-driven approach along with an upper extremity musculoskeletal model to determine muscle forces of two major muscle groups, biceps brachii and triceps brachii, consisting of seven muscle-tendon units. Estimated muscle forces are used to determine the elbow joint torque. Experimental EMG signals and motion capture data are collected for a healthy subject. The musculoskeletal model is scaled to match the geometric parameters of the subject. Then, the approach calculates muscle forces and joint moment for two tasks: simple elbow flexion extension and triceps kickback. Individual muscle forces and net joint torques for both tasks are estimated. The study also has compared the effect of muscle-tendon parameters (optimal fiber length and tendon slack length) on the estimated results. 
    more » « less
  2. Objective: Accurate, non-invasive methods for estimating joint and muscle physiological states have the potential to greatly enhance control of wearable devices during real-world ambulation. Traditional modeling approaches and current estimation methods used to predict muscle dynamics often rely on complex equipment or computationally intensive simulations and have difficulty estimating across a broad spectrum of tasks or subjects. Methods: Our approach used deep learning (DL) models trained on kinematic inputs to estimate internal physiological states at the knee, including moment, power, velocity, and force. We assessed each model's performance against ground truth labels from both a commonly used, standard OpenSim musculoskeletal model without EMG (static optimization) and an EMG-informed method (CEINMS), across 28 different cyclic and noncyclic tasks. Results: EMG provided no benefit for joint moment/power estimation (e.g., biological moment), but was critical for estimating muscle states. Models trained with EMG-informed labels but without EMG as an input to the DL system significantly outperformed models trained without EMG (e.g., 33.7% improvement for muscle moment estimation) (p < 0.05). Models that included EMG-informed labels and EMG as a model input demonstrated even higher performance (49.7% improvement for muscle moment estimation) (p < 0.05), but require the availability of EMG during model deployment, which may be impractical. Conclusion/Significance: While EMG information is not necessary for estimating joint level states, there is a clear benefit during muscle level state estimation. Our results demonstrate excellent tracking of these states with EMG included only during training, highlighting the practicality of real-time deployment of this approach. 
    more » « less
  3. This article presents an optimization formulation and experimental validation of a dynamic-joint-strength-based two-dimensional symmetric maximum weight-lifting simulation. Dynamic joint strength (the net moment capacity as a function of joint angle and angular velocity), as presented in the literature, is adopted in the optimization formulation to predict the symmetric maximum lifting weight and corresponding motion. Nineteen participants were recruited to perform a maximum-weight-box-lifting task in the laboratory, and kinetic and kinematic data including motion and ground reaction forces were collected using a motion capture system and force plates, respectively. For each individual, the predicted spine, shoulder, elbow, hip, knee, and ankle joint angles, as well as vertical and horizontal ground reaction force and box weight, were compared with the experimental data. Both root-mean-square error and Pearson’s correlation coefficient ( r) were used for the validation. The results show that the proposed two-dimensional optimization-based motion prediction formulation is able to accurately predict all joint angles, box weights, and vertical ground reaction forces, but not horizontal ground reaction forces. 
    more » « less
  4. Objective: Real-time measurement of biological joint moment could enhance clinical assessments and generalize exoskeleton control. Accessing joint moments outside clinical and laboratory settings requires harnessing non-invasive wearable sensor data for indirect estimation. Previous approaches have been primarily validated during cyclic tasks, such as walking, but these methods are likely limited when translating to non-cyclic tasks where the mapping from kinematics to moments is not unique. Methods: We trained deep learning models to estimate hip and knee joint moments from kinematic sensors, electromyography (EMG), and simulated pressure insoles from a dataset including 10 cyclic and 18 non-cyclic activities. We assessed estimation error on combinations of sensor modalities during both activity types. Results: Compared to the kinematics-only baseline, adding EMG reduced RMSE by 16.9% at the hip and 30.4% at the knee (p<0.05) and adding insoles reduced RMSE by 21.7% at the hip and 33.9% at the knee (p<0.05). Adding both modalities reduced RMSE by 32.5% at the hip and 41.2% at the knee (p<0.05) which was significantly higher than either modality individually (p<0.05). All sensor additions improved model performance on non-cyclic tasks more than cyclic tasks (p<0.05). Conclusion: These results demonstrate that adding kinetic sensor information through EMG or insoles improves joint moment estimation both individually and jointly. These additional modalities are most important during non-cyclic tasks, tasks that reflect the variable and sporadic nature of the real-world. Significance: Improved joint moment estimation and task generalization is pivotal to developing wearable robotic systems capable of enhancing mobility in everyday life. 
    more » « less
  5. Abstract Most motion capture measurements suffer from soft-tissue artifacts (STA). Especially affected are rotations about the long axis of a limb segment, such as humeral internal-external rotation (HIER) and forearm pronation-supination (FPS). Unfortunately, most existing methods to compensate for STA were designed for optoelectronic motion capture systems. We present and evaluate an STA compensation method that (1) compensates for STA in HIER and/or FPS, (2) is developed specifically for electromagnetic motion capture systems, and (3) does not require additional calibration or data. To compensate for STA, calculation of HIER angles relies on forearm orientation, and calculation of FPS angles rely on hand orientation. To test this approach, we recorded whole-arm movement data from eight subjects and compared their joint angle trajectories calculated according to progressive levels of STA compensation. Compensated HIER and FPS angles were significantly larger than uncompensated angles. Although the effect of STA compensation on other joint angles (besides HIER and FPS) was usually modest, significant effects were seen in certain degrees-of-freedom under some conditions. Overall, the method functioned as intended during most of the range of motion of the upper limb, but it becomes unstable in extreme elbow extension and extreme wrist flexion–extension. Specifically, this method is not recommended for movements within 20 deg of full elbow extension, full wrist flexion, or full wrist extension. Since this method does not require additional calibration of data, it can be applied retroactively to data collected without the intent to compensate for STA. 
    more » « less