skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Continuous joint velocity estimation using CNN-based deep learning for multi-DoF prosthetic wrist for activities of daily living
Myoelectric control of prostheses is a long-established technique, using surface electromyography (sEMG) to detect user intention and perform subsequent mechanical actions. Most machine learning models utilized in control systems are trained using isolated movements that do not reflect the natural movements occurring during daily activities. Moreover, movements are often affected by arm postures, the duration of activities, and personal habits. It is crucial to have a control system for multi-degree-of-freedom (DoF) prosthetic arms that is trained using sEMG data collected from activities of daily living (ADL) tasks. This work focuses on two major functional wrist movements: pronation-supination and dart-throwing movement (DTM), and introduces a new wrist control system that directly maps sEMG signals to the joint velocities of the multi-DoF wrist. Additionally, a specific training strategy (Quick training) is proposed that enables the controller to be applied to new subjects and handle situations where sensors may displace during daily living, muscles can become fatigued, or sensors can become contaminated (e.g., due to sweat). The prosthetic wrist controller is designed based on data from 24 participants and its performance is evaluated using the Root Mean Square Error (RMSE) and Pearson Correlation. The results are found to depend on the characteristics of the tasks. For example, tasks with dart-throwing motion show smaller RSME values (Hammer: 6.68 deg/s and Cup: 7.92 deg/s) compared to tasks with pronation-supination (Bulb: 43.98 deg/s and Screw: 53.64 deg/s). The proposed control technique utilizing Quick training demonstrates a decrease in the average root mean square error (RMSE) value by 35% and an increase in the average Pearson correlation value by 40% across all four ADL tasks.  more » « less
Award ID(s):
2221979
PAR ID:
10545296
Author(s) / Creator(s):
;
Publisher / Repository:
Frontiers
Date Published:
Journal Name:
Frontiers in Neurorobotics
Volume:
17
ISSN:
1662-5218
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Musculoskeletal modeling is a new computational tool to reverse engineer human control systems, which require efficient algorithms running in real-time. Human hand pronation-supination movement is accomplished by movement of the radius and ulna bones relative to each other via the complex proximal and distal radioulnar joints, each with multiple degrees of freedom (DOFs). Here, we report two simplified models of this complex kinematic transformation implemented as a part of a 20 DOF model of the hand and forearm. The pronation/supination DOF was implemented as a single rotation joint either within the forearm segment or separating proximal and distal parts of the forearm segment. Torques produced by the inverse dynamic simulations with anatomical architecture of the forearm (OpenSim model) were used as the "gold standard" in the comparison of two simple models. Joint placement was iteratively optimized to achieve the closest representation of torques during realistic hand movements. The model with a split forearm segment performed better than the model with a solid forearm segment in simulating pronation/supination torques. We conclude that simplifying pronation/supination DOF as a single-axis rotation between arm segments is a viable strategy to reduce the complexity of multi-DOF dynamic simulations. 
    more » « less
  2. Working towards improved neuromyoelectric control of dexterous prosthetic hands, we explored how differences in training paradigms affect the subsequent online performance of two different motor-decode algorithms. Participants included two intact subjects and one participant who had undergone a recent transradial amputation after complex regional pain syndrome (CRPS) and multi-year disuse of the affected hand. During algorithm training sessions, participants actively mimicked hand movements appearing on a computer monitor. We varied both the duration of the hold-time (0.1 s or 5 s) at the end-point of each of six different digit and wrist movements, and the order in which the training movements were presented (random or sequential). We quantified the impact of these variations on two different motordecode algorithms, both having proportional, six-degree-offreedom (DOF) control: a modified Kalman filter (MKF) previously reported by this group, and a new approach - a convolutional neural network (CNN). Results showed that increasing the hold-time in the training set improved run-time performance. By contrast, presenting training movements in either random or sequential order had a variable and relatively modest effect on performance. The relative performance of the two decode algorithms varied according to the performance metric. This work represents the first-ever amputee use of a CNN for real-time, proportional six-DOF control of a prosthetic hand. Also novel was the testing of implanted high-channelcount devices for neuromyoelectric control shortly after amputation, following CRPS and long-term hand disuse. This work identifies key factors in the training of decode algorithms that improve their subsequent run-time performance. 
    more » « less
  3. IntroductionIndividuals who have suffered a cervical spinal cord injury prioritize the recovery of upper limb function for completing activities of daily living. Hybrid FES-exoskeleton systems have the potential to assist this population by providing a portable, powered, and wearable device; however, realization of this combination of technologies has been challenging. In particular, it has been difficult to show generalizability across motions, and to define optimal distribution of actuation, given the complex nature of the combined dynamic system. MethodsIn this paper, we present a hybrid controller using a model predictive control (MPC) formulation that combines the actuation of both an exoskeleton and an FES system. The MPC cost function is designed to distribute actuation on a single degree of freedom to favor FES control effort, reducing exoskeleton power consumption, while ensuring smooth movements along different trajectories. Our controller was tested with nine able-bodied participants using FES surface stimulation paired with an upper limb powered exoskeleton. The hybrid controller was compared to an exoskeleton alone controller, and we measured trajectory error and torque while moving the participant through two elbow flexion/extension trajectories, and separately through two wrist flexion/extension trajectories. ResultsThe MPC-based hybrid controller showed a reduction in sum of squared torques by an average of 48.7 and 57.9% on the elbow flexion/extension and wrist flexion/extension joints respectively, with only small differences in tracking accuracy compared to the exoskeleton alone. DiscussionTo realize practical implementation of hybrid FES-exoskeleton systems, the control strategy requires translation to multi-DOF movements, achieving more consistent improvement across participants, and balancing control to more fully leverage the muscles' capabilities. 
    more » « less
  4. Abstract Most motion capture measurements suffer from soft-tissue artifacts (STA). Especially affected are rotations about the long axis of a limb segment, such as humeral internal-external rotation (HIER) and forearm pronation-supination (FPS). Unfortunately, most existing methods to compensate for STA were designed for optoelectronic motion capture systems. We present and evaluate an STA compensation method that (1) compensates for STA in HIER and/or FPS, (2) is developed specifically for electromagnetic motion capture systems, and (3) does not require additional calibration or data. To compensate for STA, calculation of HIER angles relies on forearm orientation, and calculation of FPS angles rely on hand orientation. To test this approach, we recorded whole-arm movement data from eight subjects and compared their joint angle trajectories calculated according to progressive levels of STA compensation. Compensated HIER and FPS angles were significantly larger than uncompensated angles. Although the effect of STA compensation on other joint angles (besides HIER and FPS) was usually modest, significant effects were seen in certain degrees-of-freedom under some conditions. Overall, the method functioned as intended during most of the range of motion of the upper limb, but it becomes unstable in extreme elbow extension and extreme wrist flexion–extension. Specifically, this method is not recommended for movements within 20 deg of full elbow extension, full wrist flexion, or full wrist extension. Since this method does not require additional calibration of data, it can be applied retroactively to data collected without the intent to compensate for STA. 
    more » « less
  5. Abstract In recent years, commercially available dexterous upper limb prostheses for children have begun to emerge. These devices derive control signals from surface electromyography (measure of affected muscle electrical activity, sEMG) to drive a variety of grasping motions. However, the ability for children with congenital upper limb deficiency to actuate their affected muscles to achieve naturalistic prosthetic control is not well understood, as compared to adults or children with acquired hand loss. To address this gap, we collected sEMG data from 9 congenital one-handed participants ages 8–20 years as they envisioned and attempted to perform 10 different movements with their missing hands. Seven sEMG electrodes were adhered circumferentially around the participant’s affected and unaffected limbs and participants mirrored the attempted missing hand motions with their intact side. To analyze the collected sEMG data, we used time and frequency domain analyses. We found that for the majority of participants, attempted hand movements produced detectable and consistent muscle activity, and the capacity to achieve this was not dissimilar across the affected and unaffected sides. These data suggest that children with congenital hand absence retain a degree of control over their affected muscles, which has important implications for translating and refining advanced prosthetic control technologies for children. 
    more » « less