An official website of the United States government
Ultrasound-based sensing of muscle deformation, known as sonomyography, has shown promise for accurately classifying the intended hand grasps of individuals with upper limb loss in offline settings. Building upon this previous work, we present the first demonstration of real-time prosthetic hand control using sonomyography to perform functional tasks. An individual with congenital bilateral limb absence was fitted with sockets containing a low-profile ultrasound transducer placed over forearm muscle tissue in the residual limbs. A classifier was trained using linear discriminant analysis to recognize ultrasound images of muscle contractions for three discrete hand configurations (rest, tripod grasp, index finger point) under a variety of arm positions designed to cover the reachable workspace. A prosthetic hand mounted to the socket was then controlled using this classifier. Using this real-time sonomyographic control, the participant was able to complete three functional tasks that required selecting different hand grasps in order to grasp and move one-inch wooden blocks over a broad range of arm positions. Additionally, these tests were successfully repeated without retraining the classifier across 3 hours of prosthesis use and following simulated donning and doffing of the socket. This study supports the feasibility of using sonomyography to control upper limb prostheses in real-world applications.
more »
« less
"Compliant Magnetic FMG Array Dataset"
{"Abstract":[""Control of powered prosthetic hands could benefit from improved techniques to infer the amputees\\u2019 desired grasp intentions. Force myography (FMG) has recently emerged as a potential alternative to electromyography, which is traditionally used in clinical practice. Due to their practical and robust attributes, we introduce innovative compliant magnetic FMG sensor arrays for forearm muscle pattern recognition. Sensor arrays with 18 or 24 compliant magnetic FMG sensor modules were created, demonstrating the customizability of our Scan2Make process for creating individualized sockets with integrated FMG sensor arrays for varying limb differences. Real-time control experiments with subject-specific artificial neural networks (ANN) showed that non-amputees were able to produce 15.43 \\u00b1 2.37 grasp gesture classes with over 90% accuracy. The three amputees produced an average of 9.00 \\u00b1 3.61 gestures with over 90% accuracy in real-time. Additionally, principal component analysis (PCA) was used to both reduce sensor count and improve classification accuracy in subsequent real-time playback analyses. Seven out of ten subjects achieved their highest accuracy with a reduced number of compliant magnetic sensor modules. To our best knowledge, this study is the first to use arrays of compliant magnetic sensors for forearm muscle pattern recognition to enable real-time control of a robotic hand. Furthermore, this is the first effort to use a 3D scanning and printing approach to create custom prosthetic sockets with any kind of integrated FMG sensor. Additionally, we are providing the first open-source database of FMG signals from 3 amputees and 7 non-amputees using the compliant magnetic sensor arrays as a resource for the research community. These novel compliant magnetic FMG sensor arrays have the potential to advance the state of the art for prosthetic hand control and could be used broadly in the fields of tactile sensing, haptics, teleoperation, and rehabilitation. ""]}
more »
« less
- Award ID(s):
- 2205205
- PAR ID:
- 10691474
- Publisher / Repository:
- IEEE DataPort
- Date Published:
- Format(s):
- Medium: X
- Right(s):
- Creative Commons Attribution 4.0 International
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Tigrini, Andrea (Ed.)Hand gesture classification is crucial for the control of many modern technologies, ranging from virtual and augmented reality systems to assistive mechatronic devices. A prominent control technique employs surface electromyography (EMG) and pattern recognition algorithms to identify specific patterns in muscle electrical activity and translate these to device commands. While being well established in consumer, clinical, and research applications, this technique suffers from misclassification errors caused by limb movements and the weight of manipulated objects, both vital aspects of how we use our hands in daily life. An emerging alternative control technique is force myography (FMG) which uses pattern recognition algorithms to predict hand gestures from the axial forces present at the skin’s surface created by contractions of the underlying muscles. As EMG and FMG capture different physiological signals associated with muscle contraction, we hypothesized that each may offer unique additional information for gesture classification, potentially improving classification accuracy in the presence of limb position and object loading effects. Thus, we tested the effect of limb position and grasped load on 3 different sensing modalities: EMG, FMG, and the fused combination of the two. 27 able-bodied participants performed a grasp and release task with 4 hand gestures at 8 positions and under 5 object weight conditions. We then examined the effects of limb position and grasped load on gesture classification accuracy across each sensing modality. It was found that position and grasped load had statistically significant effects on the classification performance of the 3 sensing modalities and that the combination of EMG and FMG provided the highest classification accuracy of hand gesture, limb position, and grasped load combinations (97.34%) followed by FMG (92.27%) and then EMG (82.84%). This points to the fact that the addition of FMG to traditional EMG control systems offers unique additional data for more effective device control and can help accommodate different limb positions and grasped object loads.more » « less
-
Abstract Electromyogram (EMG)-controlled prosthetic hands have advanced significantly during the past two decades. However, most of the currently available prosthetic hands fail to replicate human hand functionality and controllability. To measure the emulation of the human hand by a prosthetic hand, it is important to evaluate the functional characteristics. Moreover, incorporating feedback from end users during clinical testing is crucial for the precise assessment of a prosthetic hand. The work reported in this manuscript unfolds the functional characteristics of an EMG-CoNtrolled PRosthetIC Hand called ENRICH. ENRICH is a real-time EMG controlled prosthetic hand that can grasp objects in 250.8$$ \pm $$1.1 ms, fulfilling the neuromuscular constraint of a human hand. ENRICH is evaluated in comparison to 26 laboratory prototypes and 10 commercial variants of prosthetic hands. The hand was evaluated in terms of size, weight, operation time, weight lifting capacity, finger joint range of motion, control strategy, degrees of freedom, grasp force, and clinical testing. The box and block test and pick and place test showed ENRICH’s functionality and controllability. The functional evaluation reveals that ENRICH has the potential to restore functionality to hand amputees, improving their quality of life.more » « less
-
Wearable technologies for hand gesture classification are becoming increasingly prominent due to the growing need for more natural, human-centered control of complex devices. This need is particularly evident in emerging fields such as virtual reality and bionic prostheses, which require precise control with minimal delay. One method used for hand gesture recognition is force myography (FMG), which utilizes non-invasive pressure sensors to measure radial muscle forces on the skin’s surface of the forearm during hand movements. These sensors, typically force-sensitive resistors (FSRs), require additional circuitry to generate analog output signals, which are then classified using machine learning to derive corresponding control signals for the device. The performance of hand gesture classification can be influenced by the characteristics of this output signal, which may vary depending on the circuitry used. Our study examined three commonly used circuits in FMG systems: the voltage divider (VD), unity gain amplifier (UGA), and transimpedance amplifier (TIA). We first conducted benchtop testing of FSRs to characterize the impact of this circuitry on linearity, deadband, hysteresis, and drift, all metrics with the potential to influence an FMG system’s performance. To evaluate the circuit’s performance in hand gesture classification, we constructed an FMG band with 8 FSRs, using an adjustable Velcro strap and interchangeable circuitry. Wearing the FMG band, participants (N = 15) were instructed to perform 10 hand gestures commonly used in daily living. Our findings indicated that the UGA circuit outperformed others in minimizing hysteresis, drift and deadband with comparable results to the VD, while the TIA circuit excelled in ensuring linearity. Further, contemporary machine learning algorithms used to detect hand gestures were unaffected by the circuitry employed. These results suggest that applications of FMG requiring precise sensing of force values would likely benefit from use of the UGA. Alternatively, if hand gesture state classification is the only use case, developers can take advantage of benefits offered from using less complex circuitry such as the VD.more » « less
-
Abstract Electromyogram (EMG) has been a fundamental approach for prosthetic hand control. However it is limited by the functionality of residual muscles and muscle fatigue. Currently, exploring temporal shifts in brain networks and accurately classifying noninvasive electroencephalogram (EEG) for prosthetic hand control remains challenging. In this manuscript, it is hypothesized that the coordinated and synchronized temporal patterns within the brain network, termed as brain synergy, contain valuable information to decode hand movements. 32-channel EEGs were acquired from 10 healthy participants during hand grasp and open. Synergistic spatial distribution pattern and power spectra of brain activity were investigated using independent component analysis of EEG. Out of 32 EEG channels, 15 channels spanning the frontal, central and parietal regions were strategically selected based on the synergy of spatial distribution pattern and power spectrum of independent components. Time-domain and synergistic features were extracted from the selected 15 EEG channels. These features were employed to train a Bayesian optimizer-based support vector machine (SVM). The optimized SVM classifier could achieve an average testing accuracy of 94.39$$ \pm $$.84% using synergistic features. The pairedt-test showed that synergistic features yielded significantly higher area under curve values (p < .05) compared to time-domain features in classifying hand movements. The output of the classifier was employed for the control of the prosthetic hand. This synergistic approach for analyzing temporal activities in motor control and control of prosthetic hands have potential contributions to future research. It addresses the limitations of EMG-based approaches and emphasizes the effectiveness of synergy-based control for prostheses.more » « less
