More Like this

1. Online Recognition of Bimanual Coordination Provides Important Context for Movement Data in Bimanual Teleoperated Robots

   Boehm, J. ; Fey, N. ; Majewicz Fey, A. ( September 2021 , IEEE International Conference on Robotics and Automation)

   null (Ed.)

   An important problem in designing human-robot systems is the integration of human intent and performance in the robotic control loop, especially in complex tasks. Bimanual coordination is a complex human behavior that is critical in many fine motor tasks, including robot-assisted surgery. To fully leverage the capabilities of the robot as an intelligent and assistive agent, online recognition of bimanual coordination could be important. Robotic assistance for a suturing task, for example, will be fundamentally different during phases when the suture is wrapped around the instrument (i.e., making a c- loop), than when the ends of the suture are pulled apart. In this study, we develop an online recognition method of bimanual coordination modes (i.e., the directions and symmetries of right and left hand movements) using geometric descriptors of hand motion. We (1) develop this framework based on ideal trajectories obtained during virtual 2D bimanual path following tasks performed by human subjects operating Geomagic Touch haptic devices, (2) test the offline recognition accuracy of bi- manual direction and symmetry from human subject movement trials, and (3) evalaute how the framework can be used to characterize 3D trajectories of the da Vinci Surgical System’s surgeon-side manipulators during bimanual surgical training tasks. In the human subject trials, our geometric bimanual movement classification accuracy was 92.3% for movement direction (i.e., hands moving together, parallel, or away) and 86.0% for symmetry (e.g., mirror or point symmetry). We also show that this approach can be used for online classification of different bimanual coordination modes during needle transfer, making a C loop, and suture pulling gestures on the da Vinci system, with results matching the expected modes. Finally, we discuss how these online estimates are sensitive to task environment factors and surgeon expertise, and thus inspire future work that could leverage adaptive control strategies to enhance user skill during robot-assisted surgery. 

   more » « less
2. Abstracts From the 2018 Annual Meeting

   https://doi.org/10.1177/1545968318817151  

   Augenstein, T ; Washabaugh, E ; Remy, C ; Krishnan, C ( November 2018 , Neurorehabilitation and Neural Repair)

   Muscle weakness and loss of independent joint control are the 2 most common neuromotor impairments after stroke. While there are a number of approaches to improve poststroke muscle weakness, there are currently no rehabilitation strategies that directly target a patient’s inability to match and independently activate the normal patterned muscle coordination strategies, or “muscle synergies.” Our goal is to develop an EMG-based controller for retraining healthy muscle synergies in patients with stroke-related disabilities. The controller can be integrated into rehabilitation robots for their ability to structure the robot’s force output based on input EMG activity. However, developing such a controller would require a clear understanding of the relationship between the applied force from a rehabilitation robot and the resulting changes to a patient’s muscle synergies. Therefore, this study was performed to quantify how the muscle synergies of horizontal planar-reaching are affected by direction of an applied force at the end-effector (ie, hand). A 2 DOF, 10 muscle model was developed in MATLAB using parameters obtained from the OpenSim (version 3.3) open source software system. Simulation experiments were then performed in MATLAB to investigate the relationship between the applied force and the resulting muscle synergies. The simulated event was composed of several trials of the same righthanded, planar, multidirectional reaching task from 0° (to the right) to 360°. Each trial applied a different steering force direction at the subject’s hand, varying from −45° to 45° relative to the reaching direction. The simulation trials were also validated by evaluating the EMG patterns of a healthy subject when performing the same reaching task with varying steering force directions. For the 0° steering force trials, the muscle synergies and their activation timings were extracted using nonnegative matrix factorization (NMF). For all other trials, the synergy matrix was fixed and the activation timings were extracted from the product of the EMG of that trial and the pseudo-inverse of the synergy matrix from the 0° steering force trial. By fixing the synergy matrix in the trials with steering forces, we can directly track activation changes of a certain synergy as steering force is varied. For both simulation and experimental trials, circular statistics revealed a linear relationship between changes in steering force direction and principal direction of synergy activation. These results suggest that the activation of a synergy can be controlled directly by the direction of an applied steering force. This has relevant implications in synergy-based controller design because a computer can easily manipulate a patient’s muscle synergies and track the changes while avoiding the computational expense of NMF. In addition, similar analysis could be used to extract the relationship between applied forces and changes in synergies for other types of motion. 

   more » « less
3. An Approach to Modeling Closed-Loop Kinematic Chain Mechanisms, Applied to Simulations of the da Vinci Surgical System

   https://doi.org/10.12700/APH.16.8.2019.8.3  

   Gondokaryono RA, Agrawal A ( September 2019 , Acta Polytechnica Hungarica)

   Open-sourced kinematic models of the da Vinci Surgical System have previously been developed using serial chains for forward and inverse kinematics. However, these models do not describe the motion of every link in the closed-loop mechanism of the da Vinci manipulators; knowing the kinematics of all components in motion is essential for the foundation of modeling the system dynamics and implementing representative simulations. This paper proposes a modeling method of the closed-loop kinematics, using the existing da Vinci kinematics and an optical motion capture link length calibration. Resulting link lengths and DH parameters are presented and used as the basis for ROS-based simulation models. The models were simulated in RViz visualization simulation and Gazebo dynamics simulation. Additionally, the closed-loop kinematic chain was verified by comparing the remote center of motion location of simulation with the hardware. Furthermore, the dynamic simulation resulted in satisfactory joint stability and performance. All models and simulations are provided as an open-source package. 

   more » « less
4. Improvement of hand functions of spinal cord injury patients with electromyography-driven hand exoskeleton: A feasibility study

   https://doi.org/10.1017/wtc.2020.9  

   Yun, Youngmok ; Na, Youngjin ; Esmatloo, Paria ; Dancausse, Sarah ; Serrato, Alfredo ; Merring, Curtis A. ; Agarwal, Priyanshu ; Deshpande, Ashish D. ( January 2020 , Wearable Technologies)

   null (Ed.)

   Abstract We have developed a one-of-a-kind hand exoskeleton, called Maestro, which can power finger movements of those surviving severe disabilities to complete daily tasks using compliant joints. In this paper, we present results from an electromyography (EMG) control strategy conducted with spinal cord injury (SCI) patients (C5, C6, and C7) in which the subjects completed daily tasks controlling Maestro with EMG signals from their forearm muscles. With its compliant actuation and its degrees of freedom that match the natural finger movements, Maestro is capable of helping the subjects grasp and manipulate a variety of daily objects (more than 15 from a standardized set). To generate control commands for Maestro, an artificial neural network algorithm was implemented along with a probabilistic control approach to classify and deliver four hand poses robustly with three EMG signals measured from the forearm and palm. Increase in the scores of a standardized test, called the Sollerman hand function test, and enhancement in different aspects of grasping such as strength shows feasibility that Maestro can be capable of improving the hand function of SCI subjects. 

   more » « less
5. Virtual Reality for Evaluating Prosthetic Hand Control Strategies: A Preliminary Report

   https://doi.org/10.1109/embc46164.2021.9630555  

   Xie, Jason ; Hu, Xiaogang ( November 2021 , 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC))

   Improving prosthetic hand functionality is critical in reducing abandonment rates and improving the amputee’s quality of life. Techniques such as joint force estimation and gesture recognition using myoelectric signals could enable more realistic control of the prosthetic hand. To accelerate the translation of these advanced control strategies from lab to clinic, We created a virtual prosthetic control environment that enables rich user interactions and dexterity evaluation. The virtual environment is made of two parts, namely the Unity scene for rendering and user interaction, and a Python back-end to support accurate physics simulation and communication with control algorithms. By utilizing the built-in tracking capabilities of a virtual reality headset, the user can visualize and manipulate a virtual hand without additional motion tracking setups. In the virtual environment, we demonstrate actuation of the prosthetic hand through decoded EMG signal streaming, hand tracking, and the use of a VR controller. By providing a flexible platform to investigate different control modalities, we believe that our virtual environment will allow for faster experimentation and further progress in clinical translation. 

   more » « less