skip to main content


Title: Determining the Significant Kinematic Features for Characterizing Stress during Surgical Tasks Using Spatial Attention
It has been shown that intraoperative stress can have a negative effect on surgeon surgical skills during laparoscopic procedures. For novice surgeons, stressful conditions can lead to significantly higher velocity, acceleration, and jerk of the surgical instrument tips, resulting in faster but less smooth movements. However, it is still not clear which of these kinematic features (velocity, acceleration, or jerk) is the best marker for identifying the normal and stressed conditions. Therefore, in order to find the most significant kinematic feature that is affected by intraoperative stress, we implemented a spatial attention-based Long Short-Term Memory (LSTM) classifier. In a prior IRB approved experiment, we collected data from medical students performing an extended peg transfer task who were randomized into a control group and a group performing the task under external psychological stresses. In our prior work, we obtained “representative” normal or stressed movements from this dataset using kinematic data as the input. In this study, a spatial attention mechanism is used to describe the contribution of each kinematic feature to the classification of normal/stressed movements. We tested our classifier under Leave-One-User-Out (LOUO) cross-validation, and the classifier reached an overall accuracy of 77.11% for classifying “representative” normal and stressed movements using kinematic features as the input. More importantly, we also studied the spatial attention extracted from the proposed classifier. Velocity and acceleration on both sides had significantly higher attention for classifying a normal movement ([Formula: see text]); Velocity ([Formula: see text]) and jerk ([Formula: see text]) on nondominant hand had significant higher attention for classifying a stressed movement, and it is worthy noting that the attention of jerk on nondominant hand side had the largest increment when moving from describing normal movements to stressed movements ([Formula: see text]). In general, we found that the jerk on nondominant hand side can be used for characterizing the stressed movements for novice surgeons more effectively.  more » « less
Award ID(s):
2109635
NSF-PAR ID:
10380014
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Journal of Medical Robotics Research
Volume:
07
Issue:
02n03
ISSN:
2424-905X
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. This paper presents the experimental position and force testing of a 3-armed 6-DOF Parallel Robot, Robossis, that is specifically designed for the application of long-bone femur fracture surgery. Current surgical techniques require a significant amount of time and effort to restore the fractured femur fragments’ length, alignment and rotation. To address these issues, the Robossis system will facilitate the femur fracture surgical procedure and oppose the large traction forces/torques of the muscle groups surrounding the femur. As such, Robossis would subsequently improve patient outcomes by eliminating intraoperative injuries, reducing radiation exposure from X-rays during surgery and decreasing the likelihood of follow-up operations. Specifically, in this paper, we study the accuracy of the Robossis system while moving in the operational workspace under free and simulated traction loads of ([Formula: see text]–1100[Formula: see text]N). Experimental testing in this study demonstrates that Robossis can reach the most extreme points in the workspace, as defined by the theoretical workspace, while maintaining minimal deviation from those points with an average deviation of 0.324[Formula: see text]mm. Furthermore, the force testing experiment shows that Robossis can counteract loads that are clinically relevant to restoring the fractured femur fragments’ length, alignment and rotation. In addition, we study the accuracy of Robossis motion while coupled with the master controller Sigma 7. The results show that Robossis can follow the desired trajectory in real-time with an average error of less than 1[Formula: see text]mm. To conclude, these results further establish the ability of the Robossis system to facilitate the femur fracture surgical procedure and eliminate limitations faced with the current surgical techniques. 
    more » « less
  2. Within the pycnocline, where diapycnal mixing is suppressed, both the vertical movement (uplift) of isopycnal surfaces and upward motion along sloping isopycnals supply nutrients to the euphotic layer, but the relative importance of each of these mechanisms is unknown. We present a method for decomposing vertical velocity w into two components in a Lagrangian frame: vertical velocity along sloping isopycnal surfaces [Formula: see text] and the adiabatic vertical velocity of isopycnal surfaces [Formula: see text]. We show that [Formula: see text], where [Formula: see text] is the isopycnal slope and [Formula: see text] is the geometric aspect ratio of the flow, and that [Formula: see text] accounts for 10%–25% of the total vertical velocity w for isopycnal slopes representative of the midlatitude pycnocline. We perform the decomposition of w in a process study model of a midlatitude eddying flow field generated with a range of isopycnal slopes. A spectral decomposition of the velocity components shows that while [Formula: see text] is the largest contributor to vertical velocity, [Formula: see text] is of comparable magnitude at horizontal scales less than about 10 km, that is, at submesoscales. Increasing the horizontal grid resolution of models is known to increase vertical velocity; this increase is disproportionately due to better resolution of [Formula: see text], as is shown here by comparing 1- and 4-km resolution model runs. Along-isopycnal vertical transport can be an important contributor to the vertical flux of tracers, including oxygen, nutrients, and chlorophyll, although we find weak covariance between vertical velocity and nutrient anomaly in our model.

     
    more » « less
  3. Abstract

    There have been significant advances in biosignal extraction techniques to drive external biomechatronic devices or to use as inputs to sophisticated human machine interfaces. The control signals are typically derived from biological signals such as myoelectric measurements made either from the surface of the skin or subcutaneously. Other biosignal sensing modalities are emerging. With improvements in sensing modalities and control algorithms, it is becoming possible to robustly control the target position of an end-effector. It remains largely unknown to what extent these improvements can lead to naturalistic human-like movement. In this paper, we sought to answer this question. We utilized a sensing paradigm called sonomyography based on continuous ultrasound imaging of forearm muscles. Unlike myoelectric control strategies which measure electrical activation and use the extracted signals to determine the velocity of an end-effector; sonomyography measures muscle deformation directly with ultrasound and uses the extracted signals to proportionally control the position of an end-effector. Previously, we showed that users were able to accurately and precisely perform a virtual target acquisition task using sonomyography. In this work, we investigate the time course of the control trajectories derived from sonomyography. We show that the time course of the sonomyography-derived trajectories that users take to reach virtual targets reflect the trajectories shown to be typical for kinematic characteristics observed in biological limbs. Specifically, during a target acquisition task, the velocity profiles followed a minimum jerk trajectory shown for point-to-point arm reaching movements, with similar time to target. In addition, the trajectories based on ultrasound imaging result in a systematic delay and scaling of peak movement velocity as the movement distance increased. We believe this is the first evaluation of similarities in control policies in coordinated movements in jointed limbs, and those based on position control signals extracted at the individual muscle level. These results have strong implications for the future development of control paradigms for assistive technologies.

     
    more » « less
  4. Reaching movements performed from a crouched body posture require a shift of body weight from both arms to one arm. This situation has remained unexamined despite the analogous load requirements during step initiation and the many studies of reaching from a seated or standing posture. To determine whether the body weight shift involves anticipatory or exclusively reactive control, we obtained force plate records, hand kinematics, and arm muscle activity from 11 healthy right-handed participants. They performed reaching movements with their left and right arm in two speed contexts, “comfortable” and “as fast as possible,” and two postural contexts, a less stable knees-together posture and a more stable knees-apart posture. Weight-shifts involved anticipatory postural actions (APAs) by the reaching and stance arms that were opposing in the vertical axis and aligned in the side-to-side axis similar to APAs by the legs for step initiation. Weight-shift APAs were correlated in time and magnitude, present in both speed contexts, more vigorous with the knees placed together, and similar when reaching with the dominant and nondominant arm. The initial weight-shift was preceded by bursts of muscle activity in the shoulder and elbow extensors (posterior deltoid and triceps lateral) of the reach arm and shoulder flexor (pectoralis major) of the stance arm, which indicates their causal role; leg muscles may have indirectly contributed but were not recorded. The strong functional similarity of weight-shift APAs during crouched reaching to human stepping and cat reaching suggests that they are a core feature of posture-movement coordination. NEW & NOTEWORTHY This work demonstrates that reaching from a crouched posture is preceded by bimanual anticipatory postural adjustments (APAs) that shift the body weight to the stance limb. Weight-shift APAs are more robust in an unstable body posture (knees together) and involve the shoulder and elbow extensors of the reach arm and shoulder flexor of the stance arm. This pattern mirrors the forelimb coordination of cats reaching and humans initiating a step. 
    more » « less
  5. null (Ed.)
    The endoscopic camera of a surgical robot pro- vides surgeons with a magnified 3D view of the surgical field, but repositioning it increases mental workload and operation time. Poor camera placement contributes to safety-critical events when surgical tools move out of the view of the camera. This paper presents a proof of concept of an autonomous camera system for the Raven II surgical robot that aims to reduce surgeon workload and improve safety by providing an optimal view of the workspace showing all objects of interest. This system uses transfer learning to localize and classify objects of interest within the view of a stereoscopic camera. The positions and centroid of the objects are estimated and a set of control rules determines the movement of the camera towards a more desired view. Our perception module had an accuracy of 61.21% overall for identifying objects of interest and was able to localize both graspers and multiple blocks in the environment. Comparison of the commands proposed by our system with the desired commands from a survey of 13 participants indicates that the autonomous camera system proposes appropriate movements for the tilt and pan of the camera. 
    more » « less