Search for: All records

Humans possess an innate ability to incorporate tools into our body schema to perform a myriad of tasks not possible with our natural limbs. Human-in-the-loop telerobotic systems (HiLTS) are tools that extend human manipulation capabilities to remote and virtual environments. Unlike most hand-held tools, however, HiLTS often possess complex electromechanical architectures that introduce non-trivial transmission dynamics between the robot’s leader and follower, which alter or obfuscate the environment’s dynamics. While considerable research has focused on negating or circumventing these dynamics, it is not well understood how capable human operators are at incorporating these transmission dynamics into their sensorimotor control scheme. To begin answering this question, we recruited N=12 participants to use a novel reconfigurable teleoperator with varying transmission dynamics to perform a visuo-haptic tracking task. Contrary to our original hypothesis, our findings demonstrate that humans can account for substantial differences in teleoperator transmission dynamics and produce the compensatory strategies necessary to adequately control the teleoperator. These findings suggest that advances in transparency algorithms and haptic feedback approaches must be coupled with control designs that leverage the unique capabilities of the human operator in the loop. 

Haptic illusions provide unique insights into how we model our bodies separate from our environment. Popular illusions like the rubber-hand illusion and mirror-box illusion have demonstrated that we can adapt the internal representations of our limbs in response to visuo-haptic conflicts. In this manuscript, we extend this knowledge by investigating to what extent, if any, we also augment our external representations of the environment and its action on our bodies in response to visuo-haptic conflicts. Utilizing a mirror and a robotic brushstroking platform, we create a novel illusory paradigm that presents a visuo-haptic conflict using congruent and incongruent tactile stimuli applied to participants' fingers. Overall, we observed that participants perceived an illusory tactile sensation on their visually occluded finger when seeing a visual stimulus that was inconsistent with the actual tactile stimulus provided. We also found residual effects of the illusion after the conflict was removed. These findings highlight how our need to maintain a coherent internal representation of our body extends to our model of our environment. 

Clinical myoelectric prostheses lack the sensory feedback and sufficient dexterity required to complete activities of daily living efficiently and accurately. Providing haptic feedback of relevant environmental cues to the user or imbuing the prosthesis with autonomous control authority have been separately shown to improve prosthesis utility. Few studies, however, have investigated the effect of combining these two approaches in a shared control paradigm, and none have evaluated such an approach from the perspective of neural efficiency (the relationship between task performance and mental effort measured directly from the brain). In this work, we analyzed the neural efficiency of 30 non-amputee participants in a grasp-and-lift task of a brittle object. Here, a myoelectric prosthesis featuring vibrotactile feedback of grip force and autonomous control of grasping was compared with a standard myoelectric prosthesis with and without vibrotactile feedback. As a measure of mental effort, we captured the prefrontal cortex activity changes using functional near infrared spectroscopy during the experiment. It was expected that the prosthesis with haptic shared control would improve both task performance and mental effort compared to the standard prosthesis. Results showed that only the haptic shared control system enabled users to achieve high neural efficiency, and that vibrotactile feedback was important for grasping with the appropriate grip force. These results indicate that the haptic shared control system synergistically combines the benefits of haptic feedback and autonomous controllers, and is well-poised to inform such hybrid advancements in myoelectric prosthesis technology.

 

Current commercially available robotic minimally invasive surgery (RMIS) platforms provide no haptic feedback of tool interactions with the surgical environment. As a consequence, novice robotic surgeons must rely exclusively on visual feedback to sense their physical interactions with the surgical environment. This technical limitation can make it challenging and time-consuming to train novice surgeons to proficiency in RMIS. Extensive prior research has demonstrated that incorporating haptic feedback is effective at improving surgical training task performance. However, few studies have investigated the utility of providing feedback of multiple modalities of haptic feedback simultaneously (multi-modality haptic feedback) in this context, and these studies have presented mixed results regarding its efficacy. Furthermore, the inability to generalize and compare these mixed results has limited our ability to understand why they can vary significantly between studies. Therefore, we have developed a generalized, modular multi-modality haptic feedback and data acquisition framework leveraging the real-time data acquisition and streaming capabilities of the Robot Operating System (ROS). In our preliminary study using this system, participants complete a peg transfer task using a da Vinci robot while receiving haptic feedback of applied forces, contact accelerations, or both via custom wrist-worn haptic devices. Results highlight the capability of our system in running systematic comparisons between various single and dual-modality haptic feedback approaches. 

The utility of telerobotic systems is driven in large part by the quality of feedback they provide to the operator. While the dynamic interaction between a robot and the environment can often be sensed or modeled, the dynamic coupling at the human-robot interface is often overlooked. Improving dexterous manipulation through telerobots will require careful consideration of human haptic perception as it relates to human exploration dynamics at the telerobotic interface. In this manuscript, we use exploration velocity as a means of controlling the operator's exploration dynamics, and present results from two stiffness discrimination experiments designed to investigate the effects of exploration velocity on stiffness perception. The results indicate that stiffness percepts vary differently for different exploration velocities on an individual level, however, no consistent trends were found across all participants. These results suggest that exploration dynamics can affect the quality of haptic interactions through telerobotic interfaces, and also reflect the need to study the underlying mechanisms that cause our perception to vary with our choice of exploration strategy.