More Like this

1. Online Recognition of Bimanual Coordination Provides Important Context for Movement Data in Bimanual Teleoperated Robots

   Boehm, J. ; Fey, N. ; Majewicz Fey, A. ( September 2021 , IEEE International Conference on Robotics and Automation)

   null (Ed.)

   An important problem in designing human-robot systems is the integration of human intent and performance in the robotic control loop, especially in complex tasks. Bimanual coordination is a complex human behavior that is critical in many fine motor tasks, including robot-assisted surgery. To fully leverage the capabilities of the robot as an intelligent and assistive agent, online recognition of bimanual coordination could be important. Robotic assistance for a suturing task, for example, will be fundamentally different during phases when the suture is wrapped around the instrument (i.e., making a c- loop), than when the ends of the suture are pulled apart. In this study, we develop an online recognition method of bimanual coordination modes (i.e., the directions and symmetries of right and left hand movements) using geometric descriptors of hand motion. We (1) develop this framework based on ideal trajectories obtained during virtual 2D bimanual path following tasks performed by human subjects operating Geomagic Touch haptic devices, (2) test the offline recognition accuracy of bi- manual direction and symmetry from human subject movement trials, and (3) evalaute how the framework can be used to characterize 3D trajectories of the da Vinci Surgical System’s surgeon-side manipulators during bimanual surgical training tasks. In the human subject trials, our geometric bimanual movement classification accuracy was 92.3% for movement direction (i.e., hands moving together, parallel, or away) and 86.0% for symmetry (e.g., mirror or point symmetry). We also show that this approach can be used for online classification of different bimanual coordination modes during needle transfer, making a C loop, and suture pulling gestures on the da Vinci system, with results matching the expected modes. Finally, we discuss how these online estimates are sensitive to task environment factors and surgeon expertise, and thus inspire future work that could leverage adaptive control strategies to enhance user skill during robot-assisted surgery. 

   more » « less
2. Towards Enabling Complex Touch-based Human-Drone Interaction

   Chen Y. ; Alimohammadzadeh H. ; Ghandeharizadeh S. ; Culbertson H. ( October 2023 , Human Multi-Robot Interaction at the 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2023))

   Wilde N. ; Alonso-Mora J. ; Brown D. ; Mattson C. ; Sycara K. (Ed.)

   In this paper, we introduce an innovative approach to multi-human robot interaction, leveraging the capabilities of omnicopters. These agile aerial vehicles are poised to revolutionize haptic feedback by offering complex sensations with 6 degrees of freedom (6DoF) movements. Unlike traditional systems, our envisioned method enables haptic rendering without the need for tilt, offering a more intuitive and seamless interaction experience. Furthermore, we propose using omnicopter swarms in human-robot interaction, these omnicopters can collaboratively emulate and render intricate objects in real-time. This swarm-based rendering not only expands the realm of tangible human-robot interactions but also holds potential in diverse applications, from immersive virtual environments to tactile guidance in physical tasks. Our vision outlines a future where robots and humans interact in more tangible and sophisticated ways, pushing the boundaries of current haptic technology. 

   more » « less
3. Treadmill vs. overground walking: different response to physical interaction

   https://doi.org/10.1152/jn.00176.2017  

   Ochoa, Julieth ; Sternad, Dagmar ; Hogan, Neville ( October 2017 , Journal of Neurophysiology)

   Rehabilitation of human motor function is an issue of growing significance, and human-interactive robots offer promising potential to meet the need. For the lower extremity, however, robot-aided therapy has proven challenging. To inform effective approaches to robotic gait therapy, it is important to better understand unimpaired locomotor control: its sensitivity to different mechanical contexts and its response to perturbations. The present study evaluated the behavior of 14 healthy subjects who walked on a motorized treadmill and overground while wearing an exoskeletal ankle robot. Their response to a periodic series of ankle plantar flexion torque pulses, delivered at periods different from, but sufficiently close to, their preferred stride cadence, was assessed to determine whether gait entrainment occurred, how it differed across conditions, and if the adapted motor behavior persisted after perturbation. Certain aspects of locomotor control were exquisitely sensitive to walking context, while others were not. Gaits entrained more often and more rapidly during overground walking, yet, in all cases, entrained gaits synchronized the torque pulses with ankle push-off, where they provided assistance with propulsion. Furthermore, subjects entrained to perturbation periods that required an adaption toward slower cadence, even though the pulses acted to accelerate gait, indicating a neural adaptation of locomotor control. Lastly, during 15 post-perturbation strides, the entrained gait period was observed to persist more frequently during overground walking. This persistence was correlated with the number of strides walked at the entrained gait period (i.e., longer exposure), which also indicated a neural adaptation. NEW & NOTEWORTHY We show that the response of human locomotion to physical interaction differs between treadmill and overground walking. Subjects entrained to a periodic series of ankle plantar flexion torque pulses that shifted their gait cadence, synchronizing ankle push-off with the pulses (so that they assisted propulsion) even when gait cadence slowed. Entrainment was faster overground and, on removal of torque pulses, the entrained gait period persisted more prominently overground, indicating a neural adaptation of locomotor control. 

   more » « less
4. Theory of Mind Assessment with Human-Human and Human-Robot Interactions

   https://doi.org/10.1007/978-3-031-05409-9_41  

   Rabe, T. ; Callis, A. ; Zheng, Z. ; Heard, J. ; Bailey, R. ; Alm, C. ( June 2022 , Human-Computer Interaction. Technological Innovation. HCII 2022. Lecture Notes in Computer Science)

   Human-robot interaction has played an increasingly significant role in more recent research involving the Theory of Mind (ToM). As the use of robot facilitators increases, questions arise regarding the implications of their involvement in a research setting. This work addresses the effects of a humanoid robot facilitator in a ToM assessment. This paper analyzes subjects’ performances on tasks meant to test ToM as those tasks are delivered by human or robot facilitators. Various modalities of data were collected: performance on ToM tasks, subjects’ perceptions of the robot, results from a ToM survey, and response duration. This paper highlights the effects of human-robot interactions in ToM assessments, which ultimately leads to a discussion on the effectiveness of using robot facilitators in future human-subject research. 

   more » « less
5. Human–Robot Collaboration in Smart Manufacturing: Robot Reactive Behavior Intelligence

   https://doi.org/10.1115/1.4048950  

   Nicora, Matteo Lavit ; Ambrosetti, Roberto ; Wiens, Gloria J. ; Fassi, Irene ( March 2021 , Journal of Manufacturing Science and Engineering)

   null (Ed.)

   Abstract To enable safe and effective human–robot collaboration (HRC) in smart manufacturing, seamless integration of sensing, cognition, and prediction into the robot controller is critical for real-time awareness, response, and communication inside a heterogeneous environment (robots, humans, and equipment). The specific research objective is to provide the robot Proactive Adaptive Collaboration Intelligence (PACI) and switching logic within its control architecture in order to give the robot the ability to optimally and dynamically adapt its motions, given a priori knowledge and predefined execution plans for its assigned tasks. The challenge lies in augmenting the robot’s decision-making process to have greater situation awareness and to yield smart robot behaviors/reactions when subject to different levels of human–robot interaction, while maintaining safety and production efficiency. Robot reactive behaviors were achieved via cost function-based switching logic activating the best suited high-level controller. The PACI’s underlying segmentation and switching logic framework is demonstrated to yield a high degree of modularity and flexibility. The performance of the developed control structure subjected to different levels of human–robot interactions was validated in a simulated environment. Open-loop commands were sent to the physical e.DO robot to demonstrate how the proposed framework would behave in a real application. 

   more » « less