More Like this

1. Real-Time Multi-Diver Tracking and Re-identification for Underwater Human-Robot Collaboration

   De Langis, Karin Johanna; Sattar, J (June 2020, 2020 IEEE International Conference on Robotics and Automation)

   Autonomous underwater robots working with teams of human divers may need to distinguish between different divers, e.g., to recognize a lead diver or to follow a specific team member. This paper describes a technique that enables autonomous underwater robots to track divers in real time as well as to reidentify them. The approach is an extension of Simple Online Realtime Tracking (SORT) with an appearance metric (deep SORT). Initial diver detection is performed with a custom CNN designed for realtime diver detection, and appearance features are subsequently extracted for each detected diver. Next, realtime tracking by-detection is performed with an extension of the deep SORT algorithm. We evaluate this technique on a series of videos of divers performing human-robot collaborative tasks and show that our methods result in more divers being accurately identified during tracking. We also discuss the practical considerations of applying multi-person tracking to on-board autonomous robot operations, and we consider how failure cases can be addressed during on-board tracking. 

   more » « less
2. Transferable two-stream convolutional neural network for human action recognition

   https://doi.org/10.1016/j/jmsy.2020.04.007  

   Xiong, Q. (January 2020, Journal of manufacturing systems)

   Human-Robot Collaboration (HRC), which enables a workspace where human and robot can dynamically and safely collaborate for improved operational efficiency, has been identified as a key element in smart manu­facturing. Human action recognition plays a key role in the realization ofHRC, as it helps identify current human action and provides the basis for future action prediction and robot planning. While Deep Learning (DL) has demonstrated great potential in advancing human action recognition, effectively leveraging the temporal in­formation of human motions to improve the accuracy and robustness of action recognition has remained as a challenge. Furthermore, it is often difficult to obtain a large volume of data for DL network training and opti­mization, due to operational constraints in a realistic manufacturing setting. This paper presents an integrated method to address these two challenges, based on the optical flow and convolutional neural network (CNN)­based transfer learning. Specifically, optical flow images, which encode the temporal information of human motion, are extracted and serve as the input to a two-stream CNN structure for simultaneous parsing of spatial­-temporal information of human motion. Subsequently, transfer learning is investigated to transfer the feature extraction capability of a pre-trained CNN to manufacturing scenarios. Evaluation using engine block assembly confirmed the effectiveness of the developed method. 

   more » « less
3. Teleoperator-Robot-Human Interaction in Manufacturing: Perspectives from Industry, Robot Manufacturers, and Researchers

   https://doi.org/10.1080/24725838.2024.2310301  

   Kim, Sunwook; Hernandez, Ivan; Nussbaum, Maury A.; Lim, Sol (February 2024, IISE Transactions on Occupational Ergonomics and Human Factors)

   Background: The increasing prevalence of robots in industrial environments is attributed in part to advancements in collaborative robot technologies, enabling robots to work in close proximity to humans. Simultaneously, the rise of teleoperation, involving remote robot control, poses unique opportunities and challenges for human-robot collaboration (HRC) in diverse and distributed workspaces. Purpose: There is not yet a comprehensive understanding of HRC in teleoperation, specifically focusing on collaborations involving the teleoperator, the robot, and the local or onsite workers in industrial settings, here referred to as teleoperator-robot-human collaboration (tRHC). We aimed to identify opportunities, challenges, and potential applications of tRHC through insights provided from industry stakeholders, thereby supporting effective future industrial implementations. Methods: Thirteen stakeholders in robotics, specializing in different domains (i.e., safety, robot manufacturing, aerospace/automotive manufacturing, and supply chains), completed semi-structured interviews that focused on exploring diverse aspects relevant to tRHC. The interviews were then transcribed and thematic analysis was applied to group responses into broader categories, which were further compared across stakeholder industries. Results We identified three main categories and 13 themes from the interviews. These categories include Benefits, Concerns, and Technical Challenges. Interviewees highlighted accessibility, ergonomics, flexibility, safety, time & cost saving, and trust as benefits of tRHC. Concerns raised encompassed safety, standards, trust, and workplace optimization. Technical challenges consisted of critical issues such as communication time delays, the need for high dexterity in robot manipulators, the importance of establishing shared situational awareness among all agents, and the potential of augmented and virtual reality in providing immersive control interfaces. Conclusions: Despite important challenges, tRHC could offer unique benefits, facilitating seamless collaboration among the teleoperator, teleoperated robot(s), and onsite workers across physical and geographic boundaries. To realize such benefits and address the challenges, we propose several research directions to further explore and develop tRHC capabilities. 

   more » « less
4. A Survey of Mental Modeling Techniques in Human–Robot Teaming

   https://doi.org/10.1007/s43154-020-00019-0  

   Tabrez, Aaquib; Luebbers, Matthew B.; Hayes, Bradley (August 2020, Current Robotics Reports)

   This paper provides a structured overview of mental model theory and methodology as applied to human–robot teaming. Also discussed are evaluation methods and metrics for various aspects of mental modeling during human–robot interaction, as well as recent emerging applications and open challenges in the field. 

   more » « less
5. Factors Influencing The Human Preferred Interaction Distance

   https://doi.org/10.1109/RO-MAN46459.2019.8956404  

   Rajamohan, Vineeth; Scully-Allison, Connor; Dascalu, Sergiu; Feil-Seifer, David (October 2019, International Conference on Robot and Human Interactive Communication)

   Nonverbal interactions are a key component of human communication. Since robots have become significant by trying to get close to human beings, it is important that they follow social rules governing the use of space. Prior research has conceptualized personal space as physical zones which are based on static distances. This work examined how preferred interaction distance can change given different interaction scenarios. We conducted a user study using three different robot heights. We also examined the difference in preferred interaction distance when a robot approaches a human and, conversely, when a human approaches a robot. Factors included in quantitative analysis are the participants' gender, robot's height, and method of approach. Subjective measures included human comfort and perceived safety. The results obtained through this study shows that robot height, participant gender and method of approach were significant factors influencing measured proxemic zones and accordingly participant comfort. Subjective data showed that experiment respondents regarded robots in a more favorable light following their participation in this study. Furthermore, the NAO was perceived most positively by respondents according to various metrics and the PR2 Tall, most negatively. 

   more » « less