An official website of the United States government
Abstract Human intention prediction plays a critical role in human–robot collaboration, as it helps robots improve efficiency and safety by accurately anticipating human intentions and proactively assisting with tasks. While current applications often focus on predicting intent once human action is completed, recognizing human intent in advance has received less attention. This study aims to equip robots with the capability to forecast human intent before completing an action, i.e., early intent prediction. To achieve this objective, we first extract features from human motion trajectories by analyzing changes in human joint distances. These features are then utilized in a Hidden Markov Model (HMM) to determine the state transition times from uncertain intent to certain intent. Second, we propose two models including a Transformer and a Bi-LSTM for classifying motion intentions. Then, we design a human–robot collaboration experiment in which the operator reaches multiple targets while the robot moves continuously following a predetermined path. The data collected through the experiment were divided into two groups: full-length data and partial data before state transitions detected by the HMM. Finally, the effectiveness of the suggested framework for predicting intentions is assessed using two different datasets, particularly in a scenario when motion trajectories are similar but underlying intentions vary. The results indicate that using partial data prior to the motion completion yields better accuracy compared to using full-length data. Specifically, the transformer model exhibits a 2% improvement in accuracy, while the Bi-LSTM model demonstrates a 6% increase in accuracy.
more »
« less
Experimental Evaluation of Human Motion Prediction Toward Safe and Efficient Human Robot Collaboration
Human motion prediction is non-trivial in modern industrial settings. Accurate prediction of human motion can not only improve efficiency in human-robot collaboration, but also enhance human safety in close proximity to robots. Although many prediction models have been proposed with various parameterization and identification approaches, some fundamental questions remain unclear: what is the necessary parameterization of a prediction model? Is online adaptation of models necessary? Can a prediction model help improve safety and efficiency during human-robot collaboration? These unaddressed questions result from the difficulty of quantitatively evaluating different prediction models in a closed-loop fashion in real human-robot interaction. This paper develops a method to evaluate the closed-loop performance of different prediction models. In particular, we compare models with different parameterizations and models with or without online parameter adaptation. Extensive experiments were conducted on a human-robot collaboration platform. The experimental results demonstrate that human motion prediction significantly enhance the collaboration efficiency and human safety. Adaptable prediction models that are parameterized by neural networks achieve better performance.
more »
« less
- Award ID(s):
- 1734109
- PAR ID:
- 10213593
- Date Published:
- Journal Name:
- Experimental Evaluation of Human Motion Prediction Toward Safe and Efficient Human Robot Collaboration
- Page Range / eLocation ID:
- 4349 to 4354
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Activity recognition is a crucial aspect in smart manufacturing and human-robot collaboration, as robots play a vital role in improving efficiency and safety by accurately recognizing human intentions and proactively assisting with tasks. Current human intention recognition applications only consider the accuracy of recognition but ignore the importance of predicting it in advance. Given human reaching movements, we want to equip the robot with the ability to predict human intent not only with precise recognition but also at an early stage. In this paper, we propose a framework to apply Transformer-based and LSTM-based models to learn motion intentions. Second, based on the observation of distances of human joints along the motion trajectory, we explore how we can use the hidden Markov model to find intent state transitions, i.e., intent uncertainty and intent certainty. Finally, two data types are generated, one for the full data and the other for the length of data before state transitions; both data are evaluated on models to assess the robustness of intention prediction. We conducted experiments in a manufacturing workspace where the experimenter reaches multiple scattered targets and further this experimental scenario was designed to examine how intents differ, but motions are only slightly different. The proposed models were then evaluated with experimental data, and further performance comparisons were made between models and between different intents. Finally, early predictions were validated to be better than using full-length data.more » « less
-
null (Ed.)To enable safe and effective human-robot collaboration (HRC) in smart manufacturing, seamless integration of sensing, cognition and prediction into the robot controller is critical for real-time awareness, response and communication inside a heterogeneous environment (robots, humans, equipment). The specific research objective is to provide the robot Proactive Adaptive Collaboration Intelligence (PACI) and switching logic within its control architecture in order to give the robot the ability to optimally and dynamically adapt its motions, given a priori knowledge and predefined execution plans for its assigned tasks. The challenge lies in augmenting the robot’s decision-making process to have greater situation awareness and to yield smart robot behaviors/reactions when subject to different levels of human-robot interaction, while maintaining safety and production efficiency. Robot reactive behaviors were achieved via cost function-based switching logic activating the best suited high-level controller. The PACI’s underlying segmentation and switching logic framework is demonstrated to yield a high degree of modularity and flexibility. The performance of the developed control structure subjected to different levels of human-robot interactions was validated in a simulated environment. Open-loop commands were sent to the physical e.DO robot to demonstrate how the proposed framework would behave in a real application.more » « less
-
null (Ed.)Abstract To enable safe and effective human–robot collaboration (HRC) in smart manufacturing, seamless integration of sensing, cognition, and prediction into the robot controller is critical for real-time awareness, response, and communication inside a heterogeneous environment (robots, humans, and equipment). The specific research objective is to provide the robot Proactive Adaptive Collaboration Intelligence (PACI) and switching logic within its control architecture in order to give the robot the ability to optimally and dynamically adapt its motions, given a priori knowledge and predefined execution plans for its assigned tasks. The challenge lies in augmenting the robot’s decision-making process to have greater situation awareness and to yield smart robot behaviors/reactions when subject to different levels of human–robot interaction, while maintaining safety and production efficiency. Robot reactive behaviors were achieved via cost function-based switching logic activating the best suited high-level controller. The PACI’s underlying segmentation and switching logic framework is demonstrated to yield a high degree of modularity and flexibility. The performance of the developed control structure subjected to different levels of human–robot interactions was validated in a simulated environment. Open-loop commands were sent to the physical e.DO robot to demonstrate how the proposed framework would behave in a real application.more » « less
-
Advancements in collaborative robotics have significantly enhanced the potential for human–robot collaboration in manufacturing. To achieve efficient and user-friendly collaboration, prior research has predominantly focused on the robot’s perspective, including aspects such as planning, control, and adaptation. A key approach in this domain has been the recognition of human intentions to inform robot actions. However, true collaboration necessitates bidirectional communication, where both human and robot are aware of each other’s intentions. A lack of transparency in robot actions can lead to discomfort, reduced safety, and inefficiencies in the collaborative process. This study investigates the communication of robot intentions to human operators through mixed reality (MR) and evaluates its impact on human–robot collaboration. A laboratory-based physical human–robot assembly framework is developed, integrating multiple MR-based intention communication strategies. Experimental evaluations are conducted to assess the effectiveness of these strategies. The results demonstrate that conveying robot intentions via MR enhances work efficiency, trust, and user comfort in human–robot collaborative manufacturing. Furthermore, a comparative analysis of different MR-based communication designs provides insights into the optimal approaches for improving collaboration quality.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Paper:
- https://doi.org/10.23919/ACC45564.2020.9147277
