An official website of the United States government
Abstract Human intention prediction plays a critical role in human–robot collaboration, as it helps robots improve efficiency and safety by accurately anticipating human intentions and proactively assisting with tasks. While current applications often focus on predicting intent once human action is completed, recognizing human intent in advance has received less attention. This study aims to equip robots with the capability to forecast human intent before completing an action, i.e., early intent prediction. To achieve this objective, we first extract features from human motion trajectories by analyzing changes in human joint distances. These features are then utilized in a Hidden Markov Model (HMM) to determine the state transition times from uncertain intent to certain intent. Second, we propose two models including a Transformer and a Bi-LSTM for classifying motion intentions. Then, we design a human–robot collaboration experiment in which the operator reaches multiple targets while the robot moves continuously following a predetermined path. The data collected through the experiment were divided into two groups: full-length data and partial data before state transitions detected by the HMM. Finally, the effectiveness of the suggested framework for predicting intentions is assessed using two different datasets, particularly in a scenario when motion trajectories are similar but underlying intentions vary. The results indicate that using partial data prior to the motion completion yields better accuracy compared to using full-length data. Specifically, the transformer model exhibits a 2% improvement in accuracy, while the Bi-LSTM model demonstrates a 6% increase in accuracy.
more »
« less
Experimental Evaluation of Human Motion Prediction Toward Safe and Efficient Human Robot Collaboration
Human motion prediction is non-trivial in modern industrial settings. Accurate prediction of human motion can not only improve efficiency in human-robot collaboration, but also enhance human safety in close proximity to robots. Although many prediction models have been proposed with various parameterization and identification approaches, some fundamental questions remain unclear: what is the necessary parameterization of a prediction model? Is online adaptation of models necessary? Can a prediction model help improve safety and efficiency during human-robot collaboration? These unaddressed questions result from the difficulty of quantitatively evaluating different prediction models in a closed-loop fashion in real human-robot interaction. This paper develops a method to evaluate the closed-loop performance of different prediction models. In particular, we compare models with different parameterizations and models with or without online parameter adaptation. Extensive experiments were conducted on a human-robot collaboration platform. The experimental results demonstrate that human motion prediction significantly enhance the collaboration efficiency and human safety. Adaptable prediction models that are parameterized by neural networks achieve better performance.
more »
« less
- Award ID(s):
- 1734109
- PAR ID:
- 10213593
- Date Published:
- Journal Name:
- Experimental Evaluation of Human Motion Prediction Toward Safe and Efficient Human Robot Collaboration
- Page Range / eLocation ID:
- 4349 to 4354
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Activity recognition is a crucial aspect in smart manufacturing and human-robot collaboration, as robots play a vital role in improving efficiency and safety by accurately recognizing human intentions and proactively assisting with tasks. Current human intention recognition applications only consider the accuracy of recognition but ignore the importance of predicting it in advance. Given human reaching movements, we want to equip the robot with the ability to predict human intent not only with precise recognition but also at an early stage. In this paper, we propose a framework to apply Transformer-based and LSTM-based models to learn motion intentions. Second, based on the observation of distances of human joints along the motion trajectory, we explore how we can use the hidden Markov model to find intent state transitions, i.e., intent uncertainty and intent certainty. Finally, two data types are generated, one for the full data and the other for the length of data before state transitions; both data are evaluated on models to assess the robustness of intention prediction. We conducted experiments in a manufacturing workspace where the experimenter reaches multiple scattered targets and further this experimental scenario was designed to examine how intents differ, but motions are only slightly different. The proposed models were then evaluated with experimental data, and further performance comparisons were made between models and between different intents. Finally, early predictions were validated to be better than using full-length data.more » « less
-
null (Ed.)To enable safe and effective human-robot collaboration (HRC) in smart manufacturing, seamless integration of sensing, cognition and prediction into the robot controller is critical for real-time awareness, response and communication inside a heterogeneous environment (robots, humans, equipment). The specific research objective is to provide the robot Proactive Adaptive Collaboration Intelligence (PACI) and switching logic within its control architecture in order to give the robot the ability to optimally and dynamically adapt its motions, given a priori knowledge and predefined execution plans for its assigned tasks. The challenge lies in augmenting the robot’s decision-making process to have greater situation awareness and to yield smart robot behaviors/reactions when subject to different levels of human-robot interaction, while maintaining safety and production efficiency. Robot reactive behaviors were achieved via cost function-based switching logic activating the best suited high-level controller. The PACI’s underlying segmentation and switching logic framework is demonstrated to yield a high degree of modularity and flexibility. The performance of the developed control structure subjected to different levels of human-robot interactions was validated in a simulated environment. Open-loop commands were sent to the physical e.DO robot to demonstrate how the proposed framework would behave in a real application.more » « less
-
null (Ed.)Abstract To enable safe and effective human–robot collaboration (HRC) in smart manufacturing, seamless integration of sensing, cognition, and prediction into the robot controller is critical for real-time awareness, response, and communication inside a heterogeneous environment (robots, humans, and equipment). The specific research objective is to provide the robot Proactive Adaptive Collaboration Intelligence (PACI) and switching logic within its control architecture in order to give the robot the ability to optimally and dynamically adapt its motions, given a priori knowledge and predefined execution plans for its assigned tasks. The challenge lies in augmenting the robot’s decision-making process to have greater situation awareness and to yield smart robot behaviors/reactions when subject to different levels of human–robot interaction, while maintaining safety and production efficiency. Robot reactive behaviors were achieved via cost function-based switching logic activating the best suited high-level controller. The PACI’s underlying segmentation and switching logic framework is demonstrated to yield a high degree of modularity and flexibility. The performance of the developed control structure subjected to different levels of human–robot interactions was validated in a simulated environment. Open-loop commands were sent to the physical e.DO robot to demonstrate how the proposed framework would behave in a real application.more » « less
-
Robots present an innovative solution to the construction industry’s challenges, including safety concerns, skilled worker shortages, and productivity issues. Successfully collaborating with robots requires new competencies to ensure safety, smooth interaction, and accelerated adoption of robotic technologies. However, limited research exists on the specific competencies needed for human—robot collaboration in construction. Moreover, the perspectives of construction industry professionals on these competencies remain underexplored. This study examines the perceptions of construction industry professionals regarding the knowledge, skills, and abilities necessary for the effective implementation of human—robot collaboration in construction. A two-round Delphi survey was conducted with expert panel members from the construction industry to assess their views on the competencies for human—robot collaboration. The results reveal that the most critical competencies include knowledge areas such as human—robot interface, construction robot applications, human—robot collaboration safety and standards, task planning and robot control system; skills such as task planning, safety management, technical expertise, human—robot interface, and communication; and abilities such as safety awareness, continuous learning, problemsolving, critical thinking, and spatial awareness. This study contributes to knowledge by identifying the most significant competencies for human—robot collaboration in construction and highlighting their relative importance. These competencies could inform the design of educational and training programs and facilitate the integration of robotic technologies in construction. The findings also provide a foundation for future research to further explore and enhance these competencies, ultimately supporting safer, more efficient, and more productive construction practices.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.23919/ACC45564.2020.9147277
