An official website of the United States government
Abstract Human intention prediction plays a critical role in human–robot collaboration, as it helps robots improve efficiency and safety by accurately anticipating human intentions and proactively assisting with tasks. While current applications often focus on predicting intent once human action is completed, recognizing human intent in advance has received less attention. This study aims to equip robots with the capability to forecast human intent before completing an action, i.e., early intent prediction. To achieve this objective, we first extract features from human motion trajectories by analyzing changes in human joint distances. These features are then utilized in a Hidden Markov Model (HMM) to determine the state transition times from uncertain intent to certain intent. Second, we propose two models including a Transformer and a Bi-LSTM for classifying motion intentions. Then, we design a human–robot collaboration experiment in which the operator reaches multiple targets while the robot moves continuously following a predetermined path. The data collected through the experiment were divided into two groups: full-length data and partial data before state transitions detected by the HMM. Finally, the effectiveness of the suggested framework for predicting intentions is assessed using two different datasets, particularly in a scenario when motion trajectories are similar but underlying intentions vary. The results indicate that using partial data prior to the motion completion yields better accuracy compared to using full-length data. Specifically, the transformer model exhibits a 2% improvement in accuracy, while the Bi-LSTM model demonstrates a 6% increase in accuracy.
more »
« less
Optimal Real-time Scheduling of Human Attention for a Human and Multi-robot Collaboration System
We analyze a human and multi-robot collaboration system and propose a method to optimally schedule the human attention when a human operator receives collaboration requests from multiple robots at the same time. We formulate the human attention scheduling problem as a binary optimization problem which aims to maximize the overall performance among all the robots, under the constraint that a human has limited attention capacity. We first present the optimal schedule for the human to determine when to collaborate with a robot if there is no contention occurring among robots' collaboration requests. For the moments when contentions occur, we present a contention-resolving Model Predictive Control (MPC) method to dynamically schedule the human attention and determine which robot the human should collaborate with first. The optimal schedule can then be determined using a sampling based approach. The effectiveness of the proposed method is validated through simulation that shows improvements.
more »
« less
- PAR ID:
- 10212082
- Date Published:
- Journal Name:
- Proceedings of the 2020 American Control Conference
- Page Range / eLocation ID:
- 30 to 35
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Collaborative robots provide prospective and great solutions to human–robot cooperative tasks. In this paper, we present a comprehensive review for two significant topics in human–robot interaction: robots learning from demonstrations and human comfort. The collaboration quality between the human and the robot has been improved largely by taking advantage of robots learning from demonstrations. Human teaching and robot learning approaches with their corresponding applications are investigated in this review. We also discuss several important issues that need to be paid attention to and addressed in the human–robot teaching–learning process. After that, the factors that may affect human comfort in human–robot interaction are described and discussed. Moreover, the measures utilized to improve human acceptance of robots and human comfort in human–robot interaction are also presented and discussed.more » « less
-
Construction robots have drawn increased attention as a potential means of improving construction safety and productivity. However, it is still challenging to ensure safe human-robot collaboration on dynamic and unstructured construction workspaces. On construction sites, multiple entities dynamically collaborate with each other and the situational context between them evolves continually. Construction robots must therefore be equipped to visually understand the scene’s contexts (i.e., semantic relations to surrounding entities), thereby safely collaborating with humans, as a human vision system does. Toward this end, this study builds a unique deep neural network architecture and develops a construction-specialized model by experimenting multiple fine-tuning scenarios. Also, this study evaluates its performance on real construction operations data in order to examine its potential toward real-world applications. The results showed the promising performance of the tuned model: the recall@5 on training and validation dataset reached 92% and 67%, respectively. The proposed method, which supports construction co-robots with the holistic scene understanding, is expected to contribute to promoting safer human-robot collaboration in construction.more » « less
-
As humans and robots start to collaborate in close proximity, robots are tasked to perceive, comprehend, and anticipate human partners' actions, which demands a predictive model to describe how humans collaborate with each other in joint actions. Previous studies either simplify the collaborative task as an optimal control problem between two agents or do not consider the learning process of humans during repeated interaction. This idyllic representation is thus not able to model human rationality and the learning process. In this paper, a bounded-rational and game-theoretical human cooperative model is developed to describe the cooperative behaviors of the human dyad. An experiment of a joint object pushing collaborative task was conducted with 30 human subjects using haptic interfaces in a virtual environment. The proposed model uses inverse optimal control (IOC) to model the reward parameters in the collaborative task. The collected data verified the accuracy of the predicted human trajectory generated from the bounded rational model excels the one with a fully rational model. We further provide insight from the conducted experiments about the effects of leadership on the performance of human collaboration.more » « less
-
Product disassembly is essential for remanufacturing operations and recovery of end-of-use devices. However, disassembly has often been performed manually with significant safety issues for human workers. Recently, human-robot collaboration has become popular to reduce the human workload and handle hazardous materials. However, due to the current limitations of robots, they are not fully capable of performing every disassembly task. It is critical to determine whether a robot can accomplish a specific disassembly task. This study develops a disassembly score which represents how easy is to disassemble a component by robots, considering the attributes of the component along with the robotic capability. Five factors, including component weight, shape, size, accessibility, and positioning, are considered when developing the disassembly score. Further, the relationship between the five factors and robotic capabilities, such as grabbing and placing, is discussed. The MaxViT (Multi-Axis Vision Transformer) model is used to determine component sizes through image processing of the XPS 8700 desktop, demonstrating the potential for automating disassembly score generation. Moreover, the proposed disassembly score is discussed in terms of determining the appropriate work setting for disassembly operations, under three main categories: human-robot collaboration (HRC), semi-HRC, and worker-only settings. A framework for calculating disassembly time, considering human-robot collaboration, is also proposed.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.23919/ACC45564.2020.9147782
