An official website of the United States government
This paper explores the tradeoffs between different types of mixed reality robotic communication under different levels of user workload. We present the results of a within-subjects experiment in which we systematically and jointly vary robot communication style alongside level and type of cognitive load, and measure subsequent impacts on accuracy, reaction time, and perceived workload and effectiveness. Our preliminary results suggest that although humans may not notice differences, the manner of load a user is under and the type of communication style used by a robot they interact with do in fact interact to determine their task effectiveness
more »
« less
This content will become publicly available on December 27, 2025
Cognitive Load-Based Affective Workload Allocation for Multihuman Multirobot Teams
The interaction and collaboration between humans and multiple robots represent a novel field of research known as human multirobot systems. Adequately designed systems within this field allow teams composed of both humans and robots to work together effectively on tasks, such as monitoring, exploration, and search and rescue operations. This article presents a deep reinforcement learning-based affective workload allocation controller specifically for multihuman multirobot teams. The proposed controller can dynamically reallocate workloads based on the performance of the operators during collaborative missions with multirobot systems. The operators' performances are evaluated through the scores of a self-reported questionnaire (i.e., subjective measurement) and the results of a deep learning-based cognitive workload prediction algorithm that uses physiological and behavioral data (i.e., objective measurement). To evaluate the effectiveness of the proposed controller, we conduct an exploratory user experiment with various allocation strategies. The user experiment uses a multihuman multirobot CCTV monitoring task as an example and carry out comprehensive real-world experiments with 32 human subjects for both quantitative measurement and qualitative analysis. Our results demonstrate the performance and effectiveness of the proposed controller and highlight the importance of incorporating both subjective and objective measurements of the operators' cognitive workload as well as seeking consent for workload transitions, to enhance the performance of multihuman multirobot teams.
more »
« less
- Award ID(s):
- 1846221
- PAR ID:
- 10595332
- Publisher / Repository:
- IEEE
- Date Published:
- Journal Name:
- IEEE Transactions on Human-Machine Systems
- Volume:
- 55
- Issue:
- 1
- ISSN:
- 2168-2291
- Page Range / eLocation ID:
- 23 to 36
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Collaborative robots promise to transform work across many industries and promote “human-robot teaming” as a novel paradigm. However, realizing this promise requires the understanding of how existing tasks, developed for and performed by humans, can be effectively translated into tasks that robots can singularly or human-robot teams can collaboratively perform. In the interest of developing tools that facilitate this process we present Authr, an end-to-end task authoring environment that assists engineers at manufacturing facilities in translating existing manual tasks into plans applicable for human-robot teams and simulates these plans as they would be performed by the human and robot. We evaluated Authr with two user studies, which demonstrate the usability and effectiveness of Authr as an interface and the benefits of assistive task allocation methods for designing complex tasks for human-robot teams. We discuss the implications of these findings for the design of software tools for authoring human-robot collaborative plans.more » « less
-
High mental workload, in addition to changes in workload, can negatively affect operators, but it is not clear how sudden versus gradual workload transitions influence performance and visual attention allocation. This knowledge is important as sudden shifts in workload are common in multitasking domains. The objective of this study was to investigate, using performance and eye tracking metrics, how constant versus variable levels of workload affect operators in the context of a dual-task paradigm. An unmanned aerial vehicle command and control simulation varied task load between low, high, gradually transitioning from low to high, and suddenly transitioning from low to high. Performance on a primary and secondary task and several eye tracking measures were calculated. There was no significant difference between sudden and gradual workload transitions in terms of performance or attention allocation overall; however, both sudden and gradual workload transitions changed participants’ strategy in dealing with the primary and secondary task as compared to low/high workload. Also, eye tracking metrics that are not frequently used, such as transition rate and stationary entropy, provided more insight into performance differences. These metrics can potentially be used to better understand operators’ strategies and could form the basis of an adaptive display.more » « less
-
Objective: Physical and cognitive workloads and performance were studied for a corrective shared control (CSC) human–robot collaborative (HRC) sanding task. Background: Manual sanding is physically demanding. Collaborative robots (cobots) can potentially reduce physical stress, but fully autonomous implementation has been particularly challenging due to skill, task variability, and robot limitations. CSC is an HRC method where the robot operates semiautonomously while the human provides real-time corrections. Methods: Twenty laboratory participants removed paint using an orbital sander, both manually and with a CSC robot. A fully automated robot was also tested. Results: The CSC robot improved subjective discomfort compared to manual sanding in the upper arm by 29.5%, lower arm by 32%, hand by 36.5%, front of the shoulder by 24%, and back of the shoulder by 17.5%. Muscle fatigue measured using EMG, was observed in the medial deltoid and flexor carpi radialis for the manual condition. The composite cognitive workload on the NASA-TLX increased by 14.3% for manual sanding due to high physical demand and effort, while mental demand was 14% greater for the CSC robot. Digital imaging showed that the CSC robot outperformed the automated condition by 7.16% for uniformity, 4.96% for quantity, and 6.06% in total. Conclusions: In this example, we found that human skills and techniques were integral to sanding and can be successfully incorporated into HRC systems. Humans performed the task using the CSC robot with less fatigue and discomfort. Applications: The results can influence implementation of future HRC systems in manufacturing environments.more » « less
-
The emergence and recent development of collaborative robots have introduced a safer and more efficient human–robot collaboration (HRC) manufacturing environment. Since the release of COBOTs, a great amount of research efforts have been focused on improving robot working efficiency, user safety, human intention detection, etc., while one significant factor—human comfort—has been frequently ignored. The comfort factor is critical to COBOT users due to its great impact on user acceptance. In previous studies, there is a lack of a mathematical-model-based approach to quantitatively describe and predict human comfort in HRC scenarios. Also, few studies have discussed the cases when multiple comfort factors take effect simultaneously. In this study, a multi-linear-regression-based general human comfort prediction model is proposed under human–robot collaboration scenarios, which is able to accurately predict the comfort levels of humans in multi-factor situations. The proposed method in this paper tackled these two gaps at the same time and also demonstrated the effectiveness of the approach with its high prediction accuracy. The overall average accuracy among all participants is 81.33%, while the overall maximum value is 88.94%, and the overall minimum value is 72.53%. The model uses subjective comfort rating feedback from human subjects as training and testing data. Experiments have been implemented, and the final results proved the effectiveness of the proposed approach in identifying human comfort levels in HRC.more » « less
