Title: Cognitive-Aware Multi-modal Human-Swarm Interface for Optimal Collaboration
Abstract This paper presents the development of an interface that enables humans to collaborate with robot swarms (RSs) to effectively accomplish missions in highly uncertain environments. Autonomous RS technology has advanced to the point where robots can perform various tasks without human intervention. However, there are still two major uncertainties in mission execution. The first uncertainty arises from the limitations of artificial intelligence (AI), such as the inaccuracy of computer vision systems. The second uncertainty comes from the decision-making aspect, where the mission requirements may not be accurately communicated to the RS. To overcome these uncertainties, it is crucial to design a system that allows humans to monitor the RS and actively participate in critical decision-making processes during missions. We design an interface to: 1) estimate human cognitive states to monitor the human partner, 2) determine the best communication methods to calibrate cognitive states, and 3) allocate functions among humans and robots to optimize team performance. We demonstrate this approach through drone search and rescue (SAR) missions.  more » « less
Award ID(s):
1836952
PAR ID:
10661372
Author(s) / Creator(s):
; ;
Publisher / Repository:
Springer Nature Singapore
Date Published:
Page Range / eLocation ID:
155 to 164
Subject(s) / Keyword(s):
Sponsoring Org:
National Science Foundation
More Like this
  1. Unmanned systems are becoming increasingly engaged in disaster response. Human error in these applications can have severe consequences and emergency managers appear reluctant to adopt robots. This paper presents a taxonomy of normal and off-normal scenarios that, when combined with a model of impacts on cognitive and attentional resources, specify sources of human error in field robotics. In an emergency, a human is under time and consequences pressure, regardless of whether the mission is routine or whether the event requires a change in the robot, the mission, the robot’s work envelope, the interaction of the humans engaged with the robot, or their work envelope. For example, at Hurricane Michael, unmanned aerial systems were used for standard visual survey missions with minor human errors but the same systems were used at the Kilauea volcanic eruption for novel missions with more notable human errors. An examination of two cases studies suggests the physiological and psychological effects of an emergency may be the primary source of human error. 
    more » « less
  2. Rational decision-making is crucial in the later stages of engineering system design to allocate resources efficiently and minimize costs. However, human rationality is bounded by cognitive biases and limitations. Understanding how humans deviate from rationality is critical for guiding designers toward better design outcomes. In this paper, we quantify designer rationality in competitive scenarios based on utility theory. Using an experiment inspired by crowd-sourced contests, we show that designers employ varied search strategies. Some participants approximate a Bayesian agent that aimed to maximize its expected utility. Those with higher rationality reduce uncertainty more effectively. Furthermore, rationality correlates with both the proximity to optimal design and design iteration costs, with winning participants exhibiting greater rationality than losing participants. 
    more » « less
  3. Many domains of AI and its effects are established, which mainly rely on their integration modeling cognition of human and AI agents, collecting and representing knowledge using them at the human level, and maintaining decision-making processes towards physical action eligible to and in cooperation with humans. Especially in human-robot interaction, many AI and robotics technologies are focused on human- robot cognitive modeling, from visual processing to symbolic reasoning and from reactive control to action recognition and learning, which will support human-multi-agent cooperative achieving tasks. However, the main challenge is efficiently combining human motivations and AI agents’ purposes in a sharing architecture and reaching a consensus in complex environments and missions. To fill this gap, this workshop brings together researchers from different communities inter- ested in multi-agent systems (MAS) and human-robot interaction (HRI) to explore potential approaches, future research directions, and domains in human-multi-agent cognitive fusion. 
    more » « less
  4. Abstract AI assistance is readily available to humans in a variety of decision-making applications. In order to fully understand the efficacy of such joint decision-making, it is important to first understand the human’s reliance on AI. However, there is a disconnect between how joint decision-making is studied and how it is practiced in the real world. More often than not, researchers ask humans to provide independent decisions before they are shown AI assistance. This is done to make explicit the influence of AI assistance on the human’s decision. We develop a cognitive model that allows us to infer the latent reliance strategy of humans on AI assistance without asking the human to make an independent decision. We validate the model’s predictions through two behavioral experiments. The first experiment follows a concurrent paradigm where humans are shown AI assistance alongside the decision problem. The second experiment follows a sequential paradigm where humans provide an independent judgment on a decision problem before AI assistance is made available. The model’s predicted reliance strategies closely track the strategies employed by humans in the two experimental paradigms. Our model provides a principled way to infer reliance on AI-assistance and may be used to expand the scope of investigation on human-AI collaboration. 
    more » « less
  5. One of the early goals of artificial intelligence (AI) was to create algorithms that exhibited behavior indistinguishable from human behavior (i.e., human-like behavior). Today, AI has diverged, often aiming to excel in tasks inspired by human capabilities and outperform humans, rather than replicating human cogntion and action. In this paper, I explore the overarching question of whether computational algorithms have achieved this initial goal of AI. I focus on dynamic decision-making, approaching the question from the perspective of computational cognitive science. I present a general cognitive algorithm that intends to emulate human decision-making in dynamic environments, as defined in instance-based learning theory (IBLT). I use the cognitive steps proposed in IBLT to organize and discuss current evidence that supports some of the human-likeness of the decision-making mechanisms. I also highlight the significant gaps in research that are required to improve current models and to create higher fidelity in computational algorithms to represent human decision processes. I conclude with concrete steps toward advancing the construction of algorithms that exhibit human-like behavior with the ultimate goal of supporting human dynamic decision-making. 
    more » « less