A prerequisite for social coordination is bidirectional communication between teammates, each playing two roles simultaneously: as receptive listeners and expressive speakers. For robots working with humans in complex situations with multiple goals that differ in importance, failure to fulfill the expectation of either role could undermine group performance due to misalignment of values between humans and robots. Specifically, a robot needs to serve as an effective listener to infer human users’ intents from instructions and feedback and as an expressive speaker to explain its decision processes to users. Here, we investigate how to foster effective bidirectional human-robot communications in the context of value alignment—collaborative robots and users form an aligned understanding of the importance of possible task goals. We propose an explainable artificial intelligence (XAI)
system in which a group of robots predicts users’ values by taking in situ feedback into consideration while communicating their decision processes to users through explanations. To learn from human feedback, our XAI system integrates a cooperative communication model for inferring human values associated with multiple desirable goals. To be interpretable to humans, the system simulates human mental dynamics and predicts optimal explanations using graphical models. We conducted psychological experiments to examine the core components of the proposed computational framework. Our results show that real-time human-robot mutual understanding in complex cooperative tasks is achievable with a learning model based on bidirectional communication. We believe that this interaction framework can shed light on bidirectional value alignment in communicative XAI systems and, more broadly, in future human-machine teaming systems.
more »
« less
Pragmatic-Pedagogic Value Alignment
As intelligent systems gain autonomy and capability, it becomes vital to
ensure that their objectives match those of their human users; this is known as the
value-alignment problem. In robotics, value alignment is key to the design of collaborative
robots that can integrate into human workflows, successfully inferring and
adapting to their users’ objectives as they go.We argue that a meaningful solution to
value alignment must combine multi-agent decision theory with rich mathematical
models of human cognition, enabling robots to tap into people’s natural collaborative
capabilities. We present a solution to the cooperative inverse reinforcement
learning (CIRL) dynamic game based on well-established cognitive models of decision
making and theory of mind. The solution captures a key reciprocity relation: the
human will not plan her actions in isolation, but rather reason pedagogically about
how the robot might learn from them; the robot, in turn, can anticipate this and interpret
the human’s actions pragmatically. To our knowledge, this work constitutes the
first formal analysis of value alignment grounded in empirically validated cognitive
models.
more »
« less
- Award ID(s):
- 1734633
- NSF-PAR ID:
- 10063837
- Date Published:
- Journal Name:
- International Symposium on Robotics Research (ISRR)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract Despite theoretical benefits of collaborative robots, disappointing outcomes are well documented by clinical studies, spanning rehabilitation, prostheses, and surgery. Cognitive load theory provides a possible explanation for why humans in the real world are not realizing the benefits of collaborative robots: high cognitive loads may be impeding human performance. Measuring cognitive availability using an electrocardiogram, we ask 25 participants to complete a virtual-reality task alongside an invisible agent that determines optimal performance by iteratively updating the Bellman equation. Three robots assist by providing environmental information relevant to task performance. By enabling the robots to act more autonomously—managing more of their own behavior with fewer instructions from the human—here we show that robots can augment participants’ cognitive availability and decision-making. The way in which robots describe and achieve their objective can improve the human’s cognitive ability to reason about the task and contribute to human–robot collaboration outcomes. Augmenting human cognition provides a path to improve the efficacy of collaborative robots. By demonstrating how robots can improve human cognition, this work paves the way for improving the cognitive capabilities of first responders, manufacturing workers, surgeons, and other future users of collaborative autonomy systems.
-
more » « less
Abstract Collaboration enables multiple actors with different objectives to work together and achieve a goal beyond individual capabilities. However, strategic uncertainty from partners' actions introduces a potential for losses under failed collaboration relative to pursuing an independent system. The fundamental tradeoff between high‐value but uncertain outcomes from collaborative systems and lower‐value but more certain outcomes for independent systems induces a bistability strategic dynamic. Actors exhibit different risk attitudes that impact decisions under uncertainty which complicate shared understanding of collaborative dynamics. This paper investigates how risk attitudes affect design and strategy decisions in collaborative systems through the lens of game theory. First, an analytical model studies the effect of differential risk attitudes in a two‐actor problem with stag‐hunting strategic dynamics formulated as single‐ and bi‐level games. Next, a simulation model pairs actors with different risk attitudes in a 29‐game tournament based on a prior behavioral experiment. Results show that outcomes collaborative design problems change based on the risk attitudes of both actors. Results also emphasize that considering conservative lower‐level design options facilitates collaboration by providing risk‐averse actors with a safer solution. By accepting that decision‐making actors are not all risk‐neutral, future work seeks to develop new design methods to strengthen the adoption of efficient collaborative solutions.
-
null (Ed.)Explainability has emerged as a critical AI research objective, but the breadth of proposed methods and application domains suggest that criteria for explanation vary greatly. In particular, what counts as a good explanation, and what kinds of explanation are computationally feasible, has become trickier in light of oqaque “black box” systems such as deep neural networks. Explanation in such cases has drifted from what many philosophers stipulated as having to involve deductive and causal principles to mere “interpretation,” which approximates what happened in the target system to varying degrees. However, such post hoc constructed rationalizations are highly problematic for social robots that operate interactively in spaces shared with humans. For in such social contexts, explanations of behavior, and, in particular, justifications for violations of expected behavior, should make reference to socially accepted principles and norms. In this article, we show how a social robot’s actions can face explanatory demands for how it came to act on its decision, what goals, tasks, or purposes its design had those actions pursue and what norms or social constraints the system recognizes in the course of its action. As a result, we argue that explanations for social robots will need to be accurate representations of the system’s operation along causal, purposive, and justificatory lines. These explanations will need to generate appropriate references to principles and norms—explanations based on mere “interpretability” will ultimately fail to connect the robot’s behaviors to its appropriate determinants. We then lay out the foundations for a cognitive robotic architecture for HRI, together with particular component algorithms, for generating explanations and engaging in justificatory dialogues with human interactants. Such explanations track the robot’s actual decision-making and behavior, which themselves are determined by normative principles the robot can describe and use for justifications.more » « less
-
more » « less
Truly collaborative scientific field data collection between human scientists and autonomous robot systems requires a shared understanding of the search objectives and tradeoffs faced when making decisions. Therefore, critical to developing intelligent robots to aid human experts is an understanding of how scientists make such decisions and how they adapt their data collection strategies when presented with new information
in situ . In this study, we examined the dynamic data collection decisions of 108 expert geoscience researchers using a simulated field scenario. Human data collection behaviors suggested two distinct objectives: an information-based objective to maximize information coverage and a discrepancy-based objective to maximize hypothesis verification. We developed a highly simplified quantitative decision model that allows the robot to predict potential human data collection locations based on the two observed human data collection objectives. Predictions from the simple model revealed a transition from information-based to discrepancy-based objective as the level of information increased. The findings will allow robotic teammates to connect experts’ dynamic science objectives with the adaptation of their sampling behaviors and, in the long term, enable the development of more cognitively compatible robotic field assistants.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
