Teamwork is a set of interrelated reasoning, actions and behaviors
of team members that facilitate common objectives.
Teamwork theory and experiments have resulted in a set of
states and processes for team effectiveness in both human-human
and agent-agent teams. However, human-agent teaming
is less well studied because it is so new and involves
asymmetry in policy and intent not present in human teams.
To optimize team performance in human-agent teaming, it
is critical that agents infer human intent and adapt their polices
for smooth coordination. Most literature in human-agent
teaming builds agents referencing a learned human model.
Though these agents are guaranteed to perform well with the
learned model, they lay heavy assumptions on human policy
such as optimality and consistency, which is unlikely in many
real-world scenarios. In this paper, we propose a novel adaptive
agent architecture in human-model-free setting on a two-player
cooperative game, namely Team Space Fortress (TSF).
Previous human-human team research have shown complementary
policies in TSF game and diversity in human players’
skill, which encourages us to relax the assumptions on
human policy. Therefore, we discard learning human models
from human data, and instead use an adaptation strategy on
a pre-trained library of exemplar policies composed of RL
algorithms or rule-based methods with minimal assumptions
of human behavior. The adaptation strategy relies on a novel
similarity metric to infer human policy and then selects the
most complementary policy in our library to maximize the
team performance. The adaptive agent architecture can be deployed
in real-time and generalize to any off-the-shelf static
agents. We conducted human-agent experiments to evaluate
the proposed adaptive agent framework, and demonstrated
the suboptimality, diversity, and adaptability of human policies
in human-agent teams.
more »
« less
Ad Hoc Teaming Through Evolution
Cooperative Co-evolutionary Algorithms effectively train policies in multiagent systems with a single, statically defined team. However, many real-world problems, such as search and rescue, require agents to operate in multiple teams. When the structure of the team changes, these policies show reduced performance as they were trained to cooperate with only one team. In this work, we solve the cooperation problem by training agents to fill the needs of an arbitrary team, thereby gaining the ability to support a large variety of teams. We introduce Ad hoc Teaming Through Evolution (ATTE) which evolves a limited number of policy types using fitness aggregation across multiple teams. ATTE leverages agent types to reduce the dimensionality of the interaction search space, while fitness aggregation across teams selects for more adaptive policies. In a simulated multi-robot exploration task, ATTE is able to learn policies that are effective in a variety of teaming schemes, improving the performance of CCEA by a factor of up to five times.
more »
« less
- Award ID(s):
- 1815886
- NSF-PAR ID:
- 10294969
- Date Published:
- Journal Name:
- Genetic and evolutionary computation
- ISSN:
- 1932-0175
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Silva, S ; Paquete, L (Ed.)Coevolving teams of agents promises effective solutions for many coordination tasks such as search and rescue missions or deep ocean exploration. Good team performance in such domains generally relies on agents discovering complex joint policies, which is particularly difficult when the fitness functions are sparse (where many joint policies return the same or even zero fitness values). In this paper, we introduce Novelty Seeking Multiagent Evolutionary Reinforcement Learning (NS-MERL), which enables agents to more efficiently explore their joint strategy space. The key insight of NS-MERL is to promote good exploratory behaviors for individual agents using a dense, novelty-based fitness function. Though the overall team-level performance is still evaluated via a sparse fitness function, agents using NS-MERL more efficiently explore their joint action space and more readily discover good joint policies. Our results in complex coordination tasks show that teams of agents trained with NS-MERL perform significantly better than agents trained solely with task-specific fitnesses.more » « less
-
Trust has been identified as a central factor for effective human-robot teaming. Existing literature on trust modeling predominantly focuses on dyadic human-autonomy teams where one human agent interacts with one robot. There is little, if not no, research on trust modeling in teams consisting of multiple human agents and multiple robotic agents. To fill this research gap, we present the trust inference and propagation (TIP) model for trust modeling in multi-human multi-robot teams. We assert that in a multi-human multi-robot team, there exist two types of experiences that any human agent has with any robot: direct and indirect experiences. The TIP model presents a novel mathematical framework that explicitly accounts for both types of experiences. To evaluate the model, we conducted a human-subject experiment with 15 pairs of participants (N=30). Each pair performed a search and detection task with two drones. Results show that our TIP model successfully captured the underlying trust dynamics and significantly outperformed a baseline model. To the best of our knowledge, the TIP model is the first mathematical framework for computational trust modeling in multi-human multi-robot teams.more » « less
-
Agmon, N ; An, B ; Ricci, A ; Yeoh, W. (Ed.)In multiagent systems that require coordination, agents must learn diverse policies that enable them to achieve their individual and team objectives. Multiagent Quality-Diversity methods partially address this problem by filtering the joint space of policies to smaller sub-spaces that make the diversification of agent policies tractable. However, in teams of asymmetric agents (agents with different objectives and capabilities), the search for diversity is primarily driven by the need to find policies that will allow agents to assume complementary roles required to work together in teams. This work introduces Asymmetric Island Model (AIM), a multiagent framework that enables populations of asymmetric agents to learn diverse complementary policies that foster teamwork via dynamic population size allocation on a wide variety of team tasks. The key insight of AIM is that the competitive pressure arising from the distribution of policies on different team-wide tasks drives the agents to explore regions of the policy space that yield specializations that generalize across tasks. Simulation results on multiple variations of a remote habitat problem highlight the strength of AIM in discovering robust synergies that allow agents to operate near-optimally in response to the changing team composition and policies of other agents.more » « less
-
Diversity in behaviors is instrumental for robust team performance in many multiagent tasks which require agents to coordinate. Unfortunately, exhaustive search through the agents’ behavior spaces is often intractable. This paper introduces Behavior Exploration for Heterogeneous Teams (BEHT), a multi-level learning framework that enables agents to progressively explore regions of the behavior space that promote team coordination on diverse goals. By combining diversity search to maximize agent-specific rewards and evolutionary optimization to maximize the team-based fitness, our method effectively filters regions of the behavior space that are conducive to agent coordination. We demonstrate the diverse behaviors and synergies that are method allows agents to learn on a multiagent exploration problem.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
