Cooperative Co-evolutionary Algorithms effectively train policies in multiagent systems with a single, statically defined team. However, many real-world problems, such as search and rescue, require agents to operate in multiple teams. When the structure of the team changes, these policies show reduced performance as they were trained to cooperate with only one team. In this work, we solve the cooperation problem by training agents to fill the needs of an arbitrary team, thereby gaining the ability to support a large variety of teams. We introduce Ad hoc Teaming Through Evolution (ATTE) which evolves a limited number of policy types using fitness aggregation across multiple teams. ATTE leverages agent types to reduce the dimensionality of the interaction search space, while fitness aggregation across teams selects for more adaptive policies. In a simulated multi-robot exploration task, ATTE is able to learn policies that are effective in a variety of teaming schemes, improving the performance of CCEA by a factor of up to five times.
Adaptive Agent Architecture for Real-time Human-Agent Teaming
Teamwork is a set of interrelated reasoning, actions and behaviors
of team members that facilitate common objectives.
Teamwork theory and experiments have resulted in a set of
states and processes for team effectiveness in both human-human
and agent-agent teams. However, human-agent teaming
is less well studied because it is so new and involves
asymmetry in policy and intent not present in human teams.
To optimize team performance in human-agent teaming, it
is critical that agents infer human intent and adapt their polices
for smooth coordination. Most literature in human-agent
teaming builds agents referencing a learned human model.
Though these agents are guaranteed to perform well with the
learned model, they lay heavy assumptions on human policy
such as optimality and consistency, which is unlikely in many
real-world scenarios. In this paper, we propose a novel adaptive
agent architecture in human-model-free setting on a two-player
cooperative game, namely Team Space Fortress (TSF).
Previous human-human team research have shown complementary
policies in TSF game and diversity in human players’
skill, which encourages us to relax the assumptions on
human policy. Therefore, we discard learning human models
from human data, and instead use an adaptation strategy on
a pre-trained library of exemplar policies composed of RL
algorithms or rule-based methods with minimal assumptions
of human behavior. The adaptation strategy relies on a novel
similarity metric to infer human more »
- Award ID(s):
- 1950811
- Publication Date:
- NSF-PAR ID:
- 10338198
- Journal Name:
- AAAI Workshop on Plan, Activity, and Intent Recognition
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Shared autonomy provides an effective framework for human-robot collaboration that takes advantage of the complementary strengths of humans and robots to achieve common goals. Many existing approaches to shared autonomy make restrictive assumptions that the goal space, environment dynamics, or human policy are known a priori, or are limited to discrete action spaces, preventing those methods from scaling to complicated real world environments. We propose a model-free, residual policy learning algorithm for shared autonomy that alleviates the need for these assumptions. Our agents are trained to minimally adjust the human’s actions such that a set of goal-agnostic constraints are satisfied. We test our method in two continuous control environments: Lunar Lander, a 2D flight control domain, and a 6-DOF quadrotor reaching task. In experiments with human and surrogate pilots, our method significantly improves task performance without any knowledge of the human’s goal beyond the constraints. These results highlight the ability of model-free deep reinforcement learning to realize assistive agents suited to continuous control settings with little knowledge of user intent.
-
Several recent works have found the emergence of grounded com-positional language in the communication protocols developed bymostly cooperative multi-agent systems when learned end-to-endto maximize performance on a downstream task. However, humanpopulations learn to solve complex tasks involving communicativebehaviors not only in fully cooperative settings but also in scenar-ios where competition acts as an additional external pressure forimprovement. In this work, we investigate whether competitionfor performance from an external, similar agent team could actas a social influence that encourages multi-agent populations todevelop better communication protocols for improved performance,compositionality, and convergence speed. We start fromTask &Talk, a previously proposed referential game between two coopera-tive agents as our testbed and extend it intoTask, Talk & Compete,a game involving two competitive teams each consisting of twoaforementioned cooperative agents. Using this new setting, we pro-vide an empirical study demonstrating the impact of competitiveinfluence on multi-agent teams. Our results show that an externalcompetitive influence leads to improved accuracy and generaliza-tion, as well as faster emergence of communicative languages thatare more informative and compositional.
-
Objective This work examines two human–autonomy team (HAT) training approaches that target communication and trust calibration to improve team effectiveness under degraded conditions. Background Human–autonomy teaming presents challenges to teamwork, some of which may be addressed through training. Factors vital to HAT performance include communication and calibrated trust. Method Thirty teams of three, including one confederate acting as an autonomous agent, received either entrainment-based coordination training, trust calibration training, or control training before executing a series of missions operating a simulated remotely piloted aircraft. Automation and autonomy failures simulating degraded conditions were injected during missions, and measures of team communication, trust, and task efficiency were collected. Results Teams receiving coordination training had higher communication anticipation ratios, took photos of targets faster, and overcame more autonomy failures. Although autonomy failures were introduced in all conditions, teams receiving the calibration training reported that their overall trust in the agent was more robust over time. However, they did not perform better than the control condition. Conclusions Training based on entrainment of communications, wherein introduction of timely information exchange through one team member has lasting effects throughout the team, was positively associated with improvements in HAT communications and performance under degraded conditions. Training thatmore »
-
This article deals with household-level flood risk mitigation. We present an agent-based modeling framework to simulate the mechanism of natural hazard and human interactions, to allow evaluation of community flood risk, and to predict various adaptation outcomes. The framework considers each household as an autonomous, yet socially connected, agent. A Beta-Bernoulli Bayesian learning model is first applied to measure changes of agents' risk perceptions in response to stochastic storm surges. Then the risk appraisal behaviors of agents, as a function of willingness-to-pay for flood insurance, are measured. Using Miami-Dade County, Florida as a case study, we simulated four scenarios to evaluate the outcomes of alternative adaptation strategies. Results show that community damage decreases significantly after a few years when agents become cognizant of flood risks. Compared to insurance policies with pre-Flood Insurance Rate Maps subsidies, risk-based insurance policies are more effective in promoting community resilience, but it will decrease motivations to purchase flood insurance, especially for households outside of high-risk areas. We evaluated vital model parameters using a local sensitivity analysis. Simulation results demonstrate the importance of an integrated adaptation strategy in community flood risk management.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Publisher's Version of Record
- The DOI is not currently available
