More Like this

1. Heterogeneous Agent Coordination via Adaptive Quality Diversity and Specialization

   Dixit, Gaurav; Koll, Charles; Tumer, Kagan (July 2021, Genetic and evolutionary computation)

   null (Ed.)

   In many real-world multiagent systems, agents must learn diverse tasks and coordinate with other agents. This paper introduces a method to allow heterogeneous agents to specialize and only learn complementary divergent behaviors needed for coordination in a shared environment. We use a hierarchical decomposition of diversity search and fitness optimization to allow agents to speciate and learn diverse temporally extended actions. Within an agent population, diversity in niches is favored. Agents within a niche compete for optimizing the higher level coordination task. Experimental results in a multiagent rover exploration task demonstrate the diversity of acquired agent behavior that promotes coordination. 

   more » « less
2. Learning Inter-Agent Synergies in Asymmetric Multiagent Systems

   Dixit, G; Tumer, K (May 2023, International Foundation for Autonomous Agents and Multiagent Systems)

   Agmon, N; An, B; Ricci, A; Yeoh, W. (Ed.)

   In multiagent systems that require coordination, agents must learn diverse policies that enable them to achieve their individual and team objectives. Multiagent Quality-Diversity methods partially address this problem by filtering the joint space of policies to smaller sub-spaces that make the diversification of agent policies tractable. However, in teams of asymmetric agents (agents with different objectives and capabilities), the search for diversity is primarily driven by the need to find policies that will allow agents to assume complementary roles required to work together in teams. This work introduces Asymmetric Island Model (AIM), a multiagent framework that enables populations of asymmetric agents to learn diverse complementary policies that foster teamwork via dynamic population size allocation on a wide variety of team tasks. The key insight of AIM is that the competitive pressure arising from the distribution of policies on different team-wide tasks drives the agents to explore regions of the policy space that yield specializations that generalize across tasks. Simulation results on multiple variations of a remote habitat problem highlight the strength of AIM in discovering robust synergies that allow agents to operate near-optimally in response to the changing team composition and policies of other agents. 

   more » « less
3. Behavior Exploration and Team Balancing for Heterogeneous Multiagent Coordination (Extended Abstract)

   Gaurav Dixit; Kagan Tumer (January 2022, International Conference on Autonomous Agents and MultiAgent Systems)

   Diversity in behaviors is instrumental for robust team performance in many multiagent tasks which require agents to coordinate. Unfortunately, exhaustive search through the agents’ behavior spaces is often intractable. This paper introduces Behavior Exploration for Heterogeneous Teams (BEHT), a multi-level learning framework that enables agents to progressively explore regions of the behavior space that promote team coordination on diverse goals. By combining diversity search to maximize agent-specific rewards and evolutionary optimization to maximize the team-based fitness, our method effectively filters regions of the behavior space that are conducive to agent coordination. We demonstrate the diverse behaviors and synergies that are method allows agents to learn on a multiagent exploration problem. 

   more » « less
4. Robust Multiagent Reinforcement Learning for UAV Systems: Countering Byzantine Attacks

   https://doi.org/10.3390/info14110623  

   Medhi, Jishu K; Liu, Rui; Wang, Qianlong; Chen, Xuhui (November 2023, Information)

   Multiple unmanned aerial vehicle (multi-UAV) systems have gained significant attention in applications, such as aerial surveillance and search and rescue missions. With the recent development of state-of-the-art multiagent reinforcement learning (MARL) algorithms, it is possible to train multi-UAV systems in collaborative and competitive environments. However, the inherent vulnerabilities of multiagent systems pose significant privacy and security risks when deploying general and conventional MARL algorithms. The presence of even a single Byzantine adversary within the system can severely degrade the learning performance of UAV agents. This work proposes a Byzantine-resilient MARL algorithm that leverages a combination of geometric median consensus and a robust state update model to mitigate, or even eliminate, the influence of Byzantine attacks. To validate its effectiveness and feasibility, the authors include a multi-UAV threat model, provide a guarantee of robustness, and investigate key attack parameters for multiple UAV navigation scenarios. Results from the experiments show that the average rewards during a Byzantine attack increased by up to 60% for the cooperative navigation scenario compared with conventional MARL techniques. The learning rewards generated by the baseline algorithms could not converge during training under these attacks, while the proposed method effectively converged to an optimal solution, proving its viability and correctness. 

   more » « less
5. Anisotropic flocking control of distributed multi-agent systems using fluid abstraction

   Silic, M.; Song, Z.; Mohseni, K. (January 2018, Proceedings of the AIAA Information Systems-AIAA Infotech @ Aerospace)

   This paper presents a multi-agent flocking scheme for real-time control of homogeneous unmanned aerial vehicles (UAVs) based on smoothed particle hydrodynamics. Swarm cohe- sion, collision avoidance, and velocity consensus are concurrently satisfied by characterizing the emerging macroscopic flock as a continuous fluid. Two vital implementation issues are addressed in particular including latency in information fusion and directionality of com- munication due to antenna patterns. Symmetric control forces are achieved by meticulous scheduling of inter-vehicle communication to sustain the motion stability of the flock. A gener- alized, anisotropic smoothing kernel that takes into account the relative position and attitude between agents is adopted to address potential flocking instability introduced by communi- cation anisotropy due to the antenna radiation pattern. The feasibility of the technique is demonstrated experimentally using a single UAV avoiding a virtual obstacle. 

   more » « less