An official website of the United States government
Diversity in behaviors is instrumental for robust team performance in many multiagent tasks which require agents to coordinate. Unfortunately, exhaustive search through the agents’ behavior spaces is often intractable. This paper introduces Behavior Exploration for Heterogeneous Teams (BEHT), a multi-level learning framework that enables agents to progressively explore regions of the behavior space that promote team coordination on diverse goals. By combining diversity search to maximize agent-specific rewards and evolutionary optimization to maximize the team-based fitness, our method effectively filters regions of the behavior space that are conducive to agent coordination. We demonstrate the diverse behaviors and synergies that are method allows agents to learn on a multiagent exploration problem.
more »
« less
Adaptive Multi-Fitness Learning for Robust Coordination
Long term robotic deployments are well described by sparse fitness functions, which are hard to learn from and adapt to. This work introduces Adaptive Multi-Fitness Learning (A-MFL), which augments the structure of Multi-Fitness Learning (MFL) [9] by injecting new behaviors into the agents as the environment changes. A-MFL not only improves system performance in dynamic environments, but also avoids undesirable, unforeseen side-effects of new behaviors by localizing where the new behaviors are used. On a simulated multi-robot problem, A-MFL provides up to 90% improvement over MFL, and 100% over a one-step evolutionary approach.
more »
« less
- Award ID(s):
- 1815886
- PAR ID:
- 10294971
- Date Published:
- Journal Name:
- Genetic and evolutionary computation
- ISSN:
- 1932-0175
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Evolutionary biologists typically envision a trait’s genetic basis and fitness effects occurring within a single species. However, traits can be determined by and have fitness consequences for interacting species, thus evolving in multiple genomes. This is especially likely in mutualisms, where species exchange fitness benefits and can associate over long periods of time. Partners may experience evolutionary conflict over the value of a multi-genomic trait, but such conflicts may be ameliorated by mutualism’s positive fitness feedbacks. Here, we develop a simulation model of a host–microbe mutualism to explore the evolution of a multi-genomic trait. Coevolutionary outcomes depend on whether hosts and microbes have similar or different optimal trait values, strengths of selection and fitness feedbacks. We show that genome-wide association studies can map joint traits to loci in multiple genomes and describe how fitness conflict and fitness feedback generate different multi-genomic architectures with distinct signals around segregating loci. Partner fitnesses can be positively correlated even when partners are in conflict over the value of a multi-genomic trait, and conflict can generate strong mutualistic dependency. While fitness alignment facilitates rapid adaptation to a new optimum, conflict maintains genetic variation and evolvability, with implications for applied microbiome science.more » « less
-
The goal of molecular crystal structure prediction (CSP) is to find all the plausible polymorphs for a given molecule. This requires performing global optimization over a high-dimensional search space. Genetic algorithms (GAs) perform global optimization by starting from an initial population of structures and generating new candidate structures by breeding the fittest structures in the population. Typically, the fitness function is based on relative lattice energies, such that structures with lower energies have a higher probability of being selected for mating. GAs may be adapted to perform multi-modal optimization by using evolutionary niching methods that support the formation of several stable subpopulations and suppress the over-sampling of densely populated regions. Evolutionary niching is implemented in the GAtor molecular crystal structure prediction code by using techniques from machine learning to dynamically cluster the population into niches of structural similarity. A cluster-based fitness function is constructed such that structures in less populated clusters have a higher probability of being selected for breeding. Here, the effects of evolutionary niching are investigated for the crystal structure prediction of 1,3-dibromo-2-chloro-5-fluorobenzene. Using the cluster-based fitness function increases the success rate of generating the experimental structure and additional low-energy structures with similar packing motifs.more » « less
-
Silva, S; Paquete, L (Ed.)Coevolving teams of agents promises effective solutions for many coordination tasks such as search and rescue missions or deep ocean exploration. Good team performance in such domains generally relies on agents discovering complex joint policies, which is particularly difficult when the fitness functions are sparse (where many joint policies return the same or even zero fitness values). In this paper, we introduce Novelty Seeking Multiagent Evolutionary Reinforcement Learning (NS-MERL), which enables agents to more efficiently explore their joint strategy space. The key insight of NS-MERL is to promote good exploratory behaviors for individual agents using a dense, novelty-based fitness function. Though the overall team-level performance is still evaluated via a sparse fitness function, agents using NS-MERL more efficiently explore their joint action space and more readily discover good joint policies. Our results in complex coordination tasks show that teams of agents trained with NS-MERL perform significantly better than agents trained solely with task-specific fitnesses.more » « less
-
As a population grows, spreading to new environments may favor specialization. In this paper, we introduce and explore a model for specialization at the front of a colony expanding synchronously into new territory. We show through numerical simulations that, by gaining fitness through accumulating mutations, progeny of the initial seed population can differentiate into distinct specialists. With competition and selection limited to the growth front, the emerging specialists first segregate into sectors, which then expand to dominate the entire population. We quantify the scaling of the fixation time with the size of the population and observe different behaviors corresponding to distinct universality classes: unbounded and bounded gains in fitness lead to superdiffusive (z = 3/2) and diffusive (z = 2) stochastic wanderings of the sector boundaries, respectively.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
