More Like this

1. Reinforcement learning beyond the Bellman equation: Exploring critic objectives using evolution

   https://doi.org/10.1162/isal_a_00338  

   Leite, Abe ; Candadai, Madhavun ; Izquierdo, Eduardo J. ( January 2020 , ALIFE 2020: The 2020 Conference on Artificial Life)

   Living organisms learn on multiple time scales: evolutionary as well as individual-lifetime learning. These two learning modes are complementary: the innate phenotypes developed through evolution significantly influence lifetime learning. However, it is still unclear how these two learning methods interact and whether there is a benefit to part of the system being optimized on a different time scale using a population-based approach while the rest of it is trained on a different time-scale using an individualistic learning algorithm. In this work, we study the benefits of such a hybrid approach using an actor-critic framework where the critic part of an agent is optimized over evolutionary time based on its ability to train the actor part of an agent during its lifetime. Typically, critics are optimized on the same time-scale as the actor using the Bellman equation to represent long-term expected reward. We show that evolution can find a variety of different solutions that can still enable an actor to learn to perform a behavior during its lifetime. We also show that although the solutions found by evolution represent different functions, they all provide similar training signals during the lifetime. This suggests that learning on multiple time-scales can effectively simplify themore »overall optimization process in the actor-critic framework by finding one of many solutions that can still train an actor just as well. Furthermore, analysis of the evolved critics can yield additional possibilities for reinforcement learning beyond the Bellman equation.« less
2. Artificial selection methods from evolutionary computing show promise for directed evolution of microbes

   https://doi.org/10.7554/eLife.79665  

   Lalejini, Alexander ; Dolson, Emily ; Vostinar, Anya E ; Zaman, Luis ( August 2022 , eLife)

   Humans have long known how to co-opt evolutionary processes for their own benefit. Carefully choosing which individuals to breed so that beneficial traits would take hold, they have domesticated dogs, wheat, cows and many other species to fulfil their needs. Biologists have recently refined these ‘artificial selection’ approaches to focus on microorganisms. The hope is to obtain microbes equipped with desirable features, such as the ability to degrade plastic or to produce valuable molecules. However, existing ways of using artificial selection on microbes are limited and sometimes not effective. Computer scientists have also harnessed evolutionary principles for their own purposes, developing highly effective artificial selection protocols that are used to find solutions to challenging computational problems. Yet because of limited communication between the two fields, sophisticated selection protocols honed over decades in evolutionary computing have yet to be evaluated for use in biological populations. In their work, Lalejini et al. compared popular artificial selection protocols developed for either evolutionary computing or work with microorganisms. Two computing selection methods showed promise for improving directed evolution in the laboratory. Crucially, these selection protocols differed from conventionally used methods by selecting for both diversity and performance, rather than performance alone. These promising approachesmore »are now being tested in the laboratory, with potentially far-reaching benefits for medical, biotech, and agricultural applications. While evolutionary computing owes its origins to our understanding of biological processes, it has much to offer in return to help us harness those same mechanisms. The results by Lalejini et al. help to bridge the gap between computational and biological communities who could both benefit from increased collaboration.« less
3. The developmental transcriptome for Lytechinus variegatus exhibits temporally punctuated gene expression changes

   https://doi.org/https://doi.org/10.1016/j.ydbio.2019.12.002  

   Hogan JD, Keenan JL ( January 2020 , Developmental biology)

   Embryonic development is arguably the most complex process an organism undergoes during its lifetime, and understanding this complexity is best approached with a systems-level perspective. The sea urchin has become a highly valuable model organism for understanding developmental specification, morphogenesis, and evolution. As a non-chordate deuterostome, the sea urchin occupies an important evolutionary niche between protostomes and vertebrates. Lytechinus variegatus (Lv) is an Atlantic species that has been well studied, and which has provided important insights into signal transduction, patterning, and morphogenetic changes during embryonic and larval development. The Pacific species, Strongylocentrotus purpuratus (Sp), is another well-studied sea urchin, particularly for gene regulatory networks (GRNs) and cis-regulatory analyses. A well-annotated genome and transcriptome for Sp are available, but similar resources have not been developed for Lv. Here, we provide an analysis of the Lv transcriptome at 11 timepoints during embryonic and larval development. Temporal analysis suggests that the gene regulatory networks that underlie specification are well-conserved among sea urchin species. We show that the major transitions in variation of embryonic transcription divide the developmental time series into four distinct, temporally sequential phases. Our work shows that sea urchin development occurs via sequential intervals of relatively stable gene expression states thatmore »are punctuated by abrupt transitions.« less
4. Behavioral variation, adaptation, and evolution

   https://doi.org/doi:10.1037/0000011-011  

   Shelton, D. S. ; Martins, E. P. ( January 2017 , APA)

   Individual animals behave differently from each other for myriad interrelated intrinsic and extrinsic reasons, and this behavioral variation is the raw substrate for evolutionary change. Behavioral varia- tion can both enhance and constrain long-term evolution (Foster, 2013), and it provides the basic materials on which natural and sexual selection can act. A rich body of historical experimental and conceptual foundations precedes many of the topics discussed. This classic literature is vast and impor- tant, and we encourage the reader to examine it in detail (e.g., Lehrman, 1953; Lorenz, 1971; Schnei- rla, 1966; Waddington, 1959) because we discuss more recent literature. For example, the study of the mechanisms that underlie behavioral variation has a divisive history, which involves carving out the relative contributions of genes and environment to a particular phenotype. Developmental systems and reaction-norm views challenged the issue of gene or environment by arguing that the interplay between genetic substrates and environmental inputs defined adaptive phenotypes across multiple contexts (Fos- ter, 2013; Gottlieb, 1991a, 1991b; Jablonka & Lamb, 2014). Identifying the interactional relationship between components permits researchers to under- stand how behavior becomes organized (Gottlieb, 1991a, 1991b) and can reveal links between indi- vidual variation and population-level persistence, species diversificationmore »(or stasis), and community dynamics (reviewed in Dingemanse & Wolf, 2013). Similarly, the study of individual differences has a rich history situated in the areas of behavioral genet- ics, sociobiology, behavioral ecology, developmen- tal psychology, personality theory, and studies of learning and cognition. Each area has its own goals, associated techniques, and levels of explanation. The study of behavioral variation during early develop- ment, for instance, has been documented primarily by psychologists studying proximate mechanisms in laboratory animal models, whereas the study of dif- ferent adult morphs using the adaptationist perspec- tive has been dominated by behavioral ecologists examining natural populations (Foster, 1995). A more complete description of individual differences requires an integrative study of the mechanisms (e.g., developmental, physiological) that guide intra- individual flexibility and the associated adaptive fine tuning of behavioral types. It is through this integra- tion that researchers can make predictions about the response of different individual phenotypes, groups, populations, and species to novel situations (e.g., captive and urban environments).« less
5. Social behavior mediates the use of social and personal information in wild jays

   https://doi.org/10.1038/s41598-022-06496-x  

   McCune, Kelsey B. ; Valente, Jonathon J. ; Jablonski, Piotr G. ; Lee, Sang-im ; Ha, Renee R. ( February 2022 , Scientific Reports)

   The factors favoring the evolution of certain cognitive abilities in animals remain unclear. Social learning is a cognitive ability that reduces the cost of acquiring personal information and forms the foundation for cultural behavior. Theory predicts the evolutionary pressures to evolve social learning should be greater in more social species. However, research testing this theory has primarily occurred in captivity, where artificial environments can affect performance and yield conflicting results. We compared the use of social and personal information, and the social learning mechanisms used by wild, asocial California scrub-jays and social Mexican jays. We trained demonstrators to solve one door on a multi-door task, then measured the behavior of naïve conspecifics towards the task. If social learning occurs, observations of demonstrators will change the rate that naïve individuals interact with each door. We found both species socially learned, though personal information had a much greater effect on behavior in the asocial species while social information was more important for the social species. Additionally, both species used social information to avoid, rather than copy, conspecifics. Our findings demonstrate that while complex social group structures may be unnecessary for the evolution of social learning, it does affect the use ofmore »social versus personal information.

   « less