skip to main content

Title: Computational models of adaptive behavior and prefrontal cortex

The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus–action associations through rewards; (2) predictive models learning stimulus- and/or action–outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.

Award ID(s):
Publication Date:
Journal Name:
Page Range or eLocation-ID:
p. 58-71
Nature Publishing Group
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Behavior involves the ongoing interaction between an organism and its environment. One of the prevailing theories of adaptive behavior is that organisms are constantly making predictions about their future environmental stimuli. However, how they acquire that predictive information is still poorly understood. Two complementary mechanisms have been proposed: predictions are generated from an agent’s internal model of the world or predictions are extracted directly from the environmental stimulus. In this work, we demonstrate that predictive information, measured using bivariate mutual information, cannot distinguish between these two kinds of systems. Furthermore, we show that predictive information cannot distinguish between organisms that are adapted to their environments and random dynamical systems exposed to the same environment. To understand the role of predictive information in adaptive behavior, we need to be able to identify where it is generated. To do this, we decompose information transfer across the different components of the organism-environment system and track the flow of information in the system over time. To validate the proposed framework, we examined it on a set of computational models of idealized agent-environment systems. Analysis of the systems revealed three key insights. First, predictive information, when sourced from the environment, can be reflected inmore »any agent irrespective of its ability to perform a task. Second, predictive information, when sourced from the nervous system, requires special dynamics acquired during the process of adapting to the environment. Third, the magnitude of predictive information in a system can be different for the same task if the environmental structure changes.« less
  2. Abstract

    Rapid and flexible learning during behavioral choices is critical to our daily endeavors and constitutes a hallmark of dynamic reasoning. An important paradigm to examine flexible behavior involves learning new arbitrary associations mapping visual inputs to motor outputs. We conjectured that visuomotor rules are instantiated by translating visual signals into actions through dynamic interactions between visual, frontal and motor cortex. We evaluated the neural representation of such visuomotor rules by performing intracranial field potential recordings in epilepsy subjects during a rule-learning delayed match-to-behavior task. Learning new visuomotor mappings led to the emergence of specific responses associating visual signals with motor outputs in 3 anatomical clusters in frontal, anteroventral temporal and posterior parietal cortex. After learning, mapping selective signals during the delay period showed interactions with visual and motor signals. These observations provide initial steps towards elucidating the dynamic circuits underlying flexible behavior and how communication between subregions of frontal, temporal, and parietal cortex leads to rapid learning of task-relevant choices.

  3. Abstract Background How the brain develops accurate models of the external world and generates appropriate behavioral responses is a vital question of widespread multidisciplinary interest. It is increasingly understood that brain signal variability—posited to enhance perception, facilitate flexible cognitive representations, and improve behavioral outcomes—plays an important role in neural and cognitive development. The ability to perceive, interpret, and respond to complex and dynamic social information is particularly critical for the development of adaptive learning and behavior. Social perception relies on oxytocin-regulated neural networks that emerge early in development. Methods We tested the hypothesis that individual differences in the endogenous oxytocinergic system early in life may influence social behavioral outcomes by regulating variability in brain signaling during social perception. In study 1, 55 infants provided a saliva sample at 5 months of age for analysis of individual differences in the oxytocinergic system and underwent electroencephalography (EEG) while listening to human vocalizations at 8 months of age for the assessment of brain signal variability. Infant behavior was assessed via parental report. In study 2, 60 infants provided a saliva sample and underwent EEG while viewing faces and objects and listening to human speech and water sounds at 4 months of age. Infant behavior was assessedmore »via parental report and eye tracking. Results We show in two independent infant samples that increased brain signal entropy during social perception is in part explained by an epigenetic modification to the oxytocin receptor gene ( OXTR ) and accounts for significant individual differences in social behavior in the first year of life. These results are measure-, context-, and modality-specific: entropy, not standard deviation, links OXTR methylation and infant behavior; entropy evoked during social perception specifically explains social behavior only; and only entropy evoked during social auditory perception predicts infant vocalization behavior. Conclusions Demonstrating these associations in infancy is critical for elucidating the neurobiological mechanisms accounting for individual differences in cognition and behavior relevant to neurodevelopmental disorders. Our results suggest that an epigenetic modification to the oxytocin receptor gene and brain signal entropy are useful indicators of social development and may hold potential diagnostic, therapeutic, and prognostic value.« less
  4. Abstract

    Conflicts at various stages of cognition can cause interference effects on behavior. Two well-studied forms of cognitive interference are stimulus–stimulus (e.g., Flanker), where the conflict arises from incongruence between the task-relevant stimulus and simultaneously presented irrelevant stimulus information, and stimulus-response (e.g., Simon), where interference is the result of an incompatibility between the spatial location of the task-relevant stimulus and a prepotent motor mapping of the expected response. Despite substantial interest in the neural and behavioral underpinnings of cognitive interference, it remains uncertain how differing sources of cognitive conflict might interact, and the spectrally specific neural dynamics that index this phenomenon are poorly understood. Herein, we used an adapted version of the multisource interference task and magnetoencephalography to investigate the spectral, temporal, and spatial dynamics of conflict processing in healthy adults (N = 23). We found a double-dissociation such that, in isolation, stimulus–stimulus interference was indexed by alpha (8–14 Hz), but not gamma-frequency (64–76 Hz) oscillations in the lateral occipital regions, while stimulus–response interference was indexed by gamma oscillations in nearby cortices, but not by alpha oscillations. Surprisingly, we also observed a superadditive effect of simultaneously presented interference types (multisource) on task performance and gamma oscillations in superior parietal cortex.

  5. Survival relies on the ability to flexibly choose between different actions according to varying environmental circumstances. Many lines of evidence indicate that action selection involves signaling in corticostriatal circuits, including the orbitofrontal cortex (OFC) and dorsomedial striatum (DMS). While choice-specific responses have been found in individual neurons from both areas, it is unclear whether populations of OFC or DMS neurons are better at encoding an animal’s choice. To address this, we trained head-fixed mice to perform an auditory guided two-alternative choice task, which required moving a joystick forward or backward. We then used silicon microprobes to simultaneously measure the spiking activity of OFC and DMS ensembles, allowing us to directly compare population dynamics between these areas within the same animals. Consistent with previous literature, both areas contained neurons that were selective for specific stimulus-action associations. However, analysis of concurrently recorded ensemble activity revealed that the animal’s trial-by-trial behavior could be decoded more accurately from DMS dynamics. These results reveal substantial regional differences in encoding action selection, suggesting that DMS neural dynamics are more specialized than OFC at representing an animal’s choice of action. NEW & NOTEWORTHY While previous literature shows that both orbitofrontal cortex (OFC) and dorsomedial striatum (DMS) representmore »information relevant to selecting specific actions, few studies have directly compared neural signals between these areas. Here we compared OFC and DMS dynamics in mice performing a two-alternative choice task. We found that the animal’s choice could be decoded more accurately from DMS population activity. This work provides among the first evidence that OFC and DMS differentially represent information about an animal’s selected action.« less