skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Computational models of adaptive behavior and prefrontal cortex
Abstract The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus–action associations through rewards; (2) predictive models learning stimulus- and/or action–outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.  more » « less
Award ID(s):
1943767
PAR ID:
10286912
Author(s) / Creator(s):
;
Publisher / Repository:
Nature Publishing Group
Date Published:
Journal Name:
Neuropsychopharmacology
Volume:
47
Issue:
1
ISSN:
0893-133X
Page Range / eLocation ID:
p. 58-71
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Decades of research have shown that global brain states such as arousal can be indexed by measuring the properties of the eyes. The spiking responses of neurons throughout the brain have been associated with the pupil, small fixational saccades, and vigor in eye movements, but it has been difficult to isolate how internal states affect the eyes, and vice versa. While recording from populations of neurons in the visual and prefrontal cortex (PFC), we recently identified a latent dimension of neural activity called “slow drift,” which appears to reflect a shift in a global brain state. Here, we asked if slow drift is correlated with the action of the eyes in distinct behavioral tasks. We recorded from visual cortex (V4) while monkeys performed a change detection task, and PFC, while they performed a memory-guided saccade task. In both tasks, slow drift was associated with the size of the pupil and the microsaccade rate, two external indicators of the internal state of the animal. These results show that metrics related to the action of the eyes are associated with a dominant and task-independent mode of neural activity that can be accessed in the population activity of neurons across the cortex. 
    more » « less
  2. Abstract To accurately categorize items, humans learn to selectively attend to the stimulus dimensions that are most relevant to the task. Models of category learning describe how attention changes across trials as labeled stimuli are progressively observed. The Adaptive Attention Representation Model (AARM), for example, provides an account in which categorization decisions are based on the perceptual similarity of a new stimulus to stored exemplars, and dimension-wise attention is updated on every trial in the direction of a feedback-based error gradient. As such, attention modulation as described by AARM requires interactions among processes of orienting, visual perception, memory retrieval, prediction error, and goal maintenance to facilitate learning. The current study explored the neural bases of attention mechanisms using quantitative predictions from AARM to analyze behavioral and fMRI data collected while participants learned novel categories. Generalized linear model analyses revealed patterns of BOLD activation in the parietal cortex (orienting), visual cortex (perception), medial temporal lobe (memory retrieval), basal ganglia (prediction error), and pFC (goal maintenance) that covaried with the magnitude of model-predicted attentional tuning. Results are consistent with AARM's specification of attention modulation as a dynamic property of distributed cognitive systems. 
    more » « less
  3. Research suggests that evaluations of an object can be jointly influenced by (a) the mere co-occurrence of the object with a pleasant or unpleasant stimulus (e.g., mere co-occurrence of object A and negative event B) and (b) the object’s specific relation to the co-occurring stimulus (e.g., object A starts vs. stops negative event B). Three experiments investigated the impact of cognitive load during learning on the effects of stimulus co-occurrence and stimulus relations. Counter to the shared prediction of competing theories suggesting that effects of stimulus relations should be reduced by cognitive load during learning, effects of stimulus relations were greater (rather than smaller) under high-load compared with low-load conditions. Effects of stimulus co-occurrence were not significantly affected by cognitive load. The results are discussed in terms of theories suggesting that cognitive load can influence behavioral outcomes via strategic shifts in resource allocation in response to task-specific affordances. 
    more » « less
  4. Models of human categorization predict the prefrontal cortex (PFC) serves a central role in category learning. The dorsolateral prefrontal cortex (dlPFC) and ventromedial prefrontal cortex (vmPFC) have been implicated in categorization; however, it is unclear whether both are critical for categorization and whether they support unique functions. We administered three categorization tasks to patients with PFC lesions (mean age, 69.6 years; 5 men, 5 women) to examine how the prefrontal subregions contribute to categorization. These included a rule-based (RB) task that was solved via a unidimensional rule, an information integration (II) task that was solved by combining information from two stimulus dimensions, and a deterministic/probabilistic (DP) task with stimulus features that had varying amounts of category-predictive information. Compared with healthy comparison participants, both patient groups had impaired performance. Impairments in the dlPFC patients were largest during the RB task, whereas impairments in the vmPFC patients were largest during the DP task. A hierarchical model was fit to the participants’ data to assess learning deficits in the patient groups. PFC damage was correlated with a regularization term that limited updates to attention after each trial. Our results suggest that the PFC, as a whole, is important for learning to orient attention to relevant stimulus information. The dlPFC may be especially important for rule-based learning, whereas the vmPFC may be important for focusing attention on deterministic (highly diagnostic) features and ignoring less predictive features. These results support overarching functions of the dlPFC in executive functioning and the vmPFC in value-based decision-making. 
    more » « less
  5. null (Ed.)
    Abstract Behavior involves the ongoing interaction between an organism and its environment. One of the prevailing theories of adaptive behavior is that organisms are constantly making predictions about their future environmental stimuli. However, how they acquire that predictive information is still poorly understood. Two complementary mechanisms have been proposed: predictions are generated from an agent’s internal model of the world or predictions are extracted directly from the environmental stimulus. In this work, we demonstrate that predictive information, measured using bivariate mutual information, cannot distinguish between these two kinds of systems. Furthermore, we show that predictive information cannot distinguish between organisms that are adapted to their environments and random dynamical systems exposed to the same environment. To understand the role of predictive information in adaptive behavior, we need to be able to identify where it is generated. To do this, we decompose information transfer across the different components of the organism-environment system and track the flow of information in the system over time. To validate the proposed framework, we examined it on a set of computational models of idealized agent-environment systems. Analysis of the systems revealed three key insights. First, predictive information, when sourced from the environment, can be reflected in any agent irrespective of its ability to perform a task. Second, predictive information, when sourced from the nervous system, requires special dynamics acquired during the process of adapting to the environment. Third, the magnitude of predictive information in a system can be different for the same task if the environmental structure changes. 
    more » « less