skip to main content

Title: A modeling framework for adaptive lifelong learning with transfer and savings through gating in the prefrontal cortex

The prefrontal cortex encodes and stores numerous, often disparate, schemas and flexibly switches between them. Recent research on artificial neural networks trained by reinforcement learning has made it possible to model fundamental processes underlying schema encoding and storage. Yet how the brain is able to create new schemas while preserving and utilizing old schemas remains unclear. Here we propose a simple neural network framework that incorporates hierarchical gating to model the prefrontal cortex’s ability to flexibly encode and use multiple disparate schemas. We show how gating naturally leads to transfer learning and robust memory savings. We then show how neuropsychological impairments observed in patients with prefrontal damage are mimicked by lesions of our network. Our architecture, which we call DynaMoE, provides a fundamental framework for how the prefrontal cortex may handle the abundance of schemas necessary to navigate the real world.

Authors:
; ; ;
Publication Date:
NSF-PAR ID:
10200700
Journal Name:
Proceedings of the National Academy of Sciences
Volume:
117
Issue:
47
Page Range or eLocation-ID:
p. 29872-29882
ISSN:
0027-8424
Publisher:
Proceedings of the National Academy of Sciences
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus–action associations through rewards; (2) predictive models learning stimulus- and/or action–outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.

  2. Abstract

    Mental models provide a cognitive framework allowing for spatially organizing information while reasoning about the world. However, transitive reasoning studies often rely on perception of stimuli that contain visible spatial features, allowing the possibility that associated neural representations are specific to inherently spatial content. Here, we test the hypothesis that neural representations of mental models generated through transitive reasoning rely on a frontoparietal network irrespective of the spatial nature of the stimulus content. Content within three models ranges from expressly visuospatial to abstract. All mental models participants generated were based on inferred relationships never directly observed. Here, using multivariate representational similarity analysis, we show that patterns representative of mental models were revealed in both superior parietal lobule and anterior prefrontal cortex and converged across stimulus types. These results support the conclusion that, independent of content, transitive reasoning using mental models relies on neural mechanisms associated with spatial cognition.

  3. Abstract

    The neural mechanisms underlying conscious recognition remain unclear, particularly the roles played by the prefrontal cortex, deactivated brain areas and subcortical regions. We investigated neural activity during conscious object recognition using 7 Tesla fMRI while human participants viewed object images presented at liminal contrasts. Here, we show both recognized and unrecognized images recruit widely distributed cortical and subcortical regions; however, recognized images elicit enhanced activation of visual, frontoparietal, and subcortical networks and stronger deactivation of the default-mode network. For recognized images, object category information can be decoded from all of the involved cortical networks but not from subcortical regions. Phase-scrambled images trigger strong involvement of inferior frontal junction, anterior cingulate cortex and default-mode network, implicating these regions in inferential processing under increased uncertainty. Our results indicate that content-specific activity in both activated and deactivated cortical networks and non-content-specific subcortical activity support conscious recognition.

  4. Do dopaminergic reward structures represent the expected utility of information similarly to a reward? Optimal experimental design models from Bayesian decision theory and statistics have proposed a theoretical framework for quantifying the expected value of information that might result from a query. In particular, this formulation quantifies the value of information before the answer to that query is known, in situations where payoffs are unknown and the goal is purely epistemic: That is, to increase knowledge about the state of the world. Whether and how such a theoretical quantity is represented in the brain is unknown. Here we use an event-related functional MRI (fMRI) task design to disentangle information expectation, information revelation and categorization outcome anticipation, and response-contingent reward processing in a visual probabilistic categorization task. We identify a neural signature corresponding to the expectation of information, involving the left lateral ventral striatum. Moreover, we show a temporal dissociation in the activation of different reward-related regions, including the nucleus accumbens, medial prefrontal cortex, and orbitofrontal cortex, during information expectation versus reward-related processing.

  5. Emotions usually occur in a social context; yet little is known about how similar and dissimilar others influence our emotions. In the current study, we examined whether ingroup and outgroup members have differential influence on emotion processing at the behavioral and neural levels. To this end, we recruited 45 participants to rate a series of images displaying people engaged in different emotional contexts. Participants then underwent an fMRI scan where they viewed the same images along with information on how ingroup and outgroup members rated them, and they were asked to rate the images again. We found that participants shifted their emotions to be more in alignment with the ingroup over the outgroup, and that neural regions implicated in positive valuation [ventral striatum (VS) and ventromedial prefrontal cortex (vmPFC)], mentalizing [dorsomedial prefrontal cortex (dmPFC), medial prefrontal cortex (mPFC), posterior superior temporal sulcus (pSTS), and temporal pole], as well as emotion processing and salience detection (amygdala and insula), linearly tracked this behavior such that the extent of neural activity in these regions paralleled changes in participants’ emotions. Results illustrate the powerful impact that ingroup members have on our emotions.