Animals flexibly select actions that maximize future rewards despite facing uncertainty in sen-
sory inputs, action-outcome associations or contexts. The computational and circuit mechanisms
underlying this ability are poorly understood.
A clue to such computations can be found in the neural systems involved in representing sensory
features, sensorimotor-outcome associations and contexts. Specifically, the basal ganglia (BG) have
been implicated in forming sensorimotor-outcome association [1] while the thalamocortical loop
between the prefrontal cortex (PFC) and mediodorsal thalamus (MD) has been shown to engage in
contextual representations [2, 3]. Interestingly, both human and non-human animal experiments
indicate that the MD represents different forms of uncertainty [3, 4]. However, finding evidence
for uncertainty representation gives little insight into how it is utilized to drive behavior.
Normative theories have excelled at providing such computational insights. For example, de-
ploying traditional machine learning algorithms to fit human decision-making behavior has clarified
how associative uncertainty alters exploratory behavior [5, 6]. However, despite their computa-
tional insight and ability to fit behaviors, normative models cannot be directly related to neural
mechanisms. Therefore, a critical gap exists between what we know about the neural representa-
tion of uncertainty on one end and the computational functions uncertainty serves in cognition.
This gap can be filled with mechanistic neural models that can approximate normative models as
well as generate experimentally observed neural representations.
In this work, we build a mechanistic cortico-thalamo-BG loop network model that directly
fills this gap. The model includes computationally-relevant mechanistic details of both BG and
thalamocortical circuits such as distributional activities of dopamine [7] and thalamocortical pro-
jection modulating cortical effective connectivity [3] and plasticity [8] via interneurons. We show
that our network can more efficiently and flexibly explore various environments compared to com-
monly used machine learning algorithms and we show that the mechanistic features we include
are crucial for handling different types of uncertainty in decision-making. Furthermore, through
derivation and mathematical proofs, we approximate our models to two novel normative theories.
We show mathematically the first has near-optimal performance on bandit tasks. The second is
a generalization on the well-known CUMSUM algorithm, which is known to be optimal on single
change point detection tasks [9]. Our normative model expands on this by detecting multiple
sequential contextual changes. To our knowledge, our work is the first to link computational in-
sights, normative models and neural realization together in decision-making under various forms
of uncertainty.
more »
« less
A modeling framework for adaptive lifelong learning with transfer and savings through gating in the prefrontal cortex
The prefrontal cortex encodes and stores numerous, often disparate, schemas and flexibly switches between them. Recent research on artificial neural networks trained by reinforcement learning has made it possible to model fundamental processes underlying schema encoding and storage. Yet how the brain is able to create new schemas while preserving and utilizing old schemas remains unclear. Here we propose a simple neural network framework that incorporates hierarchical gating to model the prefrontal cortex’s ability to flexibly encode and use multiple disparate schemas. We show how gating naturally leads to transfer learning and robust memory savings. We then show how neuropsychological impairments observed in patients with prefrontal damage are mimicked by lesions of our network. Our architecture, which we call DynaMoE, provides a fundamental framework for how the prefrontal cortex may handle the abundance of schemas necessary to navigate the real world.
more » « less- NSF-PAR ID:
- 10200700
- Publisher / Repository:
- Proceedings of the National Academy of Sciences
- Date Published:
- Journal Name:
- Proceedings of the National Academy of Sciences
- Volume:
- 117
- Issue:
- 47
- ISSN:
- 0027-8424
- Page Range / eLocation ID:
- p. 29872-29882
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract Disruption of circadian rhythms, such as shift work and jet lag, are associated with negative physiological and behavioral outcomes, including changes in affective state, learning and memory, and cognitive function. The prefrontal cortex (PFC) is heavily involved in all of these processes. Many PFC-associated behaviors are time-of-day dependent, and disruption of daily rhythms negatively impacts these behavioral outputs. Yet how disruption of daily rhythms impacts the fundamental function of PFC neurons, and the mechanism(s) by which this occurs, remains unknown. Using a mouse model, we demonstrate that the activity and action potential dynamics of prelimbic PFC neurons are regulated by time-of-day in a sex specific manner. Further, we show that postsynaptic K+channels play a central role in physiological rhythms, suggesting an intrinsic gating mechanism mediating physiological activity. Finally, we demonstrate that environmental circadian desynchronization alters the intrinsic functioning of these neurons independent of time-of-day. These key discoveries demonstrate that daily rhythms contribute to the mechanisms underlying the essential physiology of PFC circuits and provide potential mechanisms by which circadian disruption may impact the fundamental properties of neurons.
-
more » « less
The behavioral and neural effects of the endogenous release of acetylcholine following stimulation of the Nucleus Basalis of Meynert (NB) have been recently examined in two male monkeys (Qi et al. 2021). Counterintuitively, NB stimulation enhanced behavioral performance while broadening neural tuning in the prefrontal cortex (PFC). The mechanism by which a weaker mnemonic neural code could lead to better performance remains unclear. Here, we show that increased neural excitability in a simple continuous bump attractor model can induce broader neural tuning and decrease bump diffusion, provided neural rates are saturated. Increased memory precision in the model overrides memory accuracy, improving overall task performance. Moreover, we show that bump attractor dynamics can account for the nonuniform impact of neuromodulation on distractibility, depending on distractor distance from the target. Finally, we delve into the conditions under which bump attractor tuning and diffusion balance in biologically plausible heterogeneous network models. In these discrete bump attractor networks, we show that reducing spatial correlations or enhancing excitatory transmission can improve memory precision. Altogether, we provide a mechanistic understanding of how cholinergic neuromodulation controls spatial working memory through perturbed attractor dynamics in PFC.
Significance statement Acetylcholine has been thought to improve cognitive performance by sharpening neuronal tuning in prefrontal cortex. Recent work has shown that electrical stimulation of the cholinergic forebrain in awake-behaving monkeys induces a reduction in prefrontal neural tuning under stimulation conditions that improve performance. To reconcile these divergent observations, we provide network simulations showing that these derive consistently from specific conditions in prefrontal attractor dynamics: firing rate saturation leads to increased storage precision and reduced neural tuning upon cholinergic activation via an increase in neural excitability, a reduction in neural correlations, and an increase in excitatory transmission. Our study integrates previously reported data into a consistent mechanistic view of how acetylcholine controls spatial working memory via attractor network dynamics in prefrontal cortex. -
more » « less
Key points Visual attention involves discrete multispectral oscillatory responses in visual and ‘higher‐order’ prefrontal cortices.
Prefrontal cortex laterality effects during visual selective attention are poorly characterized.
High‐definition transcranial direct current stimulation dynamically modulated right‐lateralized fronto‐visual theta oscillations compared to those observed in left fronto‐visual pathways.
Increased connectivity in right fronto‐visual networks after stimulation of the left dorsolateral prefrontal cortex resulted in faster task performance in the context of distractors.
Our findings show clear laterality effects in theta oscillatory activity along prefrontal–visual cortical pathways during visual selective attention.
Abstract Studies of visual attention have implicated oscillatory activity in the recognition, protection and temporal organization of attended representations in visual cortices. These studies have also shown that higher‐order regions such as the prefrontal cortex are critical to attentional processing, but far less is understood regarding prefrontal laterality differences in attention processing. To examine this, we selectively applied high‐definition transcranial direct current stimulation (HD‐tDCS) to the left or right dorsolateral prefrontal cortex (DLPFC). We predicted that HD‐tDCS of the left
versus right prefrontal cortex would differentially modulate performance on a visual selective attention task, and alter the underlying oscillatory network dynamics. Our randomized crossover design included 27 healthy adults that underwent three separate sessions of HD‐tDCS (sham, left DLPFC and right DLPFC) for 20 min. Following stimulation, participants completed an attention protocol during magnetoencephalography. The resulting oscillatory dynamics were imaged using beamforming, and peak task‐related neural activity was subjected to dynamic functional connectivity analyses to evaluate the impact of stimulation site (i.e. left and right DLPFC) on neural interactions. Our results indicated that HD‐tDCS over the left DLPFC differentially modulated right fronto‐visual functional connectivity within the theta band compared to HD‐tDCS of the right DLPFC and further, specifically modulated the oscillatory response for detecting targets among an array of distractors. Importantly, these findings provide network‐specific insight into the complex oscillatory mechanisms serving visual selective attention. -
more » « less
Abstract The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus–action associations through rewards; (2) predictive models learning stimulus- and/or action–outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior.
