Animals flexibly select actions that maximize future rewards despite facing uncertainty in sen-
sory inputs, action-outcome associations or contexts. The computational and circuit mechanisms
underlying this ability are poorly understood.
A clue to such computations can be found in the neural systems involved in representing sensory
features, sensorimotor-outcome associations and contexts. Specifically, the basal ganglia (BG) have
been implicated in forming sensorimotor-outcome association [1] while the thalamocortical loop
between the prefrontal cortex (PFC) and mediodorsal thalamus (MD) has been shown to engage in
contextual representations [2, 3]. Interestingly, both human and non-human animal experiments
indicate that the MD represents different forms of uncertainty [3, 4]. However, finding evidence
for uncertainty representation gives little insight into how it is utilized to drive behavior.
Normative theories have excelled at providing such computational insights. For example, de-
ploying traditional machine learning algorithms to fit human decision-making behavior has clarified
how associative uncertainty alters exploratory behavior [5, 6]. However, despite their computa-
tional insight and ability to fit behaviors, normative models cannot be directly related to neural
mechanisms. Therefore, a critical gap exists between what we know about the neural representa-
tion of uncertainty on one end and the computational functions uncertainty serves in cognition.
This gap can be filled with mechanistic neural models that can approximate normative models as
well as generate experimentally observed neural representations.
In this work, we build a mechanistic cortico-thalamo-BG loop network model that directly
fills this gap. The model includes computationally-relevant mechanistic details of both BG and
thalamocortical circuits such as distributional activities of dopamine [7] and thalamocortical pro-
jection modulating cortical effective connectivity [3] and plasticity [8] via interneurons. We show
that our network can more efficiently and flexibly explore various environments compared to com-
monly used machine learning algorithms and we show that the mechanistic features we include
are crucial for handling different types of uncertainty in decision-making. Furthermore, through
derivation and mathematical proofs, we approximate our models to two novel normative theories.
We show mathematically the first has near-optimal performance on bandit tasks. The second is
a generalization on the well-known CUMSUM algorithm, which is known to be optimal on single
change point detection tasks [9]. Our normative model expands on this by detecting multiple
sequential contextual changes. To our knowledge, our work is the first to link computational in-
sights, normative models and neural realization together in decision-making under various forms
of uncertainty.
more »
« less
Dissociable forms of uncertainty-driven representational change across the human brain
Environmental change can lead decision makers to shift rapidly among different behavioral regimes. These behavioral shifts can be accompanied by rapid changes in the firing pattern of neural networks. However, it is unknown what the populations of neurons that participate in such “network reset” phenomena are representing. Here, we investigated the following: (1) whether and where rapid changes in multivariate activity patterns are observable with fMRI during periods of rapid behavioral change and (2) what types of representations give rise to these phenomena. We did so by examining fluctuations in multivoxel patterns of BOLD activity from male and female human subjects making sequential inferences about the state of a partially observable and discontinuously changing variable. We found that, within the context of this sequential inference task, the multivariate patterns of activity in a number of cortical regions contain representations that change more rapidly during periods of uncertainty following a change in behavioral context. In motor cortex, this phenomenon was indicative of discontinuous change in behavioral outputs, whereas in visual regions, the same basic phenomenon was evoked by tracking of salient environmental changes. In most other cortical regions, including dorsolateral prefrontal and anterior cingulate cortex, the phenomenon was most consistent with directly encoding the degree of uncertainty. However, in a few other regions, including orbitofrontal cortex, the phenomenon was best explained by representations of a shifting context that evolve more rapidly during periods of rapid learning. These representations may provide a dynamic substrate for learning that facilitates rapid disengagement from learned responses during periods of change.
more »
« less
- Award ID(s):
- 1755757
- NSF-PAR ID:
- 10086992
- Date Published:
- Journal Name:
- The Journal of neuroscience
- Volume:
- 39
- ISSN:
- 0270-6474
- Page Range / eLocation ID:
- 1688-1698
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract Modulation of vocal pitch is a key speech feature that conveys important linguistic and affective information. Auditory feedback is used to monitor and maintain pitch. We examined induced neural high gamma power (HGP) (65–150 Hz) using magnetoencephalography during pitch feedback control. Participants phonated into a microphone while hearing their auditory feedback through headphones. During each phonation, a single real‐time 400 ms pitch shift was applied to the auditory feedback. Participants compensated by rapidly changing their pitch to oppose the pitch shifts. This behavioral change required coordination of the neural speech motor control network, including integration of auditory and somatosensory feedback to initiate change in motor plans. We found increases in HGP across both hemispheres within 200 ms of pitch shifts, covering left sensory and right premotor, parietal, temporal, and frontal regions, involved in sensory detection and processing of the pitch shift. Later responses to pitch shifts (200–300 ms) were right dominant, in parietal, frontal, and temporal regions. Timing of activity in these regions indicates their role in coordinating motor change and detecting and processing of the sensory consequences of this change. Subtracting out cortical responses during passive listening to recordings of the phonations isolated HGP increases specific to speech production, highlighting right parietal and premotor cortex, and left posterior temporal cortex involvement in the motor response. Correlation of HGP with behavioral compensation demonstrated right frontal region involvement in modulating participant's compensatory response. This study highlights the bihemispheric sensorimotor cortical network involvement in auditory feedback‐based control of vocal pitch.
Hum Brain Mapp 37:1474‐1485, 2016 . © 2016 Wiley Periodicals, Inc. -
more » « less
The function of long-term memory is not just to reminisce about the past, but also to make predictions that help us behave appropriately and efficiently in the future. This predictive function of memory provides a new perspective on the classic question from memory research of why we remember some things but not others. If prediction is a key outcome of memory, then the extent to which an item generates a prediction signifies that this information already exists in memory and need not be encoded. We tested this principle using human intracranial EEG as a time-resolved method to quantify prediction in visual cortex during a statistical learning task and link the strength of these predictions to subsequent episodic memory behavior. Epilepsy patients of both sexes viewed rapid streams of scenes, some of which contained regularities that allowed the category of the next scene to be predicted. We verified that statistical learning occurred using neural frequency tagging and measured category prediction with multivariate pattern analysis. Although neural prediction was robust overall, this was driven entirely by predictive items that were subsequently forgotten. Such interference provides a mechanism by which prediction can regulate memory formation to prioritize encoding of information that could help learn new predictive relationships.
SIGNIFICANCE STATEMENT When faced with a new experience, we are rarely at a loss for what to do. Rather, because many aspects of the world are stable over time, we rely on past experiences to generate expectations that guide behavior. Here we show that these expectations during a new experience come at the expense of memory for that experience. From intracranial recordings of visual cortex, we decoded what humans expected to see next in a series of photographs based on patterns of neural activity. Photographs that generated strong neural expectations were more likely to be forgotten in a later behavioral memory test. Prioritizing the storage of experiences that currently lead to weak expectations could help improve these expectations in future encounters. -
more » « less
Abstract Primary sensory cortex has long been believed to play a straightforward role in the initial processing of sensory information. Yet, the superficial layers of cortex overall are sparsely active, even during sensory stimulation; additionally, cortical activity is influenced by other modalities, task context, reward, and behavioral state. Our study demonstrates that reinforcement learning dramatically alters representations among longitudinally imaged neurons in superficial layers of mouse primary somatosensory cortex. Learning an object detection task recruits previously unresponsive neurons, enlarging the neuronal population sensitive to touch and behavioral choice. Cortical responses decrease upon repeated stimulus presentation outside of the behavioral task. Moreover, training improves population encoding of the passage of time, and unexpected deviations in trial timing elicit even stronger responses than touches do. In conclusion, the superficial layers of sensory cortex exhibit a high degree of learning-dependent plasticity and are strongly modulated by non-sensory but behaviorally-relevant features, such as timing and surprise.
-
Interactions across frontal cortex are critical for cognition. Animal studies suggest a role for mediodorsal thalamus (MD) in these interactions, but the computations performed and direct relevance to human decision making are unclear. Here, inspired by animal work, we extended a neural model of an executive frontal-MD network and trained it on a human decision-making task for which neuroimaging data were collected. Using a biologically-plausible learning rule, we found that the model MD thalamus compressed its cortical inputs (dorsolateral prefrontal cortex, dlPFC) underlying stimulus-response representations. Through direct feedback to dlPFC, this thalamic operation efficiently partitioned cortical activity patterns and enhanced task switching across different contingencies. To account for interactions with other frontal regions, we expanded the model to compute higher-order strategy signals outside dlPFC, and found that the MD offered a more efficient route for such signals to switch dlPFC activity patterns. Human fMRI data provided evidence that the MD engaged in feedback to dlPFC, and had a role in routing orbitofrontal cortex inputs when subjects switched behavioral strategy. Collectively, our findings contribute to the emerging evidence for thalamic regulation of frontal interactions in the human brain.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1523/JNEUROSCI.1713-18.2018
