An official website of the United States government
Abstract Identifying the circuits responsible for cognition and understanding their embedded computations is a challenge for neuroscience. We establish here a hierarchical cross-scale approach, from behavioral modeling and fMRI in task-performing mice to cellular recordings, in order to disentangle local network contributions to olfactory reinforcement learning. At mesoscale, fMRI identifies a functional olfactory-striatal network interacting dynamically with higher-order cortices. While primary olfactory cortices respectively contribute only some value components, the downstream olfactory tubercle of the ventral striatum expresses comprehensively reward prediction, its dynamic updating, and prediction error components. In the tubercle, recordings reveal two underlying neuronal populations with non-redundant reward prediction coding schemes. One population collectively produces stabilized predictions as distributed activity across neurons; in the other, neurons encode value individually and dynamically integrate the recent history of uncertain outcomes. These findings validate a cross-scale approach to mechanistic investigations of higher cognitive functions in rodents.
more »
« less
Phasic dopamine reinforces distinct striatal stimulus encoding in the olfactory tubercle driving dopaminergic reward prediction
Abstract The learning of stimulus-outcome associations allows for predictions about the environment. Ventral striatum and dopaminergic midbrain neurons form a larger network for generating reward prediction signals from sensory cues. Yet, the network plasticity mechanisms to generate predictive signals in these distributed circuits have not been entirely clarified. Also, direct evidence of the underlying interregional assembly formation and information transfer is still missing. Here we show that phasic dopamine is sufficient to reinforce the distinctness of stimulus representations in the ventral striatum even in the absence of reward. Upon such reinforcement, striatal stimulus encoding gives rise to interregional assemblies that drive dopaminergic neurons during stimulus-outcome learning. These assemblies dynamically encode the predicted reward value of conditioned stimuli. Together, our data reveal that ventral striatal and midbrain reward networks form a reinforcing loop to generate reward prediction coding.
more »
« less
- Award ID(s):
- 1724221
- PAR ID:
- 10377907
- Date Published:
- Journal Name:
- Nature Communications
- Volume:
- 11
- Issue:
- 1
- ISSN:
- 2041-1723
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The posterior medial (POm) thalamus is heavily interconnected with sensory and motor circuitry and is likely involved in behavioral modulation and sensorimotor integration. POm provides axonal projections to the dorsal striatum, a hotspot of sensorimotor processing, yet the role of POm-striatal projections has remained undetermined. Using optogenetics with mouse brain slice electrophysiology, we found that POm provides robust synaptic input to direct and indirect pathway striatal spiny projection neurons (D1- and D2-SPNs, respectively) and parvalbumin-expressing fast spiking interneurons (PVs). During the performance of a whisker-based tactile discrimination task in head-restrained mice, POm-striatal projections displayed learning-related activation correlating with anticipatory, but not reward-related, pupil dilation. Inhibition of POm-striatal axons across learning caused slower reaction times and an increase in the number of training sessions for expert performance. Our data indicate that POm-striatal inputs provide a behaviorally relevant arousal-related signal, which may prime striatal circuitry for efficient integration of subsequent choice-related inputs.more » « less
-
Cai, Ming Bo (Ed.)A major advance in understanding learning behavior stems from experiments showing that reward learning requires dopamine inputs to striatal neurons and arises from synaptic plasticity of cortico-striatal synapses. Numerous reinforcement learning models mimic this dopamine-dependent synaptic plasticity by using the reward prediction error, which resembles dopamine neuron firing, to learn the best action in response to a set of cues. Though these models can explain many facets of behavior, reproducing some types of goal-directed behavior, such as renewal and reversal, require additional model components. Here we present a reinforcement learning model, TD2Q, which better corresponds to the basal ganglia with two Q matrices, one representing direct pathway neurons (G) and another representing indirect pathway neurons (N). Unlike previous two-Q architectures, a novel and critical aspect of TD2Q is to update the G and N matrices utilizing the temporal difference reward prediction error. A best action is selected for N and G using a softmax with a reward-dependent adaptive exploration parameter, and then differences are resolved using a second selection step applied to the two action probabilities. The model is tested on a range of multi-step tasks including extinction, renewal, discrimination; switching reward probability learning; and sequence learning. Simulations show that TD2Q produces behaviors similar to rodents in choice and sequence learning tasks, and that use of the temporal difference reward prediction error is required to learn multi-step tasks. Blocking the update rule on the N matrix blocks discrimination learning, as observed experimentally. Performance in the sequence learning task is dramatically improved with two matrices. These results suggest that including additional aspects of basal ganglia physiology can improve the performance of reinforcement learning models, better reproduce animal behaviors, and provide insight as to the role of direct- and indirect-pathway striatal neurons.more » « less
-
Summary The disproportionate expansion of telencephalic structures during human evolution involved tradeoffs that imposed greater connectivity and metabolic demands on midbrain dopaminergic neurons. Despite the central role of dopaminergic neurons in human-enriched disorders, molecular specializations associated with human-specific features and vulnerabilities of the dopaminergic system remain unexplored. Here, we establish a phylogeny-in-a-dish approach to examine gene regulatory evolution by differentiating pools of human, chimpanzee, orangutan, and macaque pluripotent stem cells into ventral midbrain organoids capable of forming long-range projections, spontaneous activity, and dopamine release. We identify human-specific gene expression changes related to axonal transport of mitochondria and reactive oxygen species buffering and candidatecis-andtrans-regulatory mechanisms underlying gene expression divergence. Our findings are consistent with a model of evolved neuroprotection in response to tradeoffs related to brain expansion and could contribute to the discovery of therapeutic targets and strategies for treating disorders involving the dopaminergic system.more » « less
-
There is an ongoing debate on the contribution of the neuronal glutamate transporter EAAC1 to the onset of compulsive behaviors. Here, we used behavioral, electrophysiological, molecular, and viral approaches in male and female mice to identify the molecular and cellular mechanisms by which EAAC1 controls the execution of repeated motor behaviors. Our findings show that, in the striatum, a brain region implicated with movement execution, EAAC1 limits group I metabotropic glutamate receptor (mGluRI) activation, facilitates D1 dopamine receptor (D1R) expression, and ensures long-term synaptic plasticity. Blocking mGluRI in slices from mice lacking EAAC1 restores D1R expression and synaptic plasticity. Conversely, activation of intracellular signaling pathways coupled to mGluRI in D1R-containing striatal neurons of mice expressing EAAC1 leads to reduced D1R protein level and increased stereotyped movement execution. These findings identify new molecular mechanisms by which EAAC1 can shape glutamatergic and dopaminergic signals and control repeated movement execution.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1038/s41467-020-17257-7
