skip to main content

Title: Thalamocortical contribution to flexible learning in neural systems. Network Neuroscience
Animal brains evolved to optimize behavior in dynamic environments, flexibly selecting actions that maximize future rewards in different contexts. A large body of experimental work indicates that such optimization changes the wiring of neural circuits, appropriately mapping environmental input onto behavioral outputs. A major unsolved scientific question is how optimal wiring adjustments, which must target the connections responsible for rewards, can be accomplished when the relation between sensory inputs, action taken, environmental context with rewards is ambiguous. The credit assignment problem can be categorized into context-independent structural credit assignment and context-dependent continual learning. In this perspective, we survey prior approaches to these two problems and advance the notion that the brain’s specialized neural architectures provide efficient solutions. Within this framework, the thalamus with its cortical and basal ganglia interactions serves as a systems-level solution to credit assignment. Specifically, we propose that thalamocortical interaction is the locus of meta-learning where the thalamus provides cortical control functions that parametrize the cortical activity association space. By selecting among these control functions, the basal ganglia hierarchically guide thalamocortical plasticity across two timescales to enable meta-learning. The faster timescale establishes contextual associations to enable behavioral flexibility while the slower one enables generalization to new contexts.  more » « less
Award ID(s):
2139936 2003830 1810758
Author(s) / Creator(s):
Date Published:
Journal Name:
Network neuroscience
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Vocal learning in songbirds is mediated by a highly localized system of interconnected forebrain regions, including recurrent loops that traverse the cortex, basal ganglia, and thalamus. This brain-behavior system provides a powerful model for elucidating mechanisms of vocal learning, with implications for learning speech in human infants, as well as for advancing our understanding of skill learning in general. A long history of experiments in this area has tested neural responses to playback of different song stimuli in anesthetized birds at different stages of vocal development. These studies have demonstrated selectivity for different song types that provide neural signatures of learning. In contrast to the ease of obtaining responses to song playback in anesthetized birds, song-evoked responses in awake birds are greatly reduced or absent, indicating that behavioral state is an important determinant of neural responsivity. Song-evoked responses can be elicited during sleep as well as anesthesia, and the selectivity of responses to song playback in adult birds is highly similar between anesthetized and sleeping states, encouraging the idea that anesthesia and sleep are similar. In contrast to that idea, we report evidence that cortical responses to song playback in juvenile zebra finches ( Taeniopygia guttata ) differ greatly between sleep and urethane anesthesia. This finding indicates that behavioral states differ in sleep versus anesthesia and raises questions about relationships between developmental changes in sleep activity, selectivity for different song types, and the neural substrate for vocal learning. 
    more » « less
  2. Abstract

    The thalamostriatal system is a major network in the mammalian brain, originating principally from the intralaminar nuclei of thalamus. Its functions remain unclear, but a subset of these projections provides a pathway through which the cerebellum communicates with the basal ganglia. Both the cerebellum and basal ganglia play crucial roles in motor control. Although songbirds have yielded key insights into the neural basis of vocal learning, it is unknown whether a thalamostriatal system exists in the songbird brain. Thalamic nucleus DLM is an important part of the song system, the network of nuclei required for learning and producing song. DLM receives output from song system basal ganglia nucleus Area X and sits within dorsal thalamus, the proposed avian homolog of the mammalian intralaminar nuclei that also receives projections from the cerebellar nuclei. Using a viral vector that specifically labels presynaptic axon segments, we show in Bengalese finches that dorsal thalamus projects to Area X, the basal ganglia nucleus of the song system, and to surrounding medial striatum. To identify the sources of thalamic input to Area X, we map DLM and cerebellar‐recipient dorsal thalamus (DTCbN). Surprisingly, we find both DLM and dorsal anterior DTCbNadjacent to DLM project to Area X. In contrast, the ventral medial subregion of DTCbNprojects to medial striatum outside Area X. Our results suggest the basal ganglia in the song system, like the mammalian basal ganglia, integrate feedback from the thalamic region to which they project as well as thalamic regions that receive cerebellar output.

    more » « less
  3. Abstract

    Vocal learning in songbirds is mediated by cortico‐basal ganglia circuits that govern diverse functions during different stages of development. We investigated developmental changes in axonal projections to and from motor cortical regions that underlie learned vocal behavior in juvenile zebra finches (Taeniopygia guttata). Neurons in LMAN‐core project to RA, a motor cortical region that drives vocal output; these RA‐projecting neurons send a transient collateral projection to AId, a region adjacent to RA, during early vocal development. Both RA and AId project to a region of dorsal thalamus (DLM), which forms a feedback pathway to cortico‐basal ganglia circuitry. These projections provide pathways conveying efference copy and a means by which information about vocal motor output could be reintegrated into cortico‐basal ganglia circuitry, potentially aiding in the refinement of juvenile vocalizations during learning. We used tract‐tracing techniques to label the projections of LMAN‐core to AId and of RA to DLM in juvenile songbirds. The volume and density of terminal label in the LMAN‐core→AId projection declined substantially during early stages of sensorimotor learning. In contrast, the RA→DLM projection showed no developmental change. The retraction of LMAN‐core→AId axon collaterals indicates a loss of efference copy to AId and suggests that projections that are present only during early stages of sensorimotor learning mediate unique, temporally restricted processes of goal‐directed learning. Conversely, the persistence of the RA→DLM projection may serve to convey motor information forward to the thalamus to facilitate song production during both learning and maintenance of vocalizations.

    more » « less
  4. Social networks are frequently polluted by rumors, which can be detected by advanced models such as graph neural networks. However, the models are vulnerable to attacks, and discovering and understanding the vulnerabilities is critical to robust rumor detection. To discover subtle vulnerabilities, we design a attacking algorithm based on reinforcement learning to camouflage rumors against black-box detectors. We address exponentially large state spaces, high-order graph dependencies, and ranking dependencies, which are unique to the problem setting but fundamentally challenging for the state-of-the-art end-to-end approaches. We design domain-specific features that have causal effect on the reward, so that even a linear policy can arrive at powerful attacks with additional interpretability. To speed up policy optimization, we devise: (i) a credit assignment method that proportionally decomposes delayed and aggregated rewards to atomic attacking actions for enhance feature-reward associations; (ii) a time-dependent control variate to reduce prediction variance due to large state-action spaces and long attack horizon, based on reward variance analysis and a Bayesian analysis of the prediction distribution. On two real world datasets of rumor detection tasks, we demonstrate: (i) the effectiveness of the learned attacking policy on a wide spectrum of target models compared to both rule-based and end-to-end attacking approaches; (ii) the usefulness of the proposed credit assignment strategy and variance reduction components; (iii) the interpretability of the attacking policy. 
    more » « less
  5. Abstract

    The ventral striatopallidal system, a basal ganglia network thought to convert limbic information into behavioral action, includes the nucleus accumbens (NAc) and the ventral pallidum (VP), typically described as a major output of NAc. Here, to investigate how reward-related information is transformed across this circuit, we measure the activity of neurons in NAc and VP when rats receive two highly palatable but differentially preferred rewards, allowing us to track the reward-specific information contained within the neural activity of each region. In VP, we find a prominent preference-related signal that flexibly reports the relative value of reward outcomes across multiple conditions. This reward-specific firing in VP is present in a greater proportion of the population and arises sooner following reward delivery than in NAc. Our findings establish VP as a preeminent value signaler and challenge the existing model of information flow in the ventral basal ganglia.

    more » « less