skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Neural Index of Reinforcement Learning Predicts Improved Stimulus–Response Retention under High Working Memory Load
Human learning and decision-making are supported by multiple systems operating in parallel. Recent studies isolating the contributions of reinforcement learning (RL) and working memory (WM) have revealed a trade-off between the two. An interactive WM/RL computational model predicts that although high WM load slows behavioral acquisition, it also induces larger prediction errors in the RL system that enhance robustness and retention of learned behaviors. Here, we tested this account by parametrically manipulating WM load during RL in conjunction with EEG in both male and female participants and administered two surprise memory tests. We further leveraged single-trial decoding of EEG signatures of RL and WM to determine whether their interaction predicted robust retention. Consistent with the model, behavioral learning was slower for associations acquired under higher load but showed parametrically improved future retention. This paradoxical result was mirrored by EEG indices of RL, which were strengthened under higher WM loads and predictive of more robust future behavioral retention of learned stimulus–response contingencies. We further tested whether stress alters the ability to shift between the two systems strategically to maximize immediate learning versus retention of information and found that induced stress had only a limited effect on this trade-off. The present results offer a deeper understanding of the cooperative interaction between WM and RL and show that relying on WM can benefit the rapid acquisition of choice behavior during learning but impairs retention. SIGNIFICANCE STATEMENT Successful learning is achieved by the joint contribution of the dopaminergic RL system and WM. The cooperative WM/RL model was productive in improving our understanding of the interplay between the two systems during learning, demonstrating that reliance on RL computations is modulated by WM load. However, the role of WM/RL systems in the retention of learned stimulus–response associations remained unestablished. Our results show that increased neural signatures of learning, indicative of greater RL computation, under high WM load also predicted better stimulus–response retention. This result supports a trade-off between the two systems, where degraded WM increases RL processing, which improves retention. Notably, we show that this cooperative interplay remains largely unaffected by acute stress.  more » « less
Award ID(s):
2020844
PAR ID:
10446612
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
The Journal of Neuroscience
Volume:
43
Issue:
17
ISSN:
0270-6474
Page Range / eLocation ID:
3131 to 3143
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. (, Personality and Social Psychology Bulletin)
    Research suggests that evaluations of an object can be jointly influenced by (a) the mere co-occurrence of the object with a pleasant or unpleasant stimulus (e.g., mere co-occurrence of object A and negative event B) and (b) the object’s specific relation to the co-occurring stimulus (e.g., object A starts vs. stops negative event B). Three experiments investigated the impact of cognitive load during learning on the effects of stimulus co-occurrence and stimulus relations. Counter to the shared prediction of competing theories suggesting that effects of stimulus relations should be reduced by cognitive load during learning, effects of stimulus relations were greater (rather than smaller) under high-load compared with low-load conditions. Effects of stimulus co-occurrence were not significantly affected by cognitive load. The results are discussed in terms of theories suggesting that cognitive load can influence behavioral outcomes via strategic shifts in resource allocation in response to task-specific affordances. 
    more » « less
  2. ; ; ; ; ; ; ; (, PeerJ)
    We aimed to examine mechanistically the observed foraging differences across two honey bee, Apis mellifera , subspecies using the proboscis extension response assay. Specifically, we compared differences in appetitive reversal learning ability between honey bee subspecies: Apis mellifera caucasica (Pollman), and Apis mellifera syriaca (Skorikov) in a “common garden” apiary. It was hypothesized that specific learning differences could explain previously observed foraging behavior differences of these subspecies: A.m. caucasica switches between different flower color morphs in response to reward variability, and A.m. syriaca does not switch. We suggest that flower constancy allows reduced exposure by minimizing search and handling time, whereas plasticity is important when maximizing harvest in preparation for long winter is at a premium. In the initial or Acquisition phase of the test we examined specifically discrimination learning, where bees were trained to respond to a paired conditioned stimulus with an unconditioned stimulus and not to respond to a second conditioned stimulus that is not followed by an unconditioned stimulus. We found no significant differences among the subspecies in the Acquisition phase in appetitive learning. During the second, Reversal phase of the experiment, where flexibility in association was tested, the paired and unpaired conditioned stimuli were reversed. During the Reversal phase A.m. syriaca showed a reduced ability to learn the reverse association in the appetitive learning task. This observation is consistent with the hypothesis that A.m. syriaca foragers cannot change the foraging choice because of lack of flexibility in appetitive associations under changing contingencies. Interestingly, both subspecies continued responding to the previously rewarded conditioned stimulus in the reversal phase. We discuss potential ecological correlates and molecular underpinnings of these differences in learning across the two subspecies. In addition, in a supplemental experiment we demonstrated that these differences in appetitive reversal learning do not occur in other learning contexts. 
    more » « less
  3. ; ; ; (, Nature Human Behaviour)
    Abstract Previous work has identified characteristic neural signatures of value-based decision-making, including neural dynamics that closely resemble the ramping evidence accumulation process believed to underpin choice. Here we test whether these signatures of the choice process can be temporally dissociated from additional, choice-‘independent’ value signals. Indeed, EEG activity during value-based choice revealed distinct spatiotemporal clusters, with a stimulus-locked cluster reflecting affective reactions to choice sets and a response-locked cluster reflecting choice difficulty. Surprisingly, ‘neither’ of these clusters met the criteria for an evidence accumulation signal. Instead, we found that stimulus-locked activity can ‘mimic’ an evidence accumulation process when aligned to the response. Re-analysing four previous studies, including three perceptual decision-making studies, we show that response-locked signatures of evidence accumulation disappear when stimulus-locked and response-locked activity are modelled jointly. Collectively, our findings show that neural signatures of value can reflect choice-independent processes and look deceptively like evidence accumulation. 
    more » « less
  4. ; (, Diseases)
    Background: Chronic stress, driven by the persistent activation of the body’s stress response system—including the sympathetic nervous system and hypothalamic–pituitary–adrenal (HPA) axis—has far-reaching effects on both physical and mental health. This study examines the combined effects of social and behavioral factors on a latent variable consisting of stress and depressive symptoms, using a comprehensive framework to explore the complex interactions of these factors. Methods: Leveraging data from the United States Centers for Disease Control and Prevention’s (CDC’s) National Health and Nutrition Examination Survey (NHANES), we operationalized allostatic load—a measure of cumulative physiological stress—through 10 biomarkers spanning cardiovascular, inflammatory, and metabolic systems. Depressive symptoms were measured via the Patient Health Questionnaire-9 (PHQ-9), and a latent variable capturing the shared variance between stress and depressive symptoms was derived using factor analysis. To assess the influence of social (income and education) and behavioral (alcohol consumption and smoking) factors on this latent variable, we employed Bayesian Kernel Machine Regression (BKMR), allowing us to examine potential non-linear and interactive effects among these predictors. Results: Our results revealed a significant positive association between allostatic load and depressive symptoms across the sample, regardless of ethnic background. Alcohol consumption emerged as a key behavioral factor, with significant positive associations with stress. Conversely, education showed a protective effect, with higher education levels associated with decreased stress and depressive symptoms. Conclusions: These findings underscore the importance of addressing both social determinants and behavioral risk factors in mitigating the cumulative impacts of stress and depressive symptoms. By highlighting the roles of alcohol consumption and education, this study provides insights that can inform public health strategies aimed at promoting resilience and reducing stress-related health disparities. 
    more » « less
  5. ; ; ; (, Nature Communications)
    Abstract Sensory stimuli evoke spiking activities patterned across neurons and time that are hypothesized to encode information about their identity. Since the same stimulus can be encountered in a multitude of ways, how stable or flexible are these stimulus-evoked responses? Here we examine this issue in the locust olfactory system. In the antennal lobe, we find that both spatial and temporal features of odor-evoked responses vary in a stimulus-history dependent manner. The response variations are not random, but allow the antennal lobe circuit to enhance the uniqueness of the current stimulus. Nevertheless, information about the odorant identity is conf ounded due to this contrast enhancement computation. Notably, predictions from a linear logical classifier (OR-of-ANDs) that can decode information distributed in flexible subsets of neurons match results from behavioral experiments. In sum, our results suggest that a trade-off between stability and flexibility in sensory coding can be achieved using a simple computational logic. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email