Search for: All records

Abstract Learning appropriate representations of the reward environment is challenging in the real world where there are many options, each with multiple attributes or features. Despite existence of alternative solutions for this challenge, neural mechanisms underlying emergence and adoption of value representations and learning strategies remain unknown. To address this, we measure learning and choice during a multi-dimensional probabilistic learning task in humans and trained recurrent neural networks (RNNs) to capture our experimental observations. We find that human participants estimate stimulus-outcome associations by learning and combining estimates of reward probabilities associated with the informative feature followed by those of informative conjunctions. Through analyzing representations, connectivity, and lesioning of the RNNs, we demonstrate this mixed learning strategy relies on a distributed neural code and opponency between excitatory and inhibitory neurons through value-dependent disinhibition. Together, our results suggest computational and neural mechanisms underlying emergence of complex learning strategies in naturalistic settings. 

Abstract The real world is uncertain, and while ever changing, it constantly presents itself in terms of new sets of behavioral options. To attain the flexibility required to tackle these challenges successfully, most mammalian brains are equipped with certain computational abilities that rely on the prefrontal cortex (PFC). By examining learning in terms of internal models associating stimuli, actions, and outcomes, we argue here that adaptive behavior relies on specific interactions between multiple systems including: (1) selective models learning stimulus–action associations through rewards; (2) predictive models learning stimulus- and/or action–outcome associations through statistical inferences anticipating behavioral outcomes; and (3) contextual models learning external cues associated with latent states of the environment. Critically, the PFC combines these internal models by forming task sets to drive behavior and, moreover, constantly evaluates the reliability of actor task sets in predicting external contingencies to switch between task sets or create new ones. We review different models of adaptive behavior to demonstrate how their components map onto this unifying framework and specific PFC regions. Finally, we discuss how our framework may help to better understand the neural computations and the cognitive architecture of PFC regions guiding adaptive behavior. 

Real-world choice options have many features or attributes, whereas the reward outcome from those options only depends on a few features or attributes. It has been shown that humans learn and combine feature-based with more complex conjunction-based learning to tackle challenges of learning in naturalistic reward environments. However, it remains unclear how different learning strategies interact to determine what features or conjunctions should be attended to and control choice behavior, and how subsequent attentional modulations influence future learning and choice. To address these questions, we examined the behavior of male and female human participants during a three-dimensional learning task in which reward outcomes for different stimuli could be predicted based on a combination of an informative feature and conjunction. Using multiple approaches, we found that both choice behavior and reward probabilities estimated by participants were most accurately described by attention-modulated models that learned the predictive values of both the informative feature and the informative conjunction. Specifically, in the reinforcement learning model that best fit choice data, attention was controlled by the difference in the integrated feature and conjunction values. The resulting attention weights modulated learning by increasing the learning rate on attended features and conjunctions. Critically, modulating decision-making by attention weights did not improve the fit of data, providing little evidence for direct attentional effects on choice. These results suggest that in multidimensional environments, humans direct their attention not only to selectively process reward-predictive attributes but also to find parsimonious representations of the reward contingencies for more efficient learning. 

Despite its prevalence in studying the causal roles of different brain circuits in cognitive processes, electrical microstimulation often results in inconsistent behavioral effects. These inconsistencies are assumed to be due to multiple mechanisms, including habituation, compensation by other brain circuits, and contralateral suppression. Considering the presence of reinforcement in most experimental paradigms, we hypothesized that interactions between reward feedback and microstimulation could contribute to inconsistencies in behavioral effects of microstimulation. To test this, we analyzed data from electrical microstimulation of the frontal eye field of male macaques during a value-based decision–making task and constructed network models to capture choice behavior. We found evidence for microstimulation-dependent adaptation in saccadic choice, such that in stimulated trials, monkeys’ choices were biased toward the target in the response field of the microstimulated site (T_in). In contrast, monkeys showed a bias away fromT_inin nonstimulated trials following microstimulation. Critically, this bias slowly decreased as a function of the time since the last stimulation. Moreover, microstimulation-dependent adaptation was influenced by reward outcomes in preceding trials. Despite these local effects, we found no evidence for the global effects of microstimulation on learning and sensitivity to the reward schedule. By simulating choice behavior across various network models, we found a model in which microstimulation and reward-value signals interact competitively through reward-dependent plasticity can best account for our observations. Our findings indicate a reward-dependent compensatory mechanism that enhances robustness to perturbations within the oculomotor system and could explain the inconsistent outcomes observed in previous microstimulation studies.