An official website of the United States government
Abstract When people make choices, the items they consider are often embedded in a context (of other items). How this context affects the valuation of the specific item is an important question. High-value context might make items appear less attractive because of contrast—the tendency to normalize perception of an object relative to its background—or more attractive because of assimilation—the tendency to group objects together. Alternatively, a high-value context might increase prior expectations about the item's value. Here, we investigated these possibilities. We examined how unavailable context items affect choices between two target items, as well as the willingness-to-pay for single targets. Participants viewed sets of three items for several seconds before the target(s) were highlighted. In both tasks, we found a significant assimilation-like effect where participants were more likely to choose or place a higher value on a target when it was surrounded by higher-value context. However, these context effects were only significant for participants’ fastest choices. Using variants of a drift-diffusion model, we established that the unavailable context shifted participants’ prior expectations towards the average values of the sets but had an inconclusive effect on their evaluations of the targets during the decision (i.e. drift rates). In summary, we find that people use context to inform their initial valuations. This can improve efficiency by allowing people to get a head start on their decision. However, it also means that the valuation of an item can change depending on the context.
more »
« less
Intrinsic rewards explain context-sensitive valuation in reinforcement learning
When observing the outcome of a choice, people are sensitive to the choice’s context, such that the experienced value of an option depends on the alternatives: getting $1 when the possibilities were 0 or 1 feels much better than when the possibilities were 1 or 10. Context-sensitive valuation has been documented within reinforcement learning (RL) tasks, in which values are learned from experience through trial and error. Range adaptation, wherein options are rescaled according to the range of values yielded by available options, has been proposed to account for this phenomenon. However, we propose that other mechanisms—reflecting a different theoretical viewpoint—may also explain this phenomenon. Specifically, we theorize that internally defined goals play a crucial role in shaping the subjective value attributed to any given option. Motivated by this theory, we develop a new “intrinsically enhanced” RL model, which combines extrinsically provided rewards with internally generated signals of goal achievement as a teaching signal. Across 7 different studies (including previously published data sets as well as a novel, preregistered experiment with replication and control studies), we show that the intrinsically enhanced model can explain context-sensitive valuation as well as, or better than, range adaptation. Our findings indicate a more prominent role of intrinsic, goal-dependent rewards than previously recognized within formal models of human RL. By integrating internally generated signals of reward, standard RL theories should better account for human behavior, including context-sensitive valuation and beyond.
more »
« less
- Award ID(s):
- 2020844
- PAR ID:
- 10446717
- Editor(s):
- Summerfield, Christopher
- Date Published:
- Journal Name:
- PLOS Biology
- Volume:
- 21
- Issue:
- 7
- ISSN:
- 1545-7885
- Page Range / eLocation ID:
- e3002201
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Viale, R. (Ed.)Alternative-based approaches to decision making generate overall values for each option in a choice set by processing information within options before comparing options to arrive at a decision. By contrast, attribute-based approaches compare attributes (such as monetary cost and time delay to receipt of a reward) across options and use these attribute comparisons to make a decision. Because they compare attributes, they may not use all available information to make a choice, which categorizes many of them as heuristics. Attribute-based models can better predict choice compared to alternative-based models in some situations (e.g., when there are many options in the choice set, when calculating an overall value for an option is too cognitively taxing). Process data comparing alternative-based and attribute-based processing obtained from eye-tracking and mouse-tracking technology support these findings. Data on attribute-based models thus align with the notion of bounded rationality that people make use of heuristics to make good decisions when under time pressure, informational constraints, and computational constraints. Further study of attribute-based models and processing would enhance our understanding of how individuals process information and make decisions.more » « less
-
While most models of decision-making assume that individuals assign options absolute values, animals often assess options comparatively, violating principles of economic rationality. Such ‘irrational’ preferences are especially common when two rewards vary along multiple dimensions of quality and a third, ‘decoy’ option is available. Bumblebees are models of decision-making, yet whether they are subject to decoy effects is unknown. We addressed this question using bumblebees (Bombus impatiens) choosing between flowers that varied in their nectar concentration and reward rate. We first gave bees a choice between two flower types, one higher in concentration and the other higher in reward rate. Bees were then given a choice between these flowers and either a ‘concentration’ or ‘rate’ decoy, designed to be asymmetrically dominated on each axis. The rate decoy increased bees’ preference in the expected direction, while the concentration decoy did not. In a second experiment, we manipulated choices along two single reward dimensions to test whether this discrepancy was explained by differences in how concentration versus reward rate were evaluated. We found that low-concentration decoys increased bees’ preference for the medium option as predicted, whereas low-rate decoys had no effect. Our results suggest that both low- and high-value flowers can influence pollinator preferences in ways previously unconsidered.more » « less
-
null (Ed.)Non-human primates evaluate choices based on quantitative information and subjective valuation of options. Non-human primates can learn to value tokens as placeholders for primary rewards (such as food). With those tokens established as a potential form of ‘currency’, it is then possible to examine how they respond to opportunities to earn and use tokens in ways such as accumulating tokens or exchanging tokens with each other or with human experimenters to gain primary rewards. Sometimes, individuals make efficient and beneficial choices to obtain tokens and then exchange them at the right moments to gain optimal reward. Sometimes, they even accumulate such rewards through extended delay of gratification, or through other exchange-based interactions. Thus, non-human primates are capable of associating value to arbitrary tokens that may function as currency-like stimuli, but there also are strong limitations on how non-human primates can integrate such tokens into choice situations or use such tokens to fully ‘symbolize’ economic decision-making. These limitations are important to acknowledge when considering the evolutionary emergence of currency use in our species. This article is part of the theme issue ‘Existence and prevalence of economic behaviours among non-human primates’.more » « less
-
While classic models of animal decision-making assume that individuals assess the absolute value of options, decades of research have shown that rewards are often evaluated relative to recent experience, creating incentive contrast effects. Contrast effects are often assumed to be purely sensory, yet consumer and experimental psychology tell us that label-based expectations can affect value perception in humans and rodents. However, this has rarely been tested in non-model systems. Bumblebees forage on a variety of flowers that vary in their signals and rewards and show contrast when rewards are lowered. We manipulated bees' expectations of stimulus quality, before downshifting the reward to induce incentive contrast. We found that contrast effects were not solely driven by experience with a better reward, but also influenced by experience with associated stimuli. While bees' initial response did not differ between treatments, individuals were faster to accept the lower-quality reward when it was paired with a novel stimulus. We explored the boundaries of these label-based expectations by testing bees along a stimulus gradient and found that expectations generalized to similar stimuli. Such reference-dependent evaluations may play an important role in bees' foraging choices, with the potential to impact floral evolution and plant community dynamics.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1371/journal.pbio.3002201
