skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: The Wisdom of Model Crowds
A wide body of empirical research has revealed the descriptive shortcomings of expected value and expected utility models of risky decision making. In response, numerous models have been advanced to predict and explain people’s choices between gambles. Although some of these models have had a great impact in the behavioral, social, and management sciences, there is little consensus about which model offers the best account of choice behavior. In this paper, we conduct a large-scale comparison of 58 prominent models of risky choice, using 19 existing behavioral data sets involving more than 800 participants. This allows us to comprehensively evaluate models in terms of individual-level predictive performance across a range of different choice settings. We also identify the psychological mechanisms that lead to superior predictive performance and the properties of choice stimuli that favor certain types of models over others. Moreover, drawing on research on the wisdom of crowds, we argue that each of the existing models can be seen as an expert that provides unique forecasts in choice predictions. Consistent with this claim, we find that crowds of risky choice models perform better than individual models and thus provide a performance bound for assessing the historical accumulation of knowledge in our field. Our results suggest that each model captures unique aspects of the decision process and that existing risky choice models offer complementary rather than competing accounts of behavior. We discuss the implications of our results on theories of risky decision making and the quantitative modeling of choice behavior. This paper was accepted by Yuval Rottenstreich, behavioral economics and decision analysis.  more » « less
Award ID(s):
1847794
PAR ID:
10394341
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Management Science
Volume:
68
Issue:
5
ISSN:
0025-1909
Page Range / eLocation ID:
3635 to 3659
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Human decision-making involves the coordinated activity of multiple brain areas, acting in concert, to enable humans to make choices. Most decisions are carried out under conditions of uncertainty, where the desired outcome may not be achieved if the wrong decision is made. In these cases, humans deliberate before making a choice. The neural dynamics underlying deliberation are unknown and intracranial recordings in clinical settings present a unique opportunity to record high temporal resolution electrophysiological data from many (hundreds) brain locations during behavior. Combined with dynamic systems modeling, these allow identification of latent brain states that describe the neural dynamics during decision-making, providing insight into these neural dynamics and computations. Results show that the neural dynamics underlying risky decisions, but not decisions without risk, converge to separate subspaces depending on the subject’s preferred choice and that the degree of overlap between these subspaces declines as choice approaches, suggesting a network level representation of evidence accumulation. These results bridge the gap between regression analyses and data driven models of latent states and suggest that during risky decisions, deliberation and evidence accumulation toward a final decision are represented by the same neural dynamics, providing novel insights into the neural computations underlying human choice. 
    more » « less
  2. Predicting and understanding how people make decisions has been a long-standing goal in many fields, with quantitative models of human decision-making informing research in both the social sciences and engineering. We show how progress toward this goal can be accelerated by using large datasets to power machine-learning algorithms that are constrained to produce interpretable psychological theories. Conducting the largest experiment on risky choice to date and analyzing the results using gradient-based optimization of differentiable decision theories implemented through artificial neural networks, we were able to recapitulate historical discoveries, establish that there is room to improve on existing theories, and discover a new, more accurate model of human decision-making in a form that preserves the insights from centuries of research. 
    more » « less
  3. AI-assisted decision-making systems hold immense potential to enhance human judgment, but their effectiveness is often hindered by a lack of understanding of the diverse ways in which humans take AI recommendations. Current research frequently relies on simplified, ``one-size-fits-all'' models to characterize an average human decision-maker, thus failing to capture the heterogeneity of people's decision-making behavior when incorporating AI assistance. To address this, we propose Mix and Match (M&M), a novel computational framework that explicitly models the diversity of human decision-makers and their unique patterns of relying on AI assistance. M&M represents the population of decision-makers as a mixture of distinct decision-making processes, with each process corresponding to a specific type of decision-maker. This approach enables us to infer latent behavioral patterns from limited data of human decisions under AI assistance, offering valuable insights into the cognitive processes underlying human-AI collaboration. Using real-world behavioral data, our empirical evaluation demonstrates that M&M consistently outperforms baseline methods in predicting human decision behavior. Furthermore, through a detailed analysis of the decision-maker types identified in our framework, we provide quantitative insights into nuanced patterns of how different individuals adopt AI recommendations. These findings offer implications for designing personalized and effective AI systems based on the diverse landscape of human behavior patterns in AI-assisted decision-making across various domains. 
    more » « less
  4. Collective decision-making constitutes a core function of social systems and is, therefore, a central tenet of collective intelligence research. From fish schools to human crowds, we start by interrogating ourselves about the very definition of collective decision-making and the scope of the scientific research that falls under it. We then summarize its history through the lenses of social choice theory and swarm intelligence and their accelerating collaboration over the past 20 or so years. Finally, we offer our perspective on the future of collective decision-making research in 3 mutually inclusive directions. We argue (1) that the possibility to collect data of a new nature, including fine-grain tracking information, virtual reality, and brain imaging inputs, will enable a direct link between plastic individual cognitive processes and the ontogeny of collective behaviors; (2) that current theoretical frameworks are not well suited to describe the long-term consequences of individual plasticity on collective decision-making processes and that, therefore, new formalisms are necessary; and finally (3) that applying the results of collective decision-making research to real-world situations will require the development of practical tools, the implementation of monitoring processes that respect civil liberties, and, possibly, government regulations of social interventions by public and private actors. 
    more » « less
  5. Abstract AimsWe sought to explore how acute alcohol exposure alters decision-making in rats performing an approach-avoid decision-making task. Increasing concentrations of alcohol were mixed with decreasing concentrations of sucrose to mimic mixed/sweetened alcoholic beverages. MethodsRats were trained on an apparatus in which different concentrations of sucrose were available in four different corners of the arena. During daily sessions, a tone signaled each trial start, followed by illumination (15 lux, blue LEDs) of a single corner port, indicating the potential availability of sucrose at that location. The rat (one rat per arena, both females and males) then chose to approach the lit corner to have the solution dispensed or avoid it, with no solution being dispensed. We examined how the decisions to pursue sucrose rewards shifted with the addition and subsequent removal of ethanol from the sucrose ports. ResultsMales were greatly affected by the introduction of alcohol into the task environment, shifting their approach preference to solutions containing higher alcohol concentrations rather than maintaining the prior preference for high-sucrose-concentration solutions. In contrast, females’ choice patterns and task performance remained largely unchanged. We also explore a method for identifying changes in decision-making tendencies during and after alcohol consumption within individual subjects. ConclusionsThis research explores the introduction of alcohol in varying concentrations with sucrose solutions during an approach-avoid task, with male decision-making and behavioral patterns significantly impacted. We also explore a novel approach for identifying individual adaptations of decision-making behavior when alcohol becomes available, which could be expanded upon in future research. 
    more » « less