skip to main content


Title: Approaches to Uncertainty Quantification in Building Models of Human Behavior
In a group anagram game, players are provided letters to form as many words as possible. They can also request letters from their neighbors and reply to letter requests. Currently, a single agent-based model is produced from all experimental data, with dependence only on number of neighbors. In this work, we build, exercise, and evaluate enhanced agent behavior models for networked group anagram games under an uncertainty quantification framework. Specifically, we cluster game data for players based on their skill levels (forming words, requesting letters, and replying to requests), perform multinomial logistic regression for transition probabilities, and quantify uncertainty within each cluster. The result of this process is a model where players are assigned different numbers of neighbors and different skill levels in the game. We conduct simulations of ego agents with neighbors to demonstrate the efficacy of our proposed methods.  more » « less
Award ID(s):
1916670
NSF-PAR ID:
10310244
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Winter Simulation Conference
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. In group anagram games, players cooperate to form words by sharing letters that they are initially given. The aim is to form as many words as possible as a group, within five minutes. Players take several different actions: requesting letters from their neighbors, replying to letter requests, and forming words. Agent-based models (ABMs) for the game compute likelihoods of each player’s next action, which contain uncertainty, as they are estimated from experimental data. We adopt a Bayesian approach as a natural means of quantifying uncertainty, to enhance the ABM for the group anagram game. Specifically, a Bayesian nonparametric clustering method is used to group player behaviors into different clusters without pre-specifying the number of clusters. Bayesian multi nominal regression is adopted to model the transition probabilities among different actions of the players in the ABM. We describe the methodology and the benefits of it, and perform agent-based simulations of the game. 
    more » « less
  2. In a networked anagram game, each team member is given a set of letters and members collectively form as many words as possible. They can share letters through a communication network in assisting their neighbors in forming words. There is variability in behaviors of players, e.g., there can be large differences in numbers of letter requests, of replies to letter requests, and of words formed among players. Therefore, it is of great importance to understand uncertainty and variability in player behaviors. In this work, we propose versatile uncertainty quantification (VUQ) of behaviors for modeling the networked anagram game. Specifically, the proposed methods focus on building contrastive models of game player behaviors that quantify player actions in terms of worst, average, and best performance. Moreover, we construct agent-based models and perform agent-based simulations using these VUQ methods to evaluate the model building methodology and understand the impact of uncertainty. We believe that this approach is applicable to other networked games. 
    more » « less
  3. When modeling human behavior in multi-player games, it is important to understand heterogeneous aspects of player behaviors. By leveraging experimental data and agent-based simulations, various data-driven modeling methods can be applied. This provides a great opportunity to quantify and visualize the uncertainty associated with these methods, allowing for a more comprehensive understanding of the individual and collective behaviors among players. For networked anagram games, player behaviors can be heterogeneous in terms of the number of words formed and the amount of cooperation among networked neighbors. Based on game data, these games can be modeled as discrete dynamical systems characterized by probabilistic state transitions. In this work, we present both Frequentist and Bayesian approaches for visualizing uncertainty in networked anagram games. These approaches help to elaborate how players individually and collectively form words by sharing letters with their neighbors in a network. Both approaches provide valuable insights into inferring the worst, average, and best player performance within and between behavioral clusters. Moreover, interesting contrasts between the Frequentist and Bayesian approaches can be observed. The knowledge and inferences gained from these approaches are incorporated into an agent-based simulation framework to further demonstrate model uncertainty and players’ heterogeneous behaviors. 
    more » « less
  4. null (Ed.)
    In anagram games, players are provided with letters for forming as many words as possible over a specified time duration. Anagram games have been used in controlled experiments to study problems such as collective identity, effects of goal setting, internal-external attributions, test anxiety, and others. The majority of work on anagram games involves individual players. Recently, work has expanded to group anagram games where players cooperate by sharing letters. In this work, we analyze experimental data from online social networked experiments of group anagram games. We develop mechanistic and data driven models of human decision-making to predict detailed game player actions (e.g., what word to form next). With these results, we develop a composite agent-based modeling and simulation platform that incorporates the models from data analysis. We compare model predictions against experimental data, which enables us to provide explanations of human decision-making and behavior. Finally, we provide illustrative case studies using agent-based simulations to demonstrate the efficacy of models to provide insights that are beyond those from experiments alone. 
    more » « less
  5. Anagram games (i.e., word construction games in which players use letters to form words) have been researched for some 60 years. Games with individual players are the subject of over 20 published investigations. Moreover, there are many popular commercial anagram games such as Scrabble. Recently, cooperative team play of anagram games has been studied experimentally. With all of the experimental work and the popularity of such games, it is somewhat surprising that very little modeling of anagram games has been done to predict player behavior/actions in them. We devise a cooperative group anagram game and develop an agent-based modeling and simulation framework to capture player interactions of sharing letters and forming words. Our primary goals are to understand, quantitatively predict, and explain individual and aggregate group behavior, through simulations, to inform the design of a group anagram game experimental platform. 
    more » « less