skip to main content


The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, May 23 until 2:00 AM ET on Friday, May 24 due to maintenance. We apologize for the inconvenience.

This content will become publicly available on January 1, 2025

Title: A Calibration Model for Bot-Like Behaviors in Agent-Based Anagram Game Simulation
Experiments that are games played among a network of players are widely used to study human behavior. Furthermore, bots or intelligent systems can be used in these games to produce contexts that elicit particular types of human responses. Bot behaviors could be specified solely based on experimental data. In this work, we take a different perspective, called the Probability Calibration (PC) approach, to simulate networked group anagram games with certain players having bot-like behaviors. The proposed method starts with data-driven models and calibrates in principled ways the parameters that alter player behaviors. It can alter the performance of each type of agent (e.g., bot) in group anagram games. Further, statistical methods are used to test whether the PC models produce results that are statistically different from those of the original models. Case studies demonstrate the merits of the proposed method.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ;
Corlu, C. G.; Hunter, S. R.; Lam, H.; Onggo, B. S.; Shortle, J.; Biller, B.
Publisher / Repository:
Date Published:
Journal Name:
Proceedings Winter Simulation Conference
Medium: X
San Antonio, Texas
Sponsoring Org:
National Science Foundation
More Like this
  1. In group anagram games, players cooperate to form words by sharing letters that they are initially given. The aim is to form as many words as possible as a group, within five minutes. Players take several different actions: requesting letters from their neighbors, replying to letter requests, and forming words. Agent-based models (ABMs) for the game compute likelihoods of each player’s next action, which contain uncertainty, as they are estimated from experimental data. We adopt a Bayesian approach as a natural means of quantifying uncertainty, to enhance the ABM for the group anagram game. Specifically, a Bayesian nonparametric clustering method is used to group player behaviors into different clusters without pre-specifying the number of clusters. Bayesian multi nominal regression is adopted to model the transition probabilities among different actions of the players in the ABM. We describe the methodology and the benefits of it, and perform agent-based simulations of the game. 
    more » « less
  2. null (Ed.)
    In anagram games, players are provided with letters for forming as many words as possible over a specified time duration. Anagram games have been used in controlled experiments to study problems such as collective identity, effects of goal setting, internal-external attributions, test anxiety, and others. The majority of work on anagram games involves individual players. Recently, work has expanded to group anagram games where players cooperate by sharing letters. In this work, we analyze experimental data from online social networked experiments of group anagram games. We develop mechanistic and data driven models of human decision-making to predict detailed game player actions (e.g., what word to form next). With these results, we develop a composite agent-based modeling and simulation platform that incorporates the models from data analysis. We compare model predictions against experimental data, which enables us to provide explanations of human decision-making and behavior. Finally, we provide illustrative case studies using agent-based simulations to demonstrate the efficacy of models to provide insights that are beyond those from experiments alone. 
    more » « less
  3. In a group anagram game, players are provided letters to form as many words as possible. They can also request letters from their neighbors and reply to letter requests. Currently, a single agent-based model is produced from all experimental data, with dependence only on number of neighbors. In this work, we build, exercise, and evaluate enhanced agent behavior models for networked group anagram games under an uncertainty quantification framework. Specifically, we cluster game data for players based on their skill levels (forming words, requesting letters, and replying to requests), perform multinomial logistic regression for transition probabilities, and quantify uncertainty within each cluster. The result of this process is a model where players are assigned different numbers of neighbors and different skill levels in the game. We conduct simulations of ego agents with neighbors to demonstrate the efficacy of our proposed methods. 
    more » « less
  4. When modeling human behavior in multi-player games, it is important to understand heterogeneous aspects of player behaviors. By leveraging experimental data and agent-based simulations, various data-driven modeling methods can be applied. This provides a great opportunity to quantify and visualize the uncertainty associated with these methods, allowing for a more comprehensive understanding of the individual and collective behaviors among players. For networked anagram games, player behaviors can be heterogeneous in terms of the number of words formed and the amount of cooperation among networked neighbors. Based on game data, these games can be modeled as discrete dynamical systems characterized by probabilistic state transitions. In this work, we present both Frequentist and Bayesian approaches for visualizing uncertainty in networked anagram games. These approaches help to elaborate how players individually and collectively form words by sharing letters with their neighbors in a network. Both approaches provide valuable insights into inferring the worst, average, and best player performance within and between behavioral clusters. Moreover, interesting contrasts between the Frequentist and Bayesian approaches can be observed. The knowledge and inferences gained from these approaches are incorporated into an agent-based simulation framework to further demonstrate model uncertainty and players’ heterogeneous behaviors. 
    more » « less
  5. In a networked anagram game, each team member is given a set of letters and members collectively form as many words as possible. They can share letters through a communication network in assisting their neighbors in forming words. There is variability in behaviors of players, e.g., there can be large differences in numbers of letter requests, of replies to letter requests, and of words formed among players. Therefore, it is of great importance to understand uncertainty and variability in player behaviors. In this work, we propose versatile uncertainty quantification (VUQ) of behaviors for modeling the networked anagram game. Specifically, the proposed methods focus on building contrastive models of game player behaviors that quantify player actions in terms of worst, average, and best performance. Moreover, we construct agent-based models and perform agent-based simulations using these VUQ methods to evaluate the model building methodology and understand the impact of uncertainty. We believe that this approach is applicable to other networked games. 
    more » « less