skip to main content

Title: Modeling Player Engagement with Bayesian Hierarchical Models
Modeling player engagement is a key challenge in games. However, the gameplay signatures of engaged players can be highly context-sensitive, varying based on where the game is used or what population of players is using it. Traditionally, models of player engagement are investigated in a particular context, and it is unclear how effectively these models generalize to other settings and populations. In this work, we investigate a Bayesian hierarchical linear model for multi-task learning to devise a model of player engagement from a pair of datasets that were gathered in two complementary contexts: a Classroom Study with middle school students and a Laboratory Study with undergraduate students. Both groups of players used similar versions of Crystal Island, an educational interactive narrative game for science learning. Results indicate that the Bayesian hierarchical model outperforms both pooled and context-specific models in cross-validation measures of predicting player motivation from in-game behaviors, particularly for the smaller Classroom Study group. Further, we find that the posterior distributions of model parameters indicate that the coefficient for a measure of gameplay performance significantly differs between groups. Drawing upon their capacity to share information across groups, hierarchical Bayesian methods provide an effective approach for modeling player engagement with more » data from similar, but different, contexts. « less
; ; ;
Award ID(s):
Publication Date:
Journal Name:
Proceedings of the Fourteenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment
Page Range or eLocation-ID:
Sponsoring Org:
National Science Foundation
More Like this
  1. Game-based learning (GBL) has increasingly been used to promote students’ learning engagement. Although prior GBL studies have highlighted the significance of learning engagement as a mediator of students’ meaningful learning, the existing accounts failed to capture specific evidence of how exactly students’ in-game actions in GBL enhance learning engagement. Hence, this mixed-method study was designed to examine whether middle school students’ in-game actions are likely to promote certain types of learning engagement (i.e., content and cognitive engagement). This study used and examined the game E-Rebuild, a single-player three-dimensional architecture game that requires learners’ application of math knowledge. Using in-depth gameplay behavior analysis, this study sampled a total of 92 screen-recorded and video-captured gameplay sessions attended by 25 middle school students. We adopted two analytic approaches: sequential analysis and thematic analysis. Whereas sequential analysis explored which in-game actions by students were likely to promote each type of learning engagement, the thematic analysis depicted how certain gameplay contexts contributed to students’ enhanced learning engagement. The study found that refugee allocation and material trading actions promoted students’ content engagement, whereas using in-game building tools and learning support boosted their cognitive engagement. This study also found that students’ learning engagement was associated with theirmore »development of mathematical thinking in a GBL context.« less
  2. Although prior research has highlighted the significance of representations for mathematical learning, there is still a lack of research on how students use multimodal external representations (MERs) to solve mathematical tasks in digital game-based learning (DGBL) environments. This exploratory study was to examine the salient patterns problem solvers demonstrated using MERs when they engaged in a single-player, three-dimensional architecture game that requires the acquisition and application of math knowledge and thinking in game-based context problem solving. We recorded and systematically coded the behaviors of using MERs demonstrated by 20 university students during 1.5 hours of gameplay. We conducted both cluster and sequential analyses with a total of 2654 encoded behaviors. The study indicated that the maneuverable visual-spatial representation was most frequently used in the selected architecture game. All of the participants performed a high level of representational transformations, including both treatment and conversion transformations. However, compared to the students in the second cluster who were mostly non-game players, students in the first cluster (composed of mainly experienced video game players) displayed a higher frequency of interacting with various MERs and a more cautious and optimized reflective problem-solving process.
  3. Incorporating games in teaching can help students retain material and become innovative problem solvers through engagement and enjoyment. Here we describe a new board game, “Taphonomy: Dead and Fossilized,” and its use as an active learning tool (material available at doi: 10.18738/T8/NQV2CU). The educational objective is to teach the player about taphonomy and fossilization, while the gameplay objective is to preserve and recover the best fossil collection. Through competitive gameplay, students learn how chemical, physical, and environmental factors, as well as physiology and discovery biases can influence an organism’s preservation and collection potential. The game is modeled after an Early Jurassic fossil deposit for scientific accuracy and relevance. The game was incorporated in undergraduate classroom activities in 20 colleges and universities across the United States. Survey results show that students and teachers were overwhelmingly positive about the game, stating that it was fun and helped them learn or strengthen their knowledge of fossilization. When analyzed statistically, we find that students’ self-reported learning outcomes and opinions vary most significantly with college year, major, ethnicity, and race. White students and geoscience or STEM majors reported the highest levels of learning and enjoyment, with minorities and non-STEM majors responding less favorably. We suggestmore »this game is most advantageous for use in upper-level paleontology classrooms, although it is still beneficial at lower levels. It is critical to use this game as part of a larger lesson plan and tailor it to fit the needs of an individual classroom. Modifications for time and class size, as well as follow-up discussion questions, are included.« less
  4. Understanding players' mental models are crucial for game designers who wish to successfully integrate player-AI interactions into their game. However, game designers face the difficult challenge of anticipating how players model these AI agents during gameplay and how they may change their mental models with experience. In this work, we conduct a qualitative study to examine how a pair of players develop mental models of an adversarial AI player during gameplay in the multiplayer drawing game iNNk. We conducted ten gameplay sessions in which two players (n = 20, 10 pairs) worked together to defeat an AI player. As a result of our analysis, we uncovered two dominant dimensions that describe players' mental model development (i.e., focus and style). The first dimension describes the focus of development which refers to what players pay attention to for the development of their mental model (i.e., top-down vs. bottom-up focus). The second dimension describes the differences in the style of development, which refers to how players integrate new information into their mental model (i.e., systematic vs. reactive style). In our preliminary framework, we further note how players process a change when a discrepancy occurs, which we observed occur through comparisons (i.e., compare tomore »other systems, compare to gameplay, compare to self). We offer these results as a preliminary framework for player mental model development to help game designers anticipate how different players may model adversarial AI players during gameplay.« less
  5. Educational prediction games use the popularity and engagement of fantasy sports as a success model to promote learning in other domains. Fantasy sports motivate players to stay up-to-date with relevant news and explore large statistical data sets, thereby deepening their domain understanding while potentially honing their data analysis skills. We conducted a study of fantasy sports players, and discovered that while some participants performed sophisticated data analysis to support their gameplay, far more relied on news and published commentary. We used results from this study to design a prototype prediction game, Fantasy Climate, which helps players move from intuitions and advice to consuming news and analyzing data by supporting a variety of activities essential to gameplay. Because news is a key component of Fantasy Climate, we evaluated two link-based interfaces to domain-related news, one geospatial and the other organized as a list. The evaluation revealed that news presentation has a strong effect on players' engagement and performance: players using the geospatial interface not only were more engaged in the game; they also made better predictions than players who used the list-based presentation.