An official website of the United States government
In this paper we describe the need for a framework to support collaborative educational research with game data, then demonstrate a promising solution. We review existing efforts, explore a collection of use cases and requirements, then propose a new data architecture with related data standards. The approach provides modularity to the various stages of game data generation and analysis, exposing intermediate transformations and work products. Foregrounding flexibility, each stage of the pipeline generates datasets for use in other tools and workflows. A series of interconnected standards allow for the development of reusable analysis and visualization tools across games, while remaining responsive to the diversity of potential game designs. Finally, we demonstrate the feasibility of the approach through an existing implementation that uses this architecture to process and analyze data from a wide range of games developed by multiple institutions, at scale, supporting a variety of research projects.
more »
« less
A Systematic Review of Data Collection and Analysis Methods in K–12 Educational Games
This paper reports on systematic literature review that examined learning theories and data collection and analysis methods used to study game-based learning in research on educational digital games for K-12 populations. Through electronic database, hand, and ancestral searches, we identified 25 empirical studies (29 educational games) published in peer-review journals that report evidence of how students learn through in-game and out-of-game data collection and analysis methods. Taking an approach to game-based learning as identity-driven and situated, we found that while games do not take such an approach to game-based learning, games tend to collect data on players’ social interactions and collaborative experiences. The review also highlighted the opportunity for providing real-time feedback and data to players during gameplay.
more »
« less
- Award ID(s):
- 2214516
- PAR ID:
- 10537996
- Publisher / Repository:
- AERA Online Paper Repository
- Date Published:
- Subject(s) / Keyword(s):
- Computers and Learning Data Analysis Video Games
- Format(s):
- Medium: X
- Location:
- Philadelphia, Pennsylvania
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Devising models that reliably recognize player goals is a key challenge in creating player-adaptive games. Player goal recognition is the task of automatically recognizing the intent of a player from a sequence of observed player actions in a game environment. In open-world digital games, players often undertake suboptimal and varied sequences of actions to achieve goals, and the high degree of freedom afforded to players makes it challenging to identify sequential patterns that lead toward specific goals. To address these issues, we present a player goal recognition framework that utilizes a fine-tuned T5 language model, which incorporates our novel attention mechanism called Temporal Contrary Attention (TCA). The T5 language model enables the framework to exploit correlations between observations through non-sequential self-attention within input sequences, while TCA enables the framework to learn to eliminate goal hypotheses by considering counterevidence within a temporal window. We evaluate our approach using game trace data collected from 144 players' interactions with an open-world educational game. Specifically, we investigate the predictive capacity of our approach to recognize player goals as well as player plans represented as abstract actions. Results show that our approach outperforms non-linguistic machine learning approaches as well as T5 without TCA. We discuss the implications of these findings for the design and development of player goal recognition models to create player-adaptive games.more » « less
-
Modeling player engagement is a key challenge in games. However, the gameplay signatures of engaged players can be highly context-sensitive, varying based on where the game is used or what population of players is using it. Traditionally, models of player engagement are investigated in a particular context, and it is unclear how effectively these models generalize to other settings and populations. In this work, we investigate a Bayesian hierarchical linear model for multi-task learning to devise a model of player engagement from a pair of datasets that were gathered in two complementary contexts: a Classroom Study with middle school students and a Laboratory Study with undergraduate students. Both groups of players used similar versions of Crystal Island, an educational interactive narrative game for science learning. Results indicate that the Bayesian hierarchical model outperforms both pooled and context-specific models in cross-validation measures of predicting player motivation from in-game behaviors, particularly for the smaller Classroom Study group. Further, we find that the posterior distributions of model parameters indicate that the coefficient for a measure of gameplay performance significantly differs between groups. Drawing upon their capacity to share information across groups, hierarchical Bayesian methods provide an effective approach for modeling player engagement with data from similar, but different, contexts.more » « less
-
Educational prediction games use the popularity and engagement of fantasy sports as a success model to promote learning in other domains. Fantasy sports motivate players to stay up-to-date with relevant news and explore large statistical data sets, thereby deepening their domain understanding while potentially honing their data analysis skills. We conducted a study of fantasy sports players, and discovered that while some participants performed sophisticated data analysis to support their gameplay, far more relied on news and published commentary. We used results from this study to design a prototype prediction game, Fantasy Climate, which helps players move from intuitions and advice to consuming news and analyzing data by supporting a variety of activities essential to gameplay. Because news is a key component of Fantasy Climate, we evaluated two link-based interfaces to domain-related news, one geospatial and the other organized as a list. The evaluation revealed that news presentation has a strong effect on players' engagement and performance: players using the geospatial interface not only were more engaged in the game; they also made better predictions than players who used the list-based presentation.more » « less
-
Maps in video games have grown into complex interactive systems alongside video games themselves. What map systems have done and currently do have not been cataloged or evaluated. We trace the history of game map interfaces from their paper-based inspiration to their current smart phone-like appearance. Read-only map interfaces enable players to consume maps, which is sufficient for wayfinding. Game cartography interfaces enable players to persistently modify maps, expanding the range of activity to support planning and coordination. We employ thematic analysis on game cartography interfaces, contributing a near-exhaustive catalog of games featuring such interfaces, a set of properties to describe and design such interfaces, a collection of play activities that relate to cartography, and a framework to identify what properties promote the activities. We expect that designers will find the contributions enable them to promote desired play experiences through game map interface design.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.3102/2110403
