In many predictive decision-making scenarios, such as credit scoring and academic testing, a decision-maker must construct a model that accounts for agents' incentives to ``game'' their features in order to receive better decisions. Whereas the strategic classification literature generally assumes that agents' outcomes are not causally dependent on their features (and thus strategic behavior is a form of lying), we join concurrent work in modeling agents' outcomes as a function of their changeable attributes. Our formulation is the first to incorporate a crucial phenomenon: when agents act to change observable features, they may as a side effect perturb unobserved features that causally affect their true outcomes. We consider three distinct desiderata for a decision-maker's model: accurately predicting agents' post-gaming outcomes (accuracy), incentivizing agents to improve these outcomes (improvement), and, in the linear setting, estimating the visible coefficients of the true causal model (causal precision). As our main contribution, we provide the first algorithms for learning accuracy-optimizing, improvement-optimizing, and causal-precision-optimizing linear regression models directly from data, without prior knowledge of agents' possible actions. These algorithms circumvent the hardness result of Miller et al. (2019) by allowing the decision maker to observe agents' responses to a sequence of decision rules, in effect inducing agents to perform causal interventions for free.
more »
« less
This content will become publicly available on January 1, 2025
Effect of Gamification on Gamers: Evaluating Interventions for Students Who Game the System
Gaming the system is a persistent problem in Computer-Based Learning Platforms. While substantialprogress has been made in identifying and understanding such behaviors, effective interventions remainscarce. This study uses a method of causal moderation known as Fully Latent Principal Stratification toexplore the impact of two types of interventions – gamification and manipulation of assistance access –on the learning outcomes of students who tend to game the system. The results indicate that gamificationdoes not consistently mitigate these negative behaviors. One gamified condition had a consistentlypositive effect on learning regardless of students’ propensity to game the system, whereas the other had anegative effect on gamers. However, delaying access to hints and feedback may have a positive effect onthe learning outcomes of those gaming the system. This paper also illustrates the potential for integratingdetection and causal methodologies within educational data mining to evaluate effective responses to detectedbehaviors.
more »
« less
- PAR ID:
- 10541410
- Publisher / Repository:
- Zenodo
- Date Published:
- Journal Name:
- Journal of educational data mining
- ISSN:
- 2157-2100
- Subject(s) / Keyword(s):
- gamification gaming the system causal inference computer-based learning platforms
- Format(s):
- Medium: X
- Right(s):
- Creative Commons Attribution 4.0 International
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract In this study, we investigated the validity of a stealth assessment of physics understanding in an educational game, as well as the effectiveness of different game‐level delivery methods and various in‐game supports on learning. Using a game called
Physics Playground , we randomly assigned 263 ninth‐ to eleventh‐grade students into four groups: adaptive, linear, free choice and no‐treatment control. Each condition had access to the same in‐game learning supports during gameplay. Results showed that: (a) the stealth assessment estimates of physics understanding were valid—significantly correlating with the external physics test scores; (b) there was no significant effect of game‐level delivery method on students' learning; and (c) physics animations were the most effective (among eight supports tested) in predicting both learning outcome and in‐game performance (e.g. number of game levels solved). We included student enjoyment, gender and ethnicity in our analyses as moderators to further investigate the research questions. -
more » « less
Abstract Computational thinking is acknowledged as an essential competency for everyone to learn. However, teachers find it challenging to implement the existing learning approaches in K-12 settings because the existing approaches often focus on teaching computing concepts and skills (i.e., programming skills) rather than on helping students develop their computational thinking competency—a competency that can be used across disciplinary boundaries in accordance with curriculum requirements. To address this need, the current study investigated how game-based learning influenced middle school students’ learning processes, particularly on the development of computational thinking competency, self-efficacy toward computational thinking, and engagement during gameplay. Additionally, the study examined how these outcomes were moderated by individual differences. We observed evidence that the gaming experience influenced students’ computational thinking self-efficacy, but not computational thinking competency or game-based engagement. Compared to age (grade) and prior gaming experience, gender tended to play a more important role in moderating students’ computational thinking competency, self-efficacy toward computational thinking competency, and game-based engagement. Implications and possible directions for future research regarding using game-based learning to enhance computational thinking competency are discussed.
-
more » « less
The emergence of reinforcement-based AI for text generation (Chat-GPT) and image creation (Dall-E) has opened a wide range of possibilities for changing the game design and development process. While game development researchers have mostly focused on integrating these technologies to improve production workflow and demonstrate their use in the creation of content for entertainment purposes (intelligent NPCS), there is very little knowledge on how to integrate this technology into the design of educational games. In this paper, we present the results of integrating reinforcement AI (text and image generation) into educational gaming experiences by graduate students enrolled in a game-based learning course. The students were given a core set of requirements that enable the integration into their project with some flexibility on the desired educational outcome. The produced experiences were then evaluated by a small sample of experts (gaming and learning sciences) and their observations were compiled. Specifically, we describe the wide range of experiences developed by the students and the results of a qualitative study with a small group of experts that evaluated these experiences. Our results indicate that reinforcement AI-based integrations into educational game design and development helps enrich the user experience and has the potential to improve learning outcomes.
-
Prior research has shown that digital games can enhance STEM education by providing learners with immersive and authentic scientific experiences. However, optimizing the learning outcomes of students engaged in game-based environments requires aligning the game design with diverse student needs. Therefore, an in-depth understanding of player behavior is crucial for identifying students who need additional support or modifications to the game design. This study applies an Ordered Network Analysis (ONA)—a specific kind of Epistemic Network Analysis (ENA)—to examine the game trace log data of student interactions, to gain insights into how learning gains relate to the different ways that students move through an open-ended virtual world for learning microbiology. Our findings reveal that differences between students with high and low learning gains are mediated by their prior knowledge. Specifically, level of prior knowledge is related to behaviors that resemble wheel-spinning, which warrant the development of future interventions. Results also have implications for discovery with modeling approaches and for enhancing in-game support for learners and improving game design.more » « less
