More Like this

1. Ready Raider One: Exploring the Misuse of Cloud Gaming Services

   https://doi.org/10.1145/3548606.3560647  

   Liu, Guannan; Liu, Daiping; Hao, Shuai; Gao, Xing; Sun, Kun; Wang, Haining (November 2022, Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security)
2. Customer Incentives for Gaming Demand Response Baselines

   https://doi.org/10.1109/CDC40024.2019.9029860  

   Ellman, Douglas; Xiao, Yuanzhang (December 2019, IEEE)
3. Evaluating Gaming Detector Model Robustness Over Time

   https://doi.org/10.5281/zenodo.6852961  

   Levin, Nathan; Baker, Ryan; Nasiar, Nidhi; Fancsali Stephen; Hutt, Stephen (January 2022, Proceedings of the 15th International Conference on Educational Data Mining, International Educational Data Mining Society)

   Research into "gaming the system" behavior in intelligent tutoring systems (ITS) has been around for almost two decades, and detection has been developed for many ITSs. Machine learning models can detect this behavior in both real-time and in historical data. However, intelligent tutoring system designs often change over time, in terms of the design of the student interface, assessment models, and data collection log schemas. Can gaming detectors still be trusted, a decade or more after they are developed? In this research, we evaluate the robustness/degradation of gaming detectors when trained on old data logs and evaluated on current data logs. We demonstrate that some machine learning models developed using past data are still able to predict gaming behavior from student data collected 16 years later, but that there is considerable variance in how well different algorithms perform over time. We demonstrate that a classic decision tree algorithm maintained its performance while more contemporary algorithms struggled to transfer to new data, even though they exhibited better performance on both new and old data alone. Examining the feature importances provides some explanation for the differences in performance between models, and offers some insight into how we might safeguard against detector rot over time. 

   more » « less
4. CodeTracesure—Combining Gaming, CS Concepts, and Pedagogy

   Nche, O; Welter, J; Che, S M; Kraemer, E T; Zordan, V (January 2019, ACM RESPECT 2019 Proceedings)

   This paper presents an overview of the objectives and design of a video game, CodeTracesure, that we have used in a summer camp for elementary school African-American children. The game is designed to help children practice CS concepts as they play and to help teachers learn about their difficulties. It combines engaging elements of good games with pedagogy to provide a platform where students can practice CS concepts learned in class. It covers such concepts as assignment, variables, sequencing, and operators. The game is equipped with a database to facilitate collection of data that can be analyzed for trends and patterns. The current goal is for it to be a supplementary tool that can help the students practice while allowing teachers to collect useful data that can help improve the learning process. An initial study was conducted using this game with about 40 African-American elementary school children. Findings show that the game was useful in motivating the students to practice code tracing and learn CS concepts. The backend end data that was collected on the performance of the students helped to identify potential pitfalls. 

   more » « less
5. Evaluating Gaming Detector Model Robustness Over Time.

   https://doi.org/10.5281/zenodo.6852962  

   Levin, N.; Baker, R.S.; Nasiar, N.; Fancsali, S.; Hutt, S. (January 2022, Proceedings of the 15th International Conference on Educational Data Mining)

   Research into "gaming the system" behavior in intelligent tutoring systems (ITS) has been around for almost two decades, and detection has been developed for many ITSs. Machine learning models can detect this behavior in both real-time and in historical data. However, intelligent tutoring system designs often change over time, in terms of the design of the student interface, assessment models, and data collection log schemas. Can gaming detectors still be trusted, a decade or more after they are developed? In this research, we evaluate the robustness/degradation of gaming detectors when trained on older data logs and evaluated on current data logs. We demonstrate that some machine learning models developed using past data are still able to predict gaming behavior from student data collected 16 years later, but that there is considerable variance in how well different algorithms perform over time. We demonstrate that a classic decision tree algorithm maintained its performance while more contemporary algorithms struggled to transfer to new data, even though they exhibited better performance on unseen students in both New and Old data sets by themselves. Examining the feature importance values provides some explanation for the differences in performance between models, and offers some insight into how we might safeguard against detector rot over time. 

   more » « less