An official website of the United States government
Many people are learning programming on their own using various online resources. Unfortunately, learners using these resources often be- come disengaged or even quit when encountering an obstacle they cannot overcome without additional help. Teachers in a classroom can provide this type of help, but this may be impractical or impossible to implement in online educational settings. To address this issue, we added a visually- oriented hint system into an existing online educational game designed to teach novices introductory programming concepts. We implemented three versions of the hint system, providing equivalent information for each level of the game, adjusting the amount of interactivity between versions. The first version consisted of a static image with text showing how to solve a level in a single panel. The second version included a series of images that allowing users to scroll through hints step-by-step. The final version showed a short video allowing users to play, pause, and seek through animated hint(s). In total, we had 150 people play the game, randomly assigned to one of these three versions of the hint system. We found that users had a strong preference for the video version of the hint system, completing more levels. Based on these findings, we propose suggestions for designers of online educational tools to better support their users.
more »
« less
Predicting abandonment in online coding tutorials
Learners regularly abandon online coding tutorials when they get bored or frustrated, but there are few techniques for anticipating this abandonment to intervene. In this paper, we examine the feasibility of predicting abandonment with machine-learned classifiers. Using interaction logs from an online programming game, we extracted a collection of features that are potentially related to learner abandonment and engagement, then developed classifiers for each level. Across the first five levels of the game, our classifiers successfully predicted 61% to 76% of learners who did not complete the next level, achieving an average AUC of 0.68. In these classifiers, features negatively associated with abandonment included account activation and help-seeking behaviors, whereas features positively associated with abandonment included features indicating difficulty and disengagement. These findings highlight the feasibility of providing timely intervention to learners likely to quit.
more »
« less
- Award ID(s):
- 1657160
- PAR ID:
- 10054903
- Date Published:
- Journal Name:
- IEEE Visual Languages and Human-Centric Computing
- Page Range / eLocation ID:
- 191 - 199
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
As more people turn to discretionary online tools to learn new skills such as computer programming, exploring how to better support a wide range of learners is becoming increasingly essential to train the next generation of highly skilled technology workers. In our prior work, users with high learner autonomy complained that most online resources they used to learn more programming did not provide them with the flexibility they preferred to navigate through learning materials, locking them into a set sequence of topics/concepts. To explore this, we implemented a level-jumping feature into an online educational programming game. We tested it with 350 new users, tracking their progress through the game for 7 days each. We found that those with high learner autonomy did use the level jumping feature more than those with low learner autonomy. We also found that males were more likely to use this new feature, regardless of learner autonomy level, compared to their female counterparts. Finally, we found that those with low learner autonomy ultimately completed more levels than their high autonomy counterparts, and that this was particularly true of female learners (who completed the most levels overall). Based on these findings, we believe that autonomous-supportive features such as flexible navigation may be beneficial to all users of online educational tools, and that encouraging its use by a wider group of users (particularly females), may increase positive effects.more » « less
-
Many people are learning programming on their own using various online resources such as educational games. Unfortunately, little is known about how to keep online educational game learners motivated throughout their game play, especially if they become disengaged or frustrated with their task. Keeping online learners engaged is essential for learning programming, as it may have lasting effects on their views and self-efficacy towards computer science. To address this issue, we created a coarse-grained frustration detector that provided users with customized, adaptive feedback to help (re)engage them with the game content. We ran a controlled experiment with 400 participants over the course of 1.5 months, with half of the players playing the original game, and the other half playing the game with the frustration detection and adaptive feed- back. We found that the users who received the adaptive feedback when frustrated completed more levels than their counterparts who did not receive this customized feedback. Based on these findings, we believe that adaptive feedback is essential in keeping educational game learners engaged, and propose future work for researchers and designers of online educational games to better support their users.more » « less
-
This paper demonstrated how to apply Machine Learning (ML) techniques to analyze student interaction data collected in an online mathematics game. Using a data-driven approach, we examined 1) how different ML algorithms influenced the precision of middle-school students’ (N = 359) performance (i.e. posttest math knowledge scores) prediction and 2) what types of in-game features (i.e. student in-game behaviors, math anxiety, mathematical strategies) were associated with student math knowledge scores. The results indicated that the Random Forest algorithm showed the best performance (i.e. the accuracy of models, error measures) in predicting posttest math knowledge scores among the seven algorithms employed. Out of 37 features included in the model, the validity of the students’ first mathematical transformation was the most predictive of their posttest math knowledge scores. Implications for game learning analytics and supporting students’ algebraic learning are discussed based on the findings.more » « less
-
Auer, M.E.; Pachatz, W.; Rüütmann, T. (Ed.)Gamification, the use of game design elements in non-game contexts, has become a promising strategy for enhancing learners’ motivation, engagement, and performance. However, our understanding of how the motivational affordances of gamification interact with the motivational drivers engendered by a learning activity is still limited. In most of the studies the focus is on the role of the incorporated gamification elements, disregarding motivational factors associated with the learning activity, such as perceived utility, expectancy of success, and needed effort to complete it. Expectancy-Value model offers a practical method for estimating the level and quality of learners’ motivation towards a particular task as it accounts for both intrinsic and extrinsic motivators. Employing this model can shed a new light on the motivational potential of educational gamification. Accordingly, in this paper we present experiments with Expectancy-Value-Cost scale (EVC) as an instrument for estimating the level of students’ motivation towards a gamified learning activity. We studied empirically how the motivational factors measured by EVC relate to the level of learners’ engagement in gamified practicing and assessed their predictive qualities.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/VLHCC.2017.8103467
