skip to main content


Title: Using Sequence Mining to Analyze Metacognitive Monitoring and Scientific Inquiry based on Levels of Efficiency and Emotions during Game-Based Learning
Self-regulated learning conducted through metacognitive monitoring and scientific inquiry can be influenced by many factors, such as emotions and motivation, and are necessary skills needed to engage in efficient hypothesis testing during game-based learning. Although many studies have investigated metacognitive monitoring and scientific inquiry skills during game-based learning, few studies have investigated how the sequence of behaviors involved during hypothesis testing with game-based learning differ based on both efficiency level and emotions during gameplay. For this study, we analyzed 59 undergraduate students’ (59% female) metacognitive monitoring and hypothesis testing behavior during learning and gameplay with CRYSTAL ISLAND, a game-based learning environment that teaches students about microbiology. Specifically, we used sequential pattern mining and differential sequence mining to determine if there were sequences of hypothesis testing behaviors and to determine if the frequencies of occurrence of these sequences differed between high or low levels of efficiency at finishing the game and high or low levels of facial expressions of emotions during gameplay. Results revealed that students with low levels of efficiency and high levels of facial expressions of emotions had the most sequences of testing behaviors overall, specifically engaging in more sequences that were indicative of less strategic hypothesis testing behavior than the other students, where students who were more efficient with both levels of emotions demonstrated strategic testing behavior. These results have implications for the strengths of using educational data mining techniques for determining the processes underlying patterns of engaging in self-regulated learning conducted through hypothesis testing as they unfold over time; for training students on how to engage in the self-regulation, scientific inquiry, and emotion regulation processes that can result in efficient gameplay; and for developing adaptive game-based learning environments that foster effective and efficient self-regulation and scientific inquiry during learning.  more » « less
Award ID(s):
1661202
NSF-PAR ID:
10091349
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Journal of educational data mining
Volume:
10
Issue:
3
ISSN:
2157-2100
Page Range / eLocation ID:
1-26
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The goal of this study was to assess how metacognitive monitoring and scientific reasoning impacted the efficiency of game completion during learning with Crystal Island, a game-based learning environment that fosters self-regulated learning and scientific reasoning by having participants solve the mystery of what illness impacted inhabitants of the island. We conducted sequential pattern mining and differential sequence mining on 64 undergraduate participants’ hypothesis testing behavior. Patterns were coded based on the relevancy of what items were being tested for, and the items themselves. Results revealed that participants who were more efficient at solving the mystery tested significantly fewer partially-relevant and irrelevant items than less efficient participants. Additionally, more efficient participants had fewer sequences of testing items overall, and significantly lower instance support values of the PartiallyRelevant--Relevant to Relevant--Relevant and PartiallyRelevant--PartiallyRelevant to Relevant--Partially Relevant sequences compared to less efficient participants. These findings have implications for designing adaptive GBLEs that scaffold participants based on in-game behaviors. 
    more » « less
  2. Abstract Background

    Goal setting has been established in the literature as a critical component of self‐regulated learning and for effective problem‐solving. Yet, surprisingly little attention has been focused on goal‐directed behaviour in digital game‐based learning environments (GBLEs) despite their expanding use in educational contexts.

    Objectives

    The current study examined the impact of goal assignment in a GBLE aligned with curricular objectives and played in a classroom context.

    Methods

    Eighth‐grade students (N = 59) in the experiment were randomly assigned to either a mastery goal condition or a performance goal condition before playingCrystal Island—Outbreak. Goal orientation was measured before gameplay, monitoring judgements assessed during gameplay, and enjoyment was measured after game play.

    Results and Conclusions

    Two important results emerged that contribute to the existing literature on goals and GBLEs. First, assignment to the mastery goal condition resulted in significantly higher enjoyment ratings and monitoring judgements but not game performance while considering the influence of pre‐existing goal‐orientation. Second, self‐reported mastery goal orientation and videogame play frequency predicted enjoyment in the game.

    Implications

    The results suggest that enjoyment can be facilitated by orienting students to adopt mastery goals and that developers should consider integrating mastery‐based instructions in their GBLEs. Moreover, instructional goals can influence students regardless of incoming goal orientations.

     
    more » « less
  3. Abstract Background

    Providing adaptive scaffolds to help learners develop effective self‐regulated learning (SRL) behaviours has been an important goal for intelligent learning environments. Adaptive scaffolding is especially important in open‐ended learning environments (OELE), where novice learners often face difficulties in completing their learning tasks.

    Objectives

    This paper presents a systematic framework for adaptive scaffolding in Betty's Brain, a learning‐by‐teaching OELE for middle school science, where students construct a causal model to teach a virtual agent, generically named Betty. We evaluate the adaptive scaffolding framework and discuss its implications on the development of more effective scaffolds for SRL in OELEs.

    Methods

    We detect key cognitive/metacognitiveinflection points, that is, moments where students' behaviours and performance change during learning, often suggesting an inability to apply effective learning strategies. At inflection points, Mr. Davis (a mentor agent in Betty's Brain) or Betty (the teachable agent) provides context‐specific conversational feedback, focusing on strategies to help the student become a more productive learner, or encouragement to support positive emotions. We conduct a classroom study with 98 middle schoolers to analyse the impact of adaptive scaffolds on students' learning behaviours and performance. We analyse how students with differential pre‐to‐post learning outcomes receive and use the scaffolds to support their subsequent learning process in Betty's Brain.

    Results and Conclusions

    Adaptive scaffolding produced mixed results, with some scaffolds (viz., strategic hints that supported debugging and assessment of causal models) being generally more useful to students than others (viz., encouragement prompts). Additionally, there were differences in how students with high versus low learning outcomes responded to some hints, as suggested by the differences in their learning behaviours and performance in the intervals after scaffolding. Overall, our findings suggest how adaptive scaffolding in OELEs like Betty's Brain can be further improved to better support SRL behaviours and narrow the learning outcomes gap between high and low performing students.

    Implications

    This paper contributes to our understanding and impact of adaptive scaffolding in OELEs. The results of our study indicate that successful scaffolding has to combine context‐sensitive inflection points with conversational feedback that is tailored to the students' current proficiency levels and needs. Also, our conceptual framework can be used to design adaptive scaffolds that help students develop and apply SRL behaviours in other computer‐based learning environments.

     
    more » « less
  4. Telehealth technologies play a vital role in delivering quality healthcare to patients regardless of geographic location and health status. Use of telehealth peripherals allow providers a more accurate method of collecting health assessment data from the patient and delivering a more confident and accurate diagnosis, saving not only time and money but creating positive patient outcomes. Advanced Practice Nursing (APN) students should be confident in their ability to diagnose and treat patients through a virtual environment. This pilot simulation was completed to help examine how APN students interacted in a simulation-based education (SBE) experience with and without peripherals, funded by the National Science Foundation’s Future of Work at the Human-Technology Frontier (FW-HTF) program. The SBE experience was created and deployed using the INACSL Healthcare Simulation Standards of Best PracticesTM and vetted by a simulation expert. APN students (N = 24), in their first assessment course, were randomly selected to be either a patient (n = 12) or provider (n = 12) in a telehealth simulation. Student dyads (patient/provider) were randomly placed to complete a scenario with (n = 6 dyads) or without (n = 6 dyads) the use of a peripheral. Students (providers and patients) who completed the SBE experience had an increased confidence level both with and without the use of peripherals. Students evaluated the simulation via the Simulation Effectiveness Tool-Modified (SET-M), and scored their perception of the simulation on a 1 to 5 point Likert Scale. The highest scoring areas were perceived support of learning by the faculty (M=4.6), feeling challenged in decision-making skills (M=4.4), and a better understanding of didactic material (M=4.3). The lowest scoring area was feeling more confident in decision making (M=3.9). We also recorded students’ facial expressions during the task to determine a probability score (0- 100) for expressed basic emotions, and results revealed that students had the highest scores for joy (M = 8.47) and surprise (M = 4.34), followed by disgust (M = 1.43), fear (M = .76), and contempt (M = .64); and had the lowest scores of anger (M = .44) and sadness (M = .36). Students were also asked to complete a reflection assignment as part of the SBE experience. Students reported feeling nervous at the beginning of the SBE experience, but acknowledged feeling better as the SBE experience unfolded. Based on findings from this pilot study, implications point towards the effectiveness of including simulations for nurse practitioner students to increase their confidence in performing telehealth visits and engaging in decision making. For the students, understanding that patients may be just as nervous during telehealth visits was one of the main takeaways from the experience, as well as remembering to reassure the patient and how to ask the patient to work the telehealth equipment. Therefore, providing students opportunities to practice these skills will help increase their confidence, boost their self- and emotion regulation, and improve their decision-making skills in telehealth scenarios. 
    more » « less
  5. Introduction

    Self-regulated learning (SRL), or learners’ ability to monitor and change their own cognitive, affective, metacognitive, and motivational processes, encompasses several operations that should be deployed during learning including Searching, Monitoring, Assembling, Rehearsing, and Translating (SMART). Scaffolds are needed within GBLEs to both increase learning outcomes and promote the accurate and efficient use of SRL SMART operations. This study aims to examine how restricted agency (i.e., control over one’s actions) can be used to scaffold learners’ SMART operations as they learn about microbiology with Crystal Island, a game-based learning environment.

    Methods

    Undergraduate students (N = 94) were randomly assigned to one of two conditions: (1) Full Agency, where participants were able to make their own decisions about which actions they could take; and (2) Partial Agency, where participants were required to follow a pre-defined path that dictated the order in which buildings were visited, restricting one’s control. As participants played Crystal Island, participants’ multimodal data (i.e., log files, eye tracking) were collected to identify instances where participants deployed SMART operations.

    Results

    Results from this study support restricted agency as a successful scaffold of both learning outcomes and SRL SMART operations, where learners who were scaffolded demonstrated more efficient and accurate use of SMART operations.

    Discussion

    This study provides implications for future scaffolds to better support SRL SMART operations during learning and discussions for future directions for future studies scaffolding SRL during game-based learning.

     
    more » « less