An official website of the United States government
Undergraduates enrolled in large, active learning courses must self-regulate their learning (self-regulated learning [SRL]) by appraising tasks, making plans, setting goals, and enacting and monitoring strategies. SRL researchers have relied on self-report and learner-mediated methods during academic tasks studied in laboratories and now collect digital event data when learners engage with technology-based tools in classrooms. Inferring SRL processes from digital events and testing their validity is challenging. We aligned digital and verbal SRL event data to validate digital events as traces of SRL and used them to predict achievement in lab and course settings. In Study 1, we sampled a learning task from a biology course into a laboratory setting. Enrolled students (N = 48) completed the lesson using digital resources (e.g., online textbook, course site) while thinking aloud weeks before it was taught in class. Analyses confirmed that 10 digital events reliably co-occurred ≥70% of the time with verbalized task definition and strategy use macroprocesses. Some digital events co-occurred with multiple verbalized SRL macroprocesses. Variance in occurrence of validated digital events was limited in lab sessions, and they explained statistically nonsignificant variance in learners’ performance on lesson quizzes. In Study 2, lesson-specific digital event data from learners (N = 307) enrolled in the course (but not in Study 1) predicted performance on lesson-specific exam items, final exams, and course grades. Validated digital events also predicted final exam and course grades in the next semester (N = 432). Digital events can be validated to reflect SRL processes and scaled to explain achievement in naturalistic undergraduate education settings. Educational Impact and Implications Statement Instructors often have difficulty identifying and helping struggling students in courses with hundreds of students. Digital trace data can be used to efficiently and effectively identify struggling students in these large courses, but such data are often difficult to interpret with confidence. In our study, we found that using verbal trace data to augment and validate our inferences about the meaning of digital trace data resulted in a powerful set of predictors of students’ achievement. These validated digital trace data can be used to not only identify students in need of support in large classes, but also to understand how to target interventions to the aspects of learning that are causing students the most difficulty.
more »
« less
Tracking Changes in Students’ Online Self-Regulated Learning Behaviors and Achievement Goals Using Trace Clustering and Process Mining
Success in online and blended courses requires engaging in self-regulated learning (SRL), especially for challenging STEM disciplines, such as physics. This involves students planning how they will navigate course assignments and activities, setting goals for completion, monitoring their progress and content understanding, and reflecting on how they completed each assignment. Based on Winne & Hadwin’s COPES model, SRL is a series of events that temporally unfold during learning, impacted by changing internal and external factors, such as goal orientation and content difficulty. Thus, as goal orientation and content difficulty change throughout a course, so might students’ use of SRL processes. This paper studies how students’ SRL behavior and achievement goal orientation change over time in a large ( N = 250) college introductory level physics course taught online. Students’ achievement goal orientation was measured by repeated administration of the achievement goals questionnaire-revised (AGQ-R). Students’ SRL behavior was measured by analyzing their clickstream event traces interacting with online learning modules via a combination of trace clustering and process mining. Event traces were first divided into groups similar in nature using agglomerative clustering, with similarity between traces determined based on a set of derived characteristics most reflective of students’ SRL processes. We then generated causal nets for each cluster of traces via process mining and interpreted the underlying behavior and strategy of each causal net according to the COPES SRL framework. We then measured the frequency at which students adopted each causal net and assessed whether the adoption of different causal nets was associated with responses to the AGQ-R. By repeating the analysis for three sets of online learning modules assigned at the beginning, middle, and end of the semester, we examined how the frequency of each causal net changed over time, and how the change correlated with changes to the AGQ-R responses. Results have implications for measuring the temporal nature of SRL during online learning, as well as the factors impacting the use of SRL processes in an online physics course. Results also provide guidance for developing online instructional materials that foster effective SRL for students with different motivational profiles.
more »
« less
- Award ID(s):
- 1845436
- PAR ID:
- 10404015
- Date Published:
- Journal Name:
- Frontiers in Psychology
- Volume:
- 13
- ISSN:
- 1664-1078
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Self-regulated learning (SRL) is an essential factor in academic success. Self-regulated learning is a process where learners set clear goals, monitor progress toward attainment of those goals, and adapt their strategies to improve their learning. Because SRL is often not explicitly integrated into the classroom, students struggle to identify and use learning techniques empirically proven to be more successful than others. SRL is a learned skill students can develop over time that has been found to be related to high achievement and self-efficacy. This paper examines the effects of introducing SRL strategies into an undergraduate introductory physics classroom. The degree to which the students were self-regulated learners was correlated with their test averages (r = 0.23, p < 0.05). Students reported that they found the SRL instruction helpful (3.5 out of 5.0 on a 5-point scale) and 86% of the students felt the time spent on the instruction was generally appropriate. Students’ preferred study methods changed over the course of the semester, indicating that students applied SRL by adapting their learning processes based on which methods were most effective in helping them study for an upcoming exam and opting not to use techniques no longer perceived as useful. Higher achieving students were more likely to settle on highly effective techniques by the end of the semester, while lower achieving students continued to modify their learning processes.more » « less
-
Abstract Capturing evidence for dynamic changes in self‐regulated learning (SRL) behaviours resulting from interventions is challenging for researchers. In the current study, we identified students who were likely to do poorly in a biology course and those who were likely to do well. Then, we randomly assigned a portion of the students predicted to perform poorly to a science of learning to learn intervention where they were taught SRL study strategies. Learning outcome and log data (257 K events) were collected fromn = 226 students. We used a complex systems framework to model the differences in SRL including the amount, interrelatedness, density and regularity of engagement captured in digital trace data (ie, logs). Differences were compared between students who were predicted to (1) perform poorly (control,n = 48), (2) perform poorly and received intervention (treatment,n = 95) and (3) perform well (not flagged,n = 83). Results indicated that the regularity of students' engagement was predictive of course grade, and that the intervention group exhibited increased regularity in engagement over the control group immediately after the intervention and maintained that increase over the course of the semester. We discuss the implications of these findings in relation to the future of artificial intelligence and potential uses for monitoring student learning in online environments. Practitioner notesWhat is already known about this topicSelf‐regulated learning (SRL) knowledge and skills are strong predictors of postsecondary STEM student success.SRL is a dynamic, temporal process that leads to purposeful student engagement.Methods and metrics for measuring dynamic SRL behaviours in learning contexts are needed.What this paper addsA Markov process for measuring dynamic SRL processes using log data.Evidence that dynamic, interaction‐dominant aspects of SRL predict student achievement.Evidence that SRL processes can be meaningfully impacted through educational intervention.Implications for theory and practiceComplexity approaches inform theory and measurement of dynamic SRL processes.Static representations of dynamic SRL processes are promising learning analytics metrics.Engineered features of LMS usage are valuable contributions to AI models.more » « less
-
This study investigates patterns in students' learning and problem-solving behavior as they proceed through a sequence of 10 mastery-based online learning modules and how these patterns correlate with overall course outcome. Students' interaction with each module, as measured by analyzing the platform log data, was categorized into nine different states. The student population was divided into top, middle and bottom cohorts based on their total course credit, and we visualized each cohort's distribution among the nine states over the 10 modules using a series of parallel coordinates graphs. We found that the patterns of interaction were mostly similar on the first six modules, but are significantly different on modules 7-10. For the later modules, the top cohort mostly concentrated on the state corresponding to high problem-solving effort after learning, while the majority of the bottom cohort did not access the learning materials after multiple failed attempts.more » « less
-
null (Ed.)The Achievement Goal Framework describes students’ goal orientations as: task-based, focusing on the successful completion of the task; self-based, evaluating performance relative to one's own past performance; or other-based, evaluating performance relative to the performance of others. Goal orientations have been used to explain student success in a range of educational settings, but have not been used in post-secondary chemistry. This study describes the goal orientations of General Chemistry students and explores the relationship of goal orientations to success in the course. On average, students report higher task and self orientations than other orientation. Task orientation had a positive relationship with exam performance and self orientation had a negative relationship with exam performance. Clustering students showed that for the majority of students task and self orientations moved concurrently and students with low preference across the three orientations also performed lowest on exams. Finally, students in classes using Flipped-Peer Led Team Learning, a pedagogy designed to bring active learning to a large lecture class, showed higher task orientation than those in classes with lecture-based instruction.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3389/fpsyg.2022.813514
