Search for: All records

Undergraduate STEM lecture courses enroll hundreds who must master declarative, conceptual, and applied learning objectives. To support them, instructors have turned to active learning designs that require students to engage inself-regulated learning(SRL). Undergraduates struggle with SRL, and universities provide courses, workshops, and digital training to scaffold SRL skill development and enactment. We examined two theory-aligned designs of digital skill trainings that scaffold SRL and how students’ demonstration of metacognitive knowledge of learning skills predicted exam performance in biology courses where training took place. In Study 1, students’ (n = 49) responses to training activities were scored for quality and summed by training topic and level of understanding. Behavioral and environmental regulation knowledge predicted midterm and final exam grades; knowledge of SRL processes did not. Declarative and conceptual levels of skill-mastery predicted exam performance; application-level knowledge did not. When modeled by topic at each level of understanding, declarative knowledge of behavioral and environmental regulation and conceptual knowledge of cognitive strategies predicted final exam performance. In Study 2 (n = 62), knowledge demonstrated during a redesigned video-based multimedia version of behavioral and environmental regulation again predicted biology exam performance. Across studies, performance on training activities designed in alignment with skill-training models predicted course performances and predictions were sustained in a redesign prioritizing learning efficiency. Training learners’ SRL skills –and specifically cognitive strategies and environmental regulation– benefited their later biology course performances across studies, which demonstrate the value of providing brief, digital activities to develop learning skills. Ongoing refinement to materials designed to develop metacognitive processing and learners’ ability to apply skills in new contexts can increase benefits.

 

The affordances of computer‐based learning environments make them powerful tools for conveying information in higher education. However, to most effectively use these environments, students must be adept at self‐regulating their learning. This self‐regulation is effortful, including a myriad of processes, including defining tasks, making plans, using and monitoring the efficacy of high‐quality learning strategies, and reflecting on the learning process and outcomes. Therefore, higher education instructors and course designers should design computer‐based learning environments to ease learning and free up mental resources for self‐regulation. This chapter describes how design principles from the cognitive theory of multimedia learning can facilitate learning in computer‐based learning environments and promote self‐regulated learning. Examples of the multimedia, personalization, and generative activity principles are presented to show how the cognitive theory of multimedia learning can guide design and promote students’ selection, organization, and integration of content, resulting in better understanding and more mental resources available for self‐regulated learning and the deeper learning it can afford. 

Undergraduate science, technology, engineering, and mathematics (STEM) students’ motivations have a strong influence on whether and how they will persist through challenging coursework and into STEM careers. Proper conceptualization and measurement of motivation constructs, such as students’ expectancies and per- ceptions of value and cost (i.e., expectancy value theory [EVT]) and their goals (i.e., achievement goal theory [AGT]), are necessary to understand and enhance STEM persistence and success. Research findings suggest the importance of exploring multiple measurement models for motivation constructs, including traditional con- firmatory factor analysis, exploratory structural equation models (ESEM), and bifactor models, but more research is needed to determine whether the same model fits best across time and context. As such, we mea- sured undergraduate biology students’ EVT and AGT motivations and investigated which measurement model best fit the data, and whether measurement invariance held, across three semesters. Having determined the best- fitting measurement model and type of invariance, we used scores from the best performing model to predict biology achievement. Measurement results indicated a bifactor-ESEM model had the best data-model fit for EVT and an ESEM model had the best data-model fit for AGT, with evidence of measurement invariance across semesters. Motivation factors, in particular attainment value and subjective task value, predicted small yet statistically significant amounts of variance in biology course outcomes each semester. Our findings provide support for using modern measurement models to capture students’ STEM motivations and potentially refine conceptualizations of them. Such future research will enhance educators’ ability to benevolently monitor and support students’ motivation, and enhance STEM performance and career success. 

Well-designed instructional videos are powerful tools for helping students learn and prompting students to use generative strategies while learning from videos further bolsters their effectiveness. However, little is known about how individual differences in motivational factors, such as achievement goals, relate to how students learn within multimedia environments that include instructional videos and generative strategies. Therefore, in this study, we explored how achievement goals predicted undergraduate students’ behaviors when learning with instructional videos that required students to answer practice questions between videos, as well as how those activities predicted subsequent unit exam performance one week later. Additionally, we tested the best measurement models for modeling achievement goals between traditional confirmatory factor analysis and bifactor confirmatory factor analysis. The bifactor model fit our data best and was used for all subsequent analyses. Results indicated that stronger mastery goal endorsement predicted performance on the practice questions in the multimedia learning environment, which in turn positively predicted unit exam performance. In addition, students’ time spent watching videos positively predicted practice question performance. Taken together, this research emphasizes the availing role of adaptive motivations, like mastery goals, in learning from instructional videos that prompt the use of generative learning strategies. 

Abstract Using traces of behaviors to predict outcomes is useful in varied contexts ranging from buyer behaviors to behaviors collected from smart-home devices. Increasingly, higher education systems have been using Learning Management System (LMS) digital data to capture and understand students’ learning and well-being. Researchers in the social sciences are increasingly interested in the potential of using digital log data to predict outcomes and design interventions. Using LMS data for predicting the likelihood of students’ success in for-credit college courses provides a useful example of how social scientists can use these techniques on a variety of data types. Here, we provide a primer on how LMS data can be feature-mapped and analyzed to accomplish these goals. We begin with a literature review summarizing current approaches to analyzing LMS data, then discuss ethical issues of privacy when using demographic data and equitable model building. In the second part of the paper, we provide an overview of popular machine learning algorithms and review analytic considerations such as feature generation, assessment of model performance, and sampling techniques. Finally, we conclude with an empirical example demonstrating the ability of LMS data to predict student success, summarizing important features and assessing model performance across different model specifications. 

Capturing evidence for dynamic changes in self‐regulated learning (SRL) behaviours resulting from interventions is challenging for researchers. In the current study, we identified students who were likely to do poorly in a biology course and those who were likely to do well. Then, we randomly assigned a portion of the students predicted to perform poorly to a science of learning to learn intervention where they were taught SRL study strategies. Learning outcome and log data (257 K events) were collected fromn = 226 students. We used a complex systems framework to model the differences in SRL including the amount, interrelatedness, density and regularity of engagement captured in digital trace data (ie, logs). Differences were compared between students who were predicted to (1) perform poorly (control,n = 48), (2) perform poorly and received intervention (treatment,n = 95) and (3) perform well (not flagged,n = 83). Results indicated that the regularity of students' engagement was predictive of course grade, and that the intervention group exhibited increased regularity in engagement over the control group immediately after the intervention and maintained that increase over the course of the semester. We discuss the implications of these findings in relation to the future of artificial intelligence and potential uses for monitoring student learning in online environments.

What is already known about this topic

Self‐regulated learning (SRL) knowledge and skills are strong predictors of postsecondary STEM student success.

SRL is a dynamic, temporal process that leads to purposeful student engagement.

Methods and metrics for measuring dynamic SRL behaviours in learning contexts are needed.

What this paper adds

A Markov process for measuring dynamic SRL processes using log data.

Evidence that dynamic, interaction‐dominant aspects of SRL predict student achievement.

Evidence that SRL processes can be meaningfully impacted through educational intervention.

Implications for theory and practice

Complexity approaches inform theory and measurement of dynamic SRL processes.

Static representations of dynamic SRL processes are promising learning analytics metrics.

Engineered features of LMS usage are valuable contributions to AI models.

 

Even highly motivated undergraduates drift off their STEM career pathways. In large introductory STEM classes, instructors struggle to identify and support these students. To address these issues, we developed co‐redesign methods in partnership with disciplinary experts to create high‐structure STEM courses that better support students and produce informative digital event data. To those data, we applied theory‐ and context‐relevant labels to reflect active and self‐regulated learning processes involving LMS‐hosted course materials, formative assessments, and help‐seeking tools. We illustrate the predictive benefits of this process across two cycles of model creation and reapplication. In cycle 1, we used theory‐relevant features from 3 weeks of data to inform a prediction model that accurately identified struggling students and sustained its accuracy when reapplied in future semesters. In cycle 2, we refit a model with temporally contextualized features that achieved superior accuracy using data from just two class meetings. This modelling approach can produce durable learning analytics solutions that afford scaled and sustained prediction and intervention opportunities that involve explainable artificial intelligence products. Those same products that inform prediction can also guide intervention approaches and inform future instructional design and delivery.

Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies.

Prediction models based on demographic data can perpetuate systemic biases.

Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self‐regulated learning processes within learning tasks.

Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication.

The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high‐structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory‐aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters.

Co‐design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling.

High‐structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable.

Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory‐informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors.

Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications.

 

Flourishing in today's global society requires citizens that are both intelligent consumers and producers of scientific understanding. Indeed, the modern world is facing ever‐more complex problems that require innovative ways of thinking about, around, and with science. As numerous educational stakeholders have suggested, such skills and abilities are not innate and must, therefore, be taught (e.g., McNeill & Krajcik,Journal of Research in Science Teaching,45(1), 53–78. 2008). However, such instruction requires a fundamental shift in science pedagogy so as to foster knowledge and practices like deep, conceptual understanding, model‐based reasoning, and oral and written argumentation where scientific evidence is evaluated (National Research Council,Next Generation Science Standards: For States, by States, Washington, DC: The National Academies Press, 2013). The purpose of our quasi‐experimental study was to examine the effectiveness of Quality Talk Science, a professional development model and intervention, in fostering changes in teachers’ and students’ discourse practices as well as their conceptual understanding and scientific argumentation. Findings revealed treatment teachers’ and students’ discourse practices better reflected critical‐analytic thinking and argumentation at posttest relative to comparison classrooms. Similarly, at posttest treatment students produced stronger written scientific arguments than comparison students. Students’ growth in conceptual understanding was nonsignificant. These findings suggest discourse interventions such as Quality Talk Science can improve high‐school students’ ability to engage in scientific argumentation.