skip to main content

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 11:00 PM ET on Thursday, February 13 until 2:00 AM ET on Friday, February 14 due to maintenance. We apologize for the inconvenience.


Title: A learning analytics approach towards understanding collaborative inquiry in a problem‐based learning environment
Abstract   more » « less
PAR ID:
10444980
Author(s) / Creator(s):
 ;  ;  ;  ;  ;  
Publisher / Repository:
Wiley-Blackwell
Date Published:
Journal Name:
British Journal of Educational Technology
Volume:
53
Issue:
5
ISSN:
0007-1013
Page Range / eLocation ID:
p. 1321-1342
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract  
    more » « less
  2. Abstract  
    more » « less
  3. Abstract

    Capturing evidence for dynamic changes in self‐regulated learning (SRL) behaviours resulting from interventions is challenging for researchers. In the current study, we identified students who were likely to do poorly in a biology course and those who were likely to do well. Then, we randomly assigned a portion of the students predicted to perform poorly to a science of learning to learn intervention where they were taught SRL study strategies. Learning outcome and log data (257 K events) were collected fromn = 226 students. We used a complex systems framework to model the differences in SRL including the amount, interrelatedness, density and regularity of engagement captured in digital trace data (ie, logs). Differences were compared between students who were predicted to (1) perform poorly (control,n = 48), (2) perform poorly and received intervention (treatment,n = 95) and (3) perform well (not flagged,n = 83). Results indicated that the regularity of students' engagement was predictive of course grade, and that the intervention group exhibited increased regularity in engagement over the control group immediately after the intervention and maintained that increase over the course of the semester. We discuss the implications of these findings in relation to the future of artificial intelligence and potential uses for monitoring student learning in online environments.

    Practitioner notes

    What is already known about this topic

    Self‐regulated learning (SRL) knowledge and skills are strong predictors of postsecondary STEM student success.

    SRL is a dynamic, temporal process that leads to purposeful student engagement.

    Methods and metrics for measuring dynamic SRL behaviours in learning contexts are needed.

    What this paper adds

    A Markov process for measuring dynamic SRL processes using log data.

    Evidence that dynamic, interaction‐dominant aspects of SRL predict student achievement.

    Evidence that SRL processes can be meaningfully impacted through educational intervention.

    Implications for theory and practice

    Complexity approaches inform theory and measurement of dynamic SRL processes.

    Static representations of dynamic SRL processes are promising learning analytics metrics.

    Engineered features of LMS usage are valuable contributions to AI models.

     
    more » « less
  4. Abstract

    Collaborative inquiry learning affords educators a context within which to support understanding of scientific practices, disciplinary core ideas, and crosscutting concepts. One approach to supporting collaborative science inquiry is through problem‐based learning (PBL). However, there are two key challenges in scaffolding collaborative inquiry learning in technology rich environments. First, it is unclear how we might understand the impact of scaffolds that address multiple functions (e.g., to support inquiry and argumentation). Second, scaffolds take different forms, further complicating how to coordinate the forms and functions of scaffolds to support effective collaborative inquiry. To address these issues, we identify two functions that needed to be scaffolded, the PBL inquiry cycle and accountable talk. We then designed predefined hard scaffolds and just‐in‐time soft scaffolds that target the regulation of collaborative inquiry processes and accountable talk. Drawing on a mixed method approach, we examine how middle school students from a rural school engaged with Crystal Island: EcoJourneys for two weeks (N=45). Findings indicate that hard scaffolds targeting the PBL inquiry process and soft scaffolds that targeted accountable talk fostered engagement in these processes. Although the one‐to‐one mapping between form and function generated positive results, additional soft scaffolds were also needed for effective engagement in collaborative inquiry and that these soft scaffolds were often contingent on hard scaffolds. Our findings have implications for how we might design the form of scaffolds across multiple functions in game‐based learning environments.

     
    more » « less
  5. Abstract

    Even highly motivated undergraduates drift off their STEM career pathways. In large introductory STEM classes, instructors struggle to identify and support these students. To address these issues, we developed co‐redesign methods in partnership with disciplinary experts to create high‐structure STEM courses that better support students and produce informative digital event data. To those data, we applied theory‐ and context‐relevant labels to reflect active and self‐regulated learning processes involving LMS‐hosted course materials, formative assessments, and help‐seeking tools. We illustrate the predictive benefits of this process across two cycles of model creation and reapplication. In cycle 1, we used theory‐relevant features from 3 weeks of data to inform a prediction model that accurately identified struggling students and sustained its accuracy when reapplied in future semesters. In cycle 2, we refit a model with temporally contextualized features that achieved superior accuracy using data from just two class meetings. This modelling approach can produce durable learning analytics solutions that afford scaled and sustained prediction and intervention opportunities that involve explainable artificial intelligence products. Those same products that inform prediction can also guide intervention approaches and inform future instructional design and delivery.Practitioner notesWhat is already known about this topic

    Learning analytics includes an evolving collection of methods for tracing and understanding student learning through their engagements with learning technologies.

    Prediction models based on demographic data can perpetuate systemic biases.

    Prediction models based on behavioural event data can produce accurate predictions of academic success, and validation efforts can enrich those data to reflect students' self‐regulated learning processes within learning tasks.

    What this paper adds

    Learning analytics can be successfully applied to predict performance in an authentic postsecondary STEM context, and the use of context and theory as guides for feature engineering can ensure sustained predictive accuracy upon reapplication.

    The consistent types of learning resources and cyclical nature of their provisioning from lesson to lesson are hallmarks of high‐structure active learning designs that are known to benefit learners. These designs also provide opportunities for observing and modelling contextually grounded, theory‐aligned and temporally positioned learning events that informed prediction models that accurately classified students upon initial and later reapplications in subsequent semesters.

    Co‐design relationships where researchers and instructors work together toward pedagogical implementation and course instrumentation are essential to developing unique insights for feature engineering and producing explainable artificial intelligence approaches to predictive modelling.

    Implications for practice and/or policy

    High‐structure course designs can scaffold student engagement with course materials to make learning more effective and products of feature engineering more explainable.

    Learning analytics initiatives can avoid perpetuation of systemic biases when methods prioritize theory‐informed behavioural data that reflect learning processes, sensitivity to instructional context and development of explainable predictors of success rather than relying on students' demographic characteristics as predictors.

    Prioritizing behaviours as predictors improves explainability in ways that can inform the redesign of courses and design of learning supports, which further informs the refinement of learning theories and their applications.

     
    more » « less