Search for: All records

Early forecasting of student performance in a course is a critical component of building effective intervention systems. However, when the available student data is limited, accurate early forecasting is challenging. We present a language generation transfer learning approach that leverages the general knowledge of pre-trained language models to address this challenge. We hypothesize that early forecasting can be significantly improved by fine-tuning large language models (LLMs) via personalization and contextualization using data on students' distal factors (academic and socioeconomic) and proximal non-cognitive factors (e.g., motivation and engagement), respectively. Results obtained from extensive experimentation validate this hypothesis and thereby demonstrate the prowess of personalization and contextualization for tapping into the general knowledge of pre-trained LLMs for solving the downstream task of early forecasting. 

In this paper, we present a framework for quantifying the impact of interventions on the full trajectories of students' experiences. The interventions are given periodically based on student performance forecasting from an artificial intelligence (AI) model. We performed a small-scale randomized controlled trial for evaluating the impact of the AI-based intervention system on the undergraduate students of a science, technology, engineering, and mathematics (STEM) course. Intervention messaging content was based on machine learning forecasting models trained on data collected from the students in the same course over the preceding 3 years. Trial results show that the intervention produced a statistically significant increase in the proportion of students that achieved a passing grade. By applying the trajectory-analysis framework we find that the intervention impacts the stories of some types of students more than others, and use this to define new ways of identifying students who are most likely to benefit. Together these outcomes point to the potential and promise of just-in-time interventions for STEM learning and the need for larger fully-powered randomized controlled trials. 

We present results from a small-scale randomized controlled trial that evaluates the impact of just-in-time interventions on the academic outcomes of N = 65 undergraduate students in a STEM course. Intervention messaging content was based on machine learning forecasting models of data collected from 537 students in the same course over the preceding 3 years. Trial results show that the intervention produced a statistically significant increase in the proportion of students that achieved a passing grade. The outcomes point to the potential and promise of just-in-time interventions for STEM learning and the need for larger fully-powered randomized controlled trials.