An official website of the United States government
A longstanding goal of learner modeling and educational data min-ing is to improve the domain model of knowledge that is used to make inferences about learning and performance. In this report we present a tool for finding domain models that is built into an exist-ing modeling framework, logistic knowledge tracing (LKT). LKT allows the flexible specification of learner models in logistic re-gression by allowing the modeler to select whatever features of the data are relevant to prediction. Each of these features (such as the count of prior opportunities) is a function computed for a compo-nent of data (such as a student or knowledge component). In this context, we have developed the “autoKC” component, which clus-ters knowledge components and allows the modeler to compute features for the clustered components. For an autoKC, the input component (initial KC or item assignment) is clustered prior to computing the feature and the feature is a function of that cluster. Another recent new function for LKT, which allows us to specify interactions between the logistic regression predictor terms, is com-bined with autoKC for this report. Interactions allow us to move beyond just assuming the cluster information has additive effects to allow us to model situations where a second factor of the data mod-erates a first factor.
more »
« less
Logistic Knowledge Tracing Tutorial: Practical Educational Applications
Logistic Knowledge Tracing (LKT) is a framework for combining various predictive features into student models that are adaptive, interpretable, explainable, and accurate. While the name logistic knowledge tracing was coined for our R package that implements this methodology for making student models, logistic knowledge tracing originates with much older models such as Item Response Theory (IRT), the Additive Factors Model (AFM), and Perfor-mance Factors Analysis (PFA), which exemplify a type of model where student performance is represented by the sum of multiple components each with some sort of feature computed for the component. Features may range from the simple presence or ab-sence of the component to complex functions of the prior history of the component. The LKT package provides a simple interface to this methodology, allowing old models to be specified or new models to be created by mixing and matching components with features. We will provide concrete examples of how the LKT framework can provide interpretable results on real-world datasets while being highly accurate.
more »
« less
- Award ID(s):
- 2301130
- PAR ID:
- 10613530
- Editor(s):
- Benjamin, Paaßen; Carrie, Demmans Epp
- Publisher / Repository:
- International Educational Data Mining Society
- Date Published:
- Format(s):
- Medium: X
- Right(s):
- Creative Commons Attribution 4.0 International
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Facing the escalating effects of climate change, it is critical to improve the prediction and understanding of the hurricane evacuation decisions made by households in order to enhance emergency management. Current studies in this area often have relied on psychology-driven linear models, which frequently exhibited limitations in practice. The present study proposed a novel interpretable machine learning approach to predict household-level evacuation decisions by leveraging easily accessible demographic and resource-related predictors, compared to existing models that mainly rely on psychological factors. An enhanced logistic regression model (that is, an interpretable machine learning approach) was developed for accurate predictions by automatically accounting for nonlinearities and interactions (that is, univariate and bivariate threshold effects). Specifically, nonlinearity and interaction detection were enabled by low-depth decision trees, which offer transparent model structure and robustness. A survey dataset collected in the aftermath of Hurricanes Katrina and Rita, two of the most intense tropical storms of the last two decades, was employed to test the new methodology. The findings show that, when predicting the households’ evacuation decisions, the enhanced logistic regression model outperformed previous linear models in terms of both model fit and predictive capability. This outcome suggests that our proposed methodology could provide a new tool and framework for emergency management authorities to improve the prediction of evacuation traffic demands in a timely and accurate manner.more » « less
-
Benjamin, Paaßen; Carrie, Demmans Epp (Ed.)Recent advances in artificial intelligence for education leverage generative large language models, including using them to predict open-ended student responses rather than their correctness only. However, the black-box nature of these models limits the interpretability of the learned student knowledge representations. In this paper, we conduct a first exploration into interpreting latent student knowledge representations by presenting InfoOIRT, an Information regularized Open-ended Item Response Theory model, which encourages the latent student knowledge states to be interpretable while being able to generate student-written code for open-ended programming questions. InfoOIRT maximizes the mutual information between a fixed subset of latent knowledge states enforced with simple prior distributions and generated student code, which encourages the model to learn disentangled representations of salient syntactic and semantic code features including syntactic styles, mastery of programming skills, and code structures. Through experiments on a real-world programming education dataset, we show that InfoOIRT can both accurately generate student code and lead to interpretable student knowledge representations.more » « less
-
A substantial amount of research has been conducted by the educational data mining community to track and model learning. Previous work in modeling student knowledge has focused on predicting student performance at the problem level. While informative, problem-to-problem predictions leave little time for interventions within the system and relatively no time for human interventions. As such, modeling student performance at higher levels, such as by assignment, may provide a better opportunity to develop and apply learning interventions preemptively to remedy gaps in student knowledge. We aim to identify assignment-level features that predict whether or a not a student will finish their next homework assignment once started. We employ logistic regression models to test which features best predict whether a student will be a “starter” or a “finisher” on the next assignment.more » « less
-
Knowledge tracing is a popular and successful approach to modeling student learning. In this paper, we investigate whether the addition of neuroimaging observations to a knowledge tracing model enables accurate prediction of memory performance in held-out data. We propose a Hidden Markov Model of memory acquisition related to Bayesian Knowledge Tracing and show how continuous functional magnetic resonance imaging (fMRI) signals can be incorporated as observations related to latent knowledge states. We then show, using data collected from a simple second-language learning experiment, that fMRI data acquired during a learning session can be used to improve predictions about student memory at test. The fitted models can also potentially give new insight into the neural mechanisms that contribute to learning and memory.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.5281/zenodo.12730033
