This content will become publicly available on June 29, 2027

Title: Pattern-based Knowledge Component Extraction from Student Code Using Representation Learning
Abstract: Personalized instruction aims to provide learners with support that adapts to their individual knowledge. Discovering and tracing Knowledge Components (KCs) is an important step in building accurate models of student learning. However, KC discovery in computer science education is challenging due to the open-ended nature of programming, wide variability in student solutions, and intertwined use of programming structures in code. We address these challenges with a pattern-based KC discovery method that uses a data-driven approach to define KCs as recurring structural patterns in student code that reveal persistent patterns of struggle and mastery in students' solutions. We then evaluate the discovered KCs using expert evaluation and statistical student modeling to demonstrate their effectiveness in capturing student learning and struggles. We propose a framework for modeling students' learning by deriving pattern-based KCs from student code through a three-stage process. First, an attention-based code representation model identifies Abstract Syntax Tree subtrees most relevant to code correctness. Second, a Variational Autoencoder abstracts these subtrees into a smooth latent space, capturing structural similarity across student submissions. Third, the resulting representations are clustered into pattern-based KCs. To assess the effectiveness of pattern-based KCs for modeling students' learning, we adapt the Deep Knowledge Tracing model to incorporate these KCs, demonstrating significant improvements in predictive performance over baseline KT methods. Additionally, the learning curve analysis showed alignment between the derived KCs and learning theory.  more » « less
Award ID(s):
2418655 2418658
PAR ID:
10710092
Author(s) / Creator(s):
; ; ; ; ; ;
Editor(s):
Botelho, Anthony; T Rodrigo, Maria Mercedes; Singla, Adish; Ogata, Hiroaki; So, Hyojeong; Cho, Young Hoan
Publisher / Repository:
19th International Conference on Educational Data Mining (EDM 2026), International Educational Data Mining Society
Date Published:
Page Range / eLocation ID:
217-228
Subject(s) / Keyword(s):
Location:
Seoul, Republic of Korea
Right(s):
Creative Commons Attribution 4.0 International
Sponsoring Org:
National Science Foundation
More Like this
  1. Feng, Mingyu; Käser, Tanja; Talukdar, Partha (Ed.)
    Knowledge components (KCs) have many applications. In computing education, knowing the demonstration of specific KCs has been challenging. This paper introduces an entirely data-driven approach for (i) discovering KCs and (ii) demonstrating KCs, using students' actual code submissions. Our system is based on two expected properties of KCs: (i) generate learning curves following the power law of practice, and (ii) are predictive of response correctness. We train a neural architecture (named KC-Finder) that classifies the correctness of student code submissions and captures problem-KC relationships. Our evaluation on data from 351 students in an introductory Java course shows that the learned KCs can generate reasonable learning curves and predict code submission correctness. At the same time, some KCs can be interpreted to identify programming skills. We compare the learning curves described by our model to four baselines, showing that (i) identifying KCs with naive methods is a difficult task and (ii) our learning curves exhibit a substantially better curve fit. Our work represents a first step in solving the data-driven KC discovery problem in computing education. 
    more » « less
  2. Knowledge tracing is a method to model students’ knowledge and enable personalized education in many STEM disciplines such as mathematics and physics, but has so far still been a challenging task in computing disciplines. One key obstacle to successful knowledge tracing in computing education lies in the accurate extraction of knowledge components (KCs), since multiple intertwined KCs are practiced at the same time for programming problems. In this paper, we address the limitations of current methods and explore a hybrid approach for KC extraction, which combines automated code parsing with an expert-built ontology. We use an introductory (CS1) Java benchmark dataset to compare its KC extraction performance with the traditional extraction methods using a state-of-the-art evaluation approach based on learning curves. Our preliminary results show considerable improvement over traditional methods of student modeling. The results indicate the opportunity to improve automated KC extraction in CS education by incorporating expert knowledge into the process. 
    more » « less
  3. Knowledge components (KCs) have many applications. In computing education, knowing the demonstration of specific KCs has been challenging. This paper introduces an entirely data-driven approach for (i) discovering KCs and (ii) demonstrating KCs, using students’ actual code submissions. Our system is based on two expected properties of KCs: (i) generate learning curves following the power law of practice, and (ii) are predictive of response correctness. We train a neural architecture (named KC-Finder) that classifies the correctness of student code submissions and captures problem-KC relationships. Our evaluation on data from 351 students in an introductory Java course shows that the learned KCs can generate reasonable learning curves and predict code submission correctness. At the same time, some KCs can be interpreted to identify programming skills. We compare the learning curves described by our model to four baselines, showing that (i) identifying KCs with naive methods is a difficult task and (ii) our learning curves exhibit a substantially better curve fit. Our work represents a first step in solving the data-driven KC discovery problem in computing education. 
    more » « less
  4. Botelho, Anthony; T Rodrigo, Maria Mercedes; Singla, Adish; Ogata, Hiroaki; So, Hyojeong; Cho, Young Hoan (Ed.)
    Abstract: Learning to program involves repeated trial-and-error attempts, leading to substantial heterogeneity in learning behavior across student cohorts and course offerings. Code-aware knowledge tracing (KT) models aim to capture how students' programming knowledge evolves over time. However, most existing KT studies rely on data from individual courses, where student interactions are often sparse and insufficient for modeling robust performance patterns. Centralizing student data across classrooms or institutions to address this limitation is often constrained by the risk of privacy breach of sensitive code submission data, limiting the impact of existing KT methods. Federated learning (FL) provides a privacy-preserving alternative by enabling collaborative model training without sharing raw data, and recent work has shown that model personalization can further improve federated KT performance. However, the interaction between model personalization and code representation has not been systematically studied. In this paper, we present a comparative analysis of code-aware KT representations under personalized federated learning (pFL) framework, using two semesters of CodeWorkout data treated as separate clients. Our results show that personalization consistently improves predictive accuracy under federated training, but the magnitude and consistency of these improvements depend on how student code is represented. In particular, code representations that capture richer structural information and syntax-aware encodings exhibit more reliable performance gains and promise more stable learning trajectories across assignments and semesters, especially for more complex programming tasks. These findings provide practical insights for EDM researchers and practitioners on how representation choice shapes the effectiveness of personalization when deploying privacy-preserving knowledge tracing models in programming education. 
    more » « less
  5. Evaluates DKT models’ ability to track individual knowledge components (KCs) in programming tasks. Proposes two enhancements—adding an explicit KC layer and code features—and shows that the KC layer yields modest improvements in KC-level interpretability, especially when tracking incorrect submissions. 
    more » « less