Search for: All records

Misconception catalogs in CS education describe how novice programmers typically go wrong. Prior catalogs are descriptive registries: useful for instructors, but not directly actionable as input to automated assessment generation. We present a master catalog of programming misconceptions --- a Python-specific synthesis of 85 entries across 16 categories, drawn from peer-reviewed catalogs, CS1 textbooks, and the Python language specification --- augmented with operational metadata that specifies, for each entry, how the misconception manifests as a code-level change (modification, structural, or conceptual). We use this catalog to drive an LLM distractor-generation pipeline and evaluate the resulting MCQs against a catalog-free baseline using a blinded cross-model rating procedure. Both conditions use identical single-invoke generation architecture so the experimental variable is solely catalog-vs.-no-catalog. We report full misconception-fidelity rates across two LLMs (one commercial, one open-weight) and report whether the catalog effect holds across both raters 

Knowledge component annotation is important for adaptive learning systems, such as programming practice recommenders, but manually annotating programming exercises with fine-grained concepts is costly and difficult to scale. Large language models (LLMs) offer a promising alternative, yet open-ended LLM annotation can produce unstable labels that are difficult to align with existing curriculum models. This paper studies ontology-guided KC extraction as a human-AI collaboration task: human experts define a domain’s conceptual structure, such as the Python ontology explored in our work, while LLMs perform scalable exercise-level annotation within that fixed label space. Our work has been motivated by LLM-based framework to annotate programming exercises with ontology concepts through prompt windowing and aggregation. We attempted to re-produce the same approach and explore its performance on a much broader range of LLM deployment classes: Compact Generalist API, MoE Generalist API, Efficient Generalist API, Open-Weight Generalist API, Small Generalist Local, and Code Specialist Local. We compare annotation quality delivered by these models against expert labels using precision, recall, F1, Jaccard similarity, and average prediction density. The results show that the ontology-guided LLM-based KC extraction can achieve strong expert agreement, but its calibration varies substantially between model classes. 

Abstract: Personalized instruction aims to provide learners with support that adapts to their individual knowledge. Discovering and tracing Knowledge Components (KCs) is an important step in building accurate models of student learning. However, KC discovery in computer science education is challenging due to the open-ended nature of programming, wide variability in student solutions, and intertwined use of programming structures in code. We address these challenges with a pattern-based KC discovery method that uses a data-driven approach to define KCs as recurring structural patterns in student code that reveal persistent patterns of struggle and mastery in students' solutions. We then evaluate the discovered KCs using expert evaluation and statistical student modeling to demonstrate their effectiveness in capturing student learning and struggles. We propose a framework for modeling students' learning by deriving pattern-based KCs from student code through a three-stage process. First, an attention-based code representation model identifies Abstract Syntax Tree subtrees most relevant to code correctness. Second, a Variational Autoencoder abstracts these subtrees into a smooth latent space, capturing structural similarity across student submissions. Third, the resulting representations are clustered into pattern-based KCs. To assess the effectiveness of pattern-based KCs for modeling students' learning, we adapt the Deep Knowledge Tracing model to incorporate these KCs, demonstrating significant improvements in predictive performance over baseline KT methods. Additionally, the learning curve analysis showed alignment between the derived KCs and learning theory. 

This paper is a case-study on integrating Smart Learning Content (SLC) into a project-based undergraduate Python programming course delivered through an online learning platform to community college students. We showcase how practice-oriented SLC can be integrated in a project-based course and examine the connection between SLC usage and student learning outcomes. There are large publicly available repositories of SLC materials, yet their integration into a specific course and evaluation of its success are often not straightforward. By providing access to several SLC types, such as program construction examples, code animations, and Parson's puzzles, we aimed to bridge the gap between static conceptual reading and programming projects already present in the course. To evaluate the integration of SLC, we released the augmented course in the Spring 2025 semester to 252 students at 20 community colleges in the US and analyzed the interaction logs collected. 

Worked examples are step-by-step solutions to problems in a specific domain, offered to students to acquire domain-specific problem-solving skills. The power of worked examples could be magnified by combining them with self-explanations, which ask students to explain rather than passively study each problem-solving step. The main challenge of this approach is assessing the correctness of the student's explanations. In the current approach, student explanations are judged by their semantic similarity to an explanation provided by an instructor or domain expert. However, recent studies of example explanations in the domain of programming demonstrated that many students express themselves very differently from domain experts. In this situation, a traditional semantic similarity approach might introduce bias against students who correctly explain worked examples but are considerably different from expert explanations. In this paper, we use a recently published dataset to compare several explanation-assessment approaches based on semantic similarity with alternative approaches based on direct Large Language Model (LLM) prompting. Our results show that the use of LLMs enables worked example systems that follow an active learning approach to reduce bias in evaluating example explanations. 

The process of associating elements of learning content with concepts or skills that this content helps students to master is one of the critical steps in developing personalized educational systems. When these associations are properly established, the system can infer the growth of student understanding of separate knowledge components from the logs of their interactions with associated learning content and use it to adapt the learning process accordingly by targeting gaps in individual students' knowledge. Unfortunately, crafting these links between learning content and knowledge components is a very time- and expertise-demanding process that has traditionally been performed manually by domain experts with the help of knowledge engineers. Recently, the power of Large Language Models has motivated a new generation of research on concept extraction from textual learning content. The work presented in this paper contributes to this trend while introducing two important innovations. First, our concept extraction process is guided by a human-authored ontology of the target domain - Python programming. Second, alongside a traditional expert evaluation of the concept extraction quality, we apply two additional validation approaches: one based on using an educational data mining technique (learning curves) and another utilizing the pedagogical expertise of teaching the target domain (learning content placement). 

In this paper, we present an approach that enables a broad re-use of English-authored smart learning content by translating it into other languages. To make it possible, we integrated translation functionalities directly into a smart content authoring system and engaging human-AI collaboration. The approach has been used to translate a large volume of worked examples and completion problems in Python from English to Spanish. The translated content has been piloted in several universities in a Spanish-speaking country.\end{abstract} 

Textbooks have evolved over the last several decades in many aspects. Many textbooks can be accessed online, some of them freely. They often come with libraries of supplementary educational resources or online educational services built on top of them. These developments encouraged researchers working on different Artificial Intelligence (AI) technology to explore how AI can enhance digital textbooks and learners’ interaction with them. Over the years, research teams all over the world attempted to answer several important research questions in the new area of intelligent textbooks. How can we use AI technology to facilitate access to digital textbooks and improve the learning process? What new insights about knowledge and learning can be extracted from textbook content and data-mined from the logs of students interacting with it? How can these insights be leveraged to develop improved intelligent textbooks? How can we leverage generative AI technology to manage and augment textbooks? This special issue brings together a collection of papers that report results or recent research in the field of intelligent textbooks. These papers address the challenge of making textbooks ``more intelligent'' from different perspectives and make a step towards establishing intelligent textbooks as a new, interdisciplinary research field. 

Textbooks have evolved over the last several decades in many aspects. Most textbooks can be accessed online, many of them freely. They often come with libraries of supplementary educational resources or online educational services built on top of them. As a result of these enrichments, new research challenges and opportunities emerge that call for the application of AIED methods to enhance digital textbooks and learners’ interaction with them. Therefore, we ask: How can we use intelligent and adaptive technologies to facilitate access to digital textbooks and improve the learning process? What new insights about knowledge and learning can be extracted from textbook content and data-mined from the logs of students interacting with it? How can these insights be leveraged to develop improved intelligent texts? How can we leverage new language technology to manage and augment textbooks? This workshop will feature research contributions addressing these and other research questions related to intelligent textbooks. It will bring together researchers working on different aspects of learning technologies to establish intelligent textbooks as a new, interdisciplinary research field.