Search for: All records

There is a growing community of researchers at the intersection- tion of data mining, AI, and computing education research. The objective of the CSEDM workshop is to facilitate a dis- Discussion among this research community, with a focus on how data mining can be uniquely applied in computing ed- ucation research. For example, what new techniques are needed to analyze program code and CS log data? How do results from CS education inform our analysis of this data? The workshop is meant to be an interdisciplinary event at the intersection of EDM and Computing Education Research. Researchers, faculty, and students are encouraged to share their AI- and data-driven approaches, methodological- gies, and experiences where data transforms how students learn Computer Science (CS) skills. This full-day workshop will feature paper presentations and discussions to promote collaboration. 

Understanding student practice behavior and its connection to their learning is essential for effective recommender systems that provide personalized learning support. In this study, we apply a sequential pattern mining approach to analyze student practice behavior in a practice system for introductory Python programming. Our goal is to identify different types of practice behavior and connect them to student performance. We examine two types of practice sequences: (1) by login session and (2) by learning topic. For each sequence type, we use SPAM (Sequential PAttern Mining) to identify the most frequent micro-patterns and build behavior profiles of individual learners as vectors of micro-pattern frequencies observed in their behavior. We confirm that these vectors are stable for both sequence types (p < 0.03 for session sequences and p < 0.003 for topic sequences). Using the vectors, we perform K-means clustering where we identify two practice behaviors: example explorers and persistent finishers. We repeat this experiment using different coding approaches for student sequences and obtain similar clusters. Our results suggest that example explorers and persistent finishers might represent two typical types of divergent student behaviors in a programming practice system. Finally, to better understand the relationship between students' background knowledge, learning outcomes, and practice behavior, we perform statistical analyses to assess the significance of the associations among pre-test scores, cluster assignments, and final course grades. 

The “Doer Effect” is the empirical phenomenon observed as a stronger correlational relationship between students who complete more activities and their course learning outcomes compared to those who complete fewer activities or watch fewer videos. In this paper, we extended prior evidence of a “Doer Effect” to investigate how doing more can be related not only to better learning outcomes but also to motivational ones. Specifically, we investigated persistence as the student’s willingness to continue working on course activities. We used secondary analyses of data from MOOC that taught Advanced Placement (AP) Introductory Java Programming to high school students using the digital textbook platform RuneStone. Although we failed to identify a doer effect in learning outcomes, our analyses do suggest that completing more activities is related to longer persistence in the course than reading more pages or watching more videos. This effect does not appear to be limited to highly motivated students. 

Knowledge tracing is a method to model students’ knowledge and enable personalized education in many STEM disciplines such as mathematics and physics, but has so far still been a challenging task in computing disciplines. One key obstacle to successful knowledge tracing in computing education lies in the accurate extraction of knowledge components (KCs), since multiple intertwined KCs are practiced at the same time for programming problems. In this paper, we address the limitations of current methods and explore a hybrid approach for KC extraction, which combines automated code parsing with an expert-built ontology. We use an introductory (CS1) Java benchmark dataset to compare its KC extraction performance with the traditional extraction methods using a state-of-the-art evaluation approach based on learning curves. Our preliminary results show considerable improvement over traditional methods of student modeling. The results indicate the opportunity to improve automated KC extraction in CS education by incorporating expert knowledge into the process. 

Novice programmers can greatly improve their understanding of challenging programming concepts by studying worked examples that demonstrate the implementation of these concepts. Despite the extensive repositories of effective worked examples created by CS education experts, a key challenge remains: identifying the most relevant worked example for a given programming problem and the specific difficulties a student faces solving the problem. Previous studies have explored similar example recommendation approaches. Our research introduces a novel method by utilizing deep learning code representation models to generate code vectors, capturing both syntactic and semantic similarities among programming examples. Driven by the need to provide relevant and personalized examples to programming students, our approach emphasizes similarity assessment and clustering techniques to identify similar code problems, examples, and challenges. This method aims to deliver more accurate and contextually relevant recommendations based on individual learning needs. Providing tailored support to students in real-time facilitates better problem-solving strategies and enhances students' learning experiences, contributing to the advancement of programming education. 

Many educational recommender systems (EdRecSys) rely on commercial recommendation strategies that emphasize content relevance while neglecting learners’ views on recommendation effectiveness. To address this, we conducted a co-design study with computer science students in an introductory programming course to explore their vision of an ideal EdRecSys. The subjects shared preferences and concerns related to three areas: recommendation approaches, transparency, and control. We used Zimmerman’s model of self-regulated learning to contextualize their expectations within a broader educational framework. Findings offer actionable insights for designing learner-centered AIED systems that foster engagement, agency, and self-regulation. 

The ability to predict student performance in introductory programming courses is important to help struggling students and enhance their persistence. However, for this prediction to be impactful, it is crucial that it remains transparent and accessible for both instructors and students, ensuring effective utilization of the predicted results. Machine learning models with explainable features provide an effective means for students and instructors to comprehend students' diverse programming behaviors and problem-solving strategies, elucidating the factors contributing to both successful and suboptimal performance. This study develops an explainable model that predicts student performance based on programming assignment submission information in different stages of the course to enable early explainable predictions. We extract data-driven features from student programming submissions and utilize a stacked ensemble model for predicting final exam grades. The experimental results suggest that our model successfully predicts student performance based on their programming submissions earlier in the semester. Employing SHAP, a game-theory-based framework, we explain the model's predictions, aiding stakeholders in understanding the influence of diverse programming behaviors on students' success. Additionally, we analyze crucial features, employing a mix of descriptive statistics and mixture models to identify distinct student profiles based on their problem-solving patterns, enhancing overall explainability. Furthermore, we dive deeper and analyze the profiles using different programming patterns of the students to elucidate the characteristics of different students where SHAP explanations are not comprehensible. Our explainable early prediction model elucidates common problem-solving patterns in students relative to their expertise, facilitating effective intervention and adaptive support. 

Worked examples have consistently demonstrated their value in education, serving as the model solutions for solving specific problem types. Past studies indicate that combining worked examples with practice problems is more effective than providing either problems or examples in isolation. Despite these findings, the exploration of the effects of grouping worked examples and problems for programming practice is limited, especially in learning environments designed for practice. This paper compares two content organization approaches in a practice system. The first one is explicitly connecting worked examples and completion problems, allowing students to access them in smaller bundles. The other one is delivering the same set of activities separately but keeping an implicit connection by grouping them under a topic. We examined the effects of these two approaches on student engagement and performance in a semester-long classroom experiment conducted in a CS1 programming course. The results indicate that explicitly connecting worked examples and completion problems increased engagement with the completion problems and supported problem-solving performance by leading to higher success rates and persistence. 

Personalized practice systems focus on supporting self-organized learning in a free practice mode. Adapting to the learners’ knowledge and goals, these systems help them navigate the increasing volumes of smart learning content, guide them to practice opportunities that are most appropriate to their level of knowledge and increase their motivation to practice. In this paper, we distill the experience generated by 20 years of research on personalized practice systems into a set of AI-based technologies that make these systems efficient, engaging, and user-friendly.