More Like this

1. Explaining Explainability: Early Performance Prediction with Student Programming Pattern Profiling

   Hoq, Muntasir; Brusilovsky, Peter; Akram, Bita (November 2024, Journal of Educational Data Mining)

   The ability to predict student performance in introductory programming courses is important to help struggling students and enhance their persistence. However, for this prediction to be impactful, it is crucial that it remains transparent and accessible for both instructors and students, ensuring effective utilization of the predicted results. Machine learning models with explainable features provide an effective means for students and instructors to comprehend students' diverse programming behaviors and problem-solving strategies, elucidating the factors contributing to both successful and suboptimal performance. This study develops an explainable model that predicts student performance based on programming assignment submission information in different stages of the course to enable early explainable predictions. We extract data-driven features from student programming submissions and utilize a stacked ensemble model for predicting final exam grades. The experimental results suggest that our model successfully predicts student performance based on their programming submissions earlier in the semester. Employing SHAP, a game-theory-based framework, we explain the model's predictions, aiding stakeholders in understanding the influence of diverse programming behaviors on students' success. Additionally, we analyze crucial features, employing a mix of descriptive statistics and mixture models to identify distinct student profiles based on their problem-solving patterns, enhancing overall explainability. Furthermore, we dive deeper and analyze the profiles using different programming patterns of the students to elucidate the characteristics of different students where SHAP explanations are not comprehensible. Our explainable early prediction model elucidates common problem-solving patterns in students relative to their expertise, facilitating effective intervention and adaptive support. 

   more » « less
2. Explaining Explainability: Early Performance Prediction with Student Programming Pattern Profiling

   https://doi.org/10.5281/zenodo.14246435  

   Hoq, Muntasir; Brusilovsky, Peter; Akram, Bita (July 2025, Educational Data Mining)

   The ability to predict student performance in introductory programming courses is important to help struggling students and enhance their persistence. However, for this prediction to be impactful, it is crucial that it remains transparent and accessible for both instructors and students, ensuring effective utilization of the predicted results. Machine learning models with explainable features provide an effective means for students and instructors to comprehend students' diverse programming behaviors and problem-solving strategies, elucidating the factors contributing to both successful and suboptimal performance. This study develops an explainable model that predicts student performance based on programming assignment submission information in different stages of the course to enable early explainable predictions. We extract data-driven features from student programming submissions and utilize a stacked ensemble model for predicting final exam grades. The experimental results suggest that our model successfully predicts student performance based on their programming submissions earlier in the semester. Employing SHAP, a game-theory-based framework, we explain the model's predictions, aiding stakeholders in understanding the influence of diverse programming behaviors on students' success. Additionally, we analyze crucial features, employing a mix of descriptive statistics and mixture models to identify distinct student profiles based on their problem-solving patterns, enhancing overall explainability. Furthermore, we dive deeper and analyze the profiles using different programming patterns of the students to elucidate the characteristics of different students where SHAP explanations are not comprehensible. Our explainable early prediction model elucidates common problem-solving patterns in students relative to their expertise, facilitating effective intervention and adaptive support. 

   more » « less
3. Analysis of an Explainable Student Performance Prediction Model in an Introductory Programming Course

   Hoq, Muntasir; Brusilovsky, Peter; Akram, Bita (July 2023, the 16th International Conference on Educational Data Mining (EDM 2023))

   Prediction of student performance in Introductory programming courses can assist struggling students and improve their persistence. On the other hand, it is important for the prediction to be transparent for the instructor and students to effectively utilize the results of this prediction. Explainable Machine Learning models can effectively help students and instructors gain insights into students’ different programming behaviors and problem-solving strategies that can lead to good or poor performance. This study develops an explainable model that predicts students’ performance based on programming assignment submission information. We extract different data-driven features from students’ programming submissions and employ a stacked ensemble model to predict students’ final exam grades. We use SHAP, a game-theory-based framework, to explain the model’s predictions to help the stakeholders understand the impact of different programming behaviors on students’ success. Moreover, we analyze the impact of important features and utilize a combination of descriptive statistics and mixture models to identify different profiles of students based on their problem-solving patterns to bolster explainability. The experimental results suggest that our model significantly outperforms other Machine Learning models, including KNN, SVM, XGBoost, Bagging, Boosting, and Linear regression. Our explainable and transparent model can help explain students’ common problem-solving patterns in relationship with their level of expertise resulting in effective intervention and adaptive support to students. 

   more » « less
4. Investigating Elements of Student Persistence in an Introductory Computer Science Course

   Pinto, J.D.; Zhang, Y.; Paquette, L.; Fan, A.X. (November 2021, 5th Educational Data Mining in Computer Science Education (CSEDM) Workshop)

   null; null (Ed.)

   We explore how different elements of student persistence on computer programming problems may be related to learning outcomes and inform us about which elements may distinguish between productive and unproductive persistence. We collected data from an introductory computer science course at a large midwestern university in the U.S. hosted on an open-source, problem-driven learning system. We defined a set of features quantifying various aspect of persistence during problem solving and used a predictive modeling approach to predict student scores on subsequent and related quiz questions. We focused on careful feature engineering and model interpretation to shed light on the intricacies of both productive and unproductive persistence. Feature importance was analyzed using SHapley Additive exPlanations (SHAP) values. We found that the most impactful features were persisting until solving the problem, rapid guessing, and taking a break, while those with the strongest correlation between their values and their impact on prediction were the number of submissions, total time, and (again) taking a break. This suggests that the former are important features for accurate prediction, while the latter are indicative of the differences between productive persistence and wheel spinning in a computer science context. 

   more » « less
5. Identifying Collaborative Problem-Solving Behaviors Using Sequential Pattern Mining

   Zhou, Yiqiu (June 2023, 2023 American Society for Engineering Education Annual Conference & Exposition)

   With the increasing adoption of collaborative learning approaches, instructors must understand students’ problem-solving approaches during collaborative activities to better design their class. Among the multiple ways to reveal collaborative problem-solving processes, temporal submission patterns is one that is more scalable and generalizable in Computer Science education. In this paper, we provide a temporal analysis of a large dataset of students’ submissions to collaborative learning assignments in an upper-level database course offered at a large public university. The log data was collected from an online assessment and learning system, containing the timestamps of each student’s submissions to a problem on the collaborative assignment. Each submission was labeled as quick (Q), medium (M), or slow (S) based on its duration and whether it was shorter or longer than the 25th and 75th percentile. Sequential compacting and mining techniques were employed to identify pairs of transitions highly associated with one another. This preliminary research sheds light on the recurring submission patterns derived from the amount of time spent on each problem, warranting further examination on these patterns to unpack collaborative problem-solving behaviors. Our study demonstrates the potential of temporal analysis to identify meaningful problem-solving patterns based on log traces, which may help flag key moments and alert instructors to provide in-time scaffolding when students work on group assignments. 

   more » « less