More Like this

1. You asked, now what? Modeling Students’ Help-Seeking and Coding actions from Re- quest to Resolution

   Zhikai Gao; Bradley Erickson; Yiqiao Xu; Collin Lynch; Sarah Heckman; Tiffany Barnes (December 2022, Journal of educational data mining)

   Nigel Bosch; Antonija Mitrovic; Agathe Merceron (Ed.)

   Demand for education in Computer Science has increased markedly in recent years. With increased demand has come to an increased need for student support, especially for courses with large programming projects. Instructors commonly provide online post forums or office hours to address this massive demand for help requests. Identifying what types of questions students are asking in those interactions and what triggers their help requests can in turn assist instructors in better managing limited help-providing resources. In this study, we aim to explore students’ help-seeking actions from the two separate approaches we mentioned before and investigate their coding actions before help requests to understand better what motivates students to seek help in programming projects. We collected students’ help request data and commit logs from two Fall offerings of a CS2 course. In our analysis, we first believe that different types of questions should be related to different behavioral patterns. Therefore, we first categorized students’ help requests based on their content (e.g., Implementation, General Debugging, or Addressing Teaching Staff (TS) Test Failures). We found that General Debugging is the most frequently asked question. Then we analyzed how the popularity of each type of request changed over time. Our results suggest that implementation is more popular in the early stage of the project cycle, and it changes to General Debugging and Addressing TS Failures in the later stage. We also calculated the accuracy of students’ commit frequency one hour before their help requests; the results show that before Implementation requests, the commit frequency is significantly lower, and before TS failure requests, the frequency is significantly higher. Moreover, we checked before any help request whether students changed their source code or test code. The results show implementation requests related to higher chances of source code changes and coverage questions related to more test code changes. Moreover, we use a Markov Chain model to show students’ action sequences before, during, and after the requests. And finally, we explored students’ progress after the office hours interaction and found that over half of the students improved the correctness of their code after 20 minutes of their office hours interaction addressing TS failures ends. 

   more » « less
2. Board 196: A Framework to Assess Debugging Skills for Computational Thinking in Science and Engineering

   https://doi.org/10.18260/1-2--42591  

   Hylton, Derrick; Sung, Shannon; Ding, Xiaotong; Van_Vleet, Mary (June 2023, ASEE Conferences)

   A rubric is presented to assess debugging skills for students particularly in the natural sciences and engineering. The three categories that are assessed for the cognitive processes in debugging skills are identification, isolation, and iteration. These are defined, and the characteristics of each process are listed. We discuss the method used to develop this rubric that was based on intentional errors in a programming assignment given to students in an introductory physics course. The programming in this assignment was in Python and a visual-based programming platform, called iFlow. We believe that visual-based programming will help elicit weaknesses in debugging because it removes students' familiarity with particular programming languages. Our focus on debugging skills came from a survey of students to self-identify barriers in computational work in an introductory physics course that included engineering majors. This skill was the primary self-identified barrier along with abstraction skills, which will be the focus of another work. We also present the results of this survey. The Python assignment (ntext = 9) was used to create the rubric and the iFlow assignment (ngraphic = 11) was used to test the rubric. Scoring was based on a scale of six levels in each category. Although the sample size was too small to establish rigorous scoring reliability, we discussed how the two researchers attained agreement in scoring the assignments after iterative modifications of the rubric and rescoring. For the Python assignment, the average for identification was 2.75/5, for isolation 2.30/5, and for iteration 3.33/5. For the iFlow assignment, the average for identification was 2.63/5, for isolation 2.23/5, and for iterate 3.32/5. A consistent trend from these assignments showed that students' approach to debugging is mainly to identify and iterate without a full understanding of the error (i.e., isolation). The lack of a full understanding of the error implies that students are prone to repeat the error. Thus, the important outcome of debugging is to understand the source of error by systematically investigating different parts of the computational solution. Our preliminary results led to the hypothesis that students with weak debugging skills are mainly due the isolation process. This hypothesis will be tested in a future experiment. Results from such an experiment will be significant to those who are designing intervention strategies to integrate computational thinking in science and engineering curricula. 

   more » « less
3. Analyzing debugging processes during collaborative, computational modeling in science

   Hutchins, N.M. (June 2021, Computersupported collaborative learning)

   Hmelo-Silver, C. E. (Ed.)

   This paper develops a systematic approach to identifying and analyzing high school students’ debugging strategies when they work together to construct computational models of scientific processes in a block-based programming environment. We combine Markov models derived from students’ activity logs with epistemic network analysis of their collaborative discourse to interpret and analyze their model building and debugging processes. We present a contrasting case study that illustrates the differences in debugging strategies between two groups of students and its impact on their model-building effectiveness. 

   more » « less
4. Analyzing debugging processes during collaborative, computational modeling in science

   Hutchins, N.M.; Snyder, C.; Emara, M.; Grover, S.; Biswas, G. (June 2021, Computersupported collaborative learning)

   Hmelo-Silver, C. E. (Ed.)

   This paper develops a systematic approach to identifying and analyzing high school students’ debugging strategies when they work together to construct computational models of scientific processes in a block-based programming environment. We combine Markov models derived from students’ activity logs with epistemic network analysis of their collaborative discourse to interpret and analyze their model building and debugging processes. We present a contrasting case study that illustrates the differences in debugging strategies between two groups of students and its impact on their model-building effectiveness. 

   more » « less
5. Efficient Counter-factual Type Error Debugging

   https://doi.org/10.1109/TASE.2019.00-13  

   Chen, Sheng; Wu, Baijun (July 2019, 2019 International Symposium on Theoretical Aspects of Software Engineering)

   Type inference is an important part of functional programming languages and has been increasingly adopted to imperative programming. However, providing effective error messages in response to type inference failures (due to type errors in programs) continues to be a challenge. Type error messages generated by compilers and existing error debugging approaches often point to bogus error locations or lack sufficient information for removing the type error, making error debugging ineffective. Counter-factual typing (CFT) addressed this problem by generating comprehensive error messages with each message includes a rich set of information. However, CFT has a large response time, making it too slow for interactive use. In particular, our recent study shows that programmers usually have to go through multiple iterations of updating and recompiling programs to remove a type error. Interestingly, our study also reveals that program updates are minor in each iteration during type error debugging. We exploit this fact and develop eCFT, an efficient version of CFT, which doesn't recompute all error fixes from scratch for each updated program but only recomputes error fixes that are changed in response to the update. Our key observation is that minor program changes lead to minor error suggestion changes. eCFT is based on principal typing, a typing scheme more amenable to reuse previous typing results. We have evaluated our approach and found it is about 12.4× faster than CFT in updating error fixes. 

   more » « less