More Like this

1. Tool-Aided Loop Invariant Development: Insights into Student Conceptions and Difficulties

   https://doi.org/10.1145/3430665.3456351  

   Fowler, Megan; Kraemer, Eileen; Sitaraman, Murali; Hollingsworth, Joseph E. (June 2021, ITiCSE '21: Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education)

   null (Ed.)

   To develop code that meets its specification and is verifiably correct, such as in a software engineering course, students must be able to understand formal contracts and annotate their code with assertions such as loop invariants. To assist in developing suitable instructor and automated tool interventions, this research aims to go beyond simple pre- and post-conditions and gain insight into student learning of loop invariants involving objects. As students develop suitable loop invariants for given code with the aid of an online system backed by a verification engine, each student attempt, either correct or incorrect, was collected and analyzed automatically, and catalogued using an iterative process to capture common difficulties. Students were also asked to explain their thought process in arriving at their answer for each submission. The collected explanations were analyzed manually and found to be useful to assess their level of understanding as well as to extract actionable information for instructors and automated tutoring systems. Qualitative conclusions include the impact of the medium. 

   more » « less
2. Visualization of Students’ Solutions as a Sequential Network

   https://doi.org/10.1109/EDUCON52537.2022.9766502  

   Hurtig, Nathan; Hollingsworth, Joseph; Scrivner, Olga (March 2022, IEEE Global Engineering Education Conference (EDUCON))

   It is known that timely and personalized feedback is vital to the learning process, and because of increasing enroll- ment, instructors can find it harder to provide that feedback. Learning analytics presents a solution to this problem. The growth in popularity of online education systems better enables learning analytics by providing additional educational data. This work focuses on the analysis of students’ incorrect short answers and their pathways to correct solutions. By considering student submissions as sequences, this work uses a dimension called “distance” which can be used to predict how far off a student’s incorrect answer is from a correct one. This distance metric can be used for recognizing students who may need help, understanding which concepts students struggle with, evaluating assessment questions, and improving multiple-choice answers. This paper discusses the methods, relevant learning scenarios, and applications of the learning analytics system. It features the results and analysis of a usability test conducted on 56 faculty members. 

   more » « less
3. Investigating a collaborative group exam as an instructional tool to address student reasoning difficulties that remain even after instruction

   https://doi.org/10.1119/perc.2020.pr.mcinerny  

   McInerny, Alistair; Kryjevskaia, Mila (September 2020, Proceedings of the 2020 Physics Education Research Conference)

   Many students tend to provide intuitively appealing (but incorrect) responses to some physics questions despite demonstrating (on isomorphic questions) the formal knowledge necessary to reason correctly. These inconsistencies in reasoning are persistent and remain even after evidence-based instruction. This project probed whether a collaborative group exam could serve not only as an innovative assessment tool but also as an instructional intervention that helps address persistent reasoning difficulties. Specifically, students were given opportunities to revisit their answers to questions known to elicit intuitively appealing responses in a collaborative group exam component immediately following a traditional individual exam. The efficacy of this approach was compared to that of a more traditional instructor-led exam review session. Both approaches yielded moderate improvements in performance on the final exam. However, additional multi-faceted data analysis provided further insights into student reasoning difficulties that suggested further implication for instruction and research. 

   more » « less
4. From Typing to Insights: An Interactive Code Visualization and Error Detection for Enhanced Student Support Using Keystroke Data

   https://doi.org/10.18260/1-2--56621  

   James, Friday; Weese, Joshua; Feldhausen, Russell; Bean, Nathan; Friend, Michelle; Allen, David (June 2025, ASEE Conferences)

   The continuous rising of digital learning platforms have unarguably brought about a surge in the amount of data obtained from different learning environments, which presents a great opportunity for computer science teachers to gain an understanding of students’ coding processes. This is vital for enhancing student support as teachers can gain insights into students’ thought process, strategies and identify areas where students might be struggling while tacking programming tasks. Traditional assessment methods such as feedback after homework submissions or completed lab assignments often results in late and untimely intervention that would prevent early dropouts and massive failures as the students’ learning journey is neglected in the process. We leverage students’ keystroke data obtained from Python and Java-based introductory programming courses delivered through CODIO learning platform, to design an interactive code visualization and error detection platform using Streamlit. The application features an interface that reproduces students’ code snippets with JavaScript enabled syntax highlighting. It includes a combination of dropdown menu and an adjustable slider which enables an instructor to navigate through the timestamps and have a detailed view of students’ coding processes. The application also includes a navigation to an environment that enables instructors to run the generated code snippets for error detection, giving the instructors clear idea of the difficult part(s) of the course for intervention purposes. The intervention ultimately fosters a more supportive learning environment and helps boost students’ confidence. 

   more » « less
5. Learning Loop Invariants

   https://doi.org/10.1145/3328778.3372715  

   Fowler, Megan (February 2020, SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education)

   One aspect of developing correct code, code that functions as specified, is annotating loops with suitable invariants. Loop invariants are useful for human reasoning and are necessary for tool-assisted automated reasoning. Writing loop invariants can be a difficult task for all students, especially beginning software engineering students. In helping students learn to write adequate invariants, we need to understand not only what errors they make, but also why they make them. This poster discusses the use of a Web IDE backed by the RESOLVE verification engine to aid students in developing loop invariants and to collect performance data. In addition to collecting submitted invariant answers, students are asked to provide their steps or thought processes regarding how they arrived at their answers for each submission. The answers and reasons are then analyzed using a mixed-methods approach. Resulting categories of answers indicate that students are able to use formal method concepts with which they are already familiar, such as, pre and post-conditions as a starting place to develop adequate loop invariants. Additionally, some common trouble spots in learning to write invariants are identified. The results will be useful to guide classroom instruction and automated tutoring. 

   more » « less