Search for: All records

Understanding the thought processes of students as they progress from initial (incorrect) answers toward correct answers is a challenge for instructors, both in this pandemic and beyond. This paper presents a general network visualization learning analytics system that helps instructors to view a sequence of answers input by students in a way that makes student learning progressions apparent. The system allows instructors to study individual and group learning at various levels of granularity. The paper illustrates how the visualization system is employed to analyze student responses collected through an intervention. The intervention is BeginToReason, an online tool that helps students learn and use symbolic reasoning-reasoning about code behavior through abstract values instead of concrete inputs. The specific focus is analysis of tool-collected student responses as they perform reasoning activities on code involving conditional statements. Student learning is analyzed using the visualization system and a post-test. Visual analytics highlights include instances where students producing one set of incorrect answers initially perform better than a different set and instances where student thought processes do not cluster well. Post-test data analysis provides a measure of student ability to apply what they have learned and their holistic understanding. 

It is known that timely and personalized feedback is vital to the learning process, and because of increasing enroll- ment, instructors can find it harder to provide that feedback. Learning analytics presents a solution to this problem. The growth in popularity of online education systems better enables learning analytics by providing additional educational data. This work focuses on the analysis of students’ incorrect short answers and their pathways to correct solutions. By considering student submissions as sequences, this work uses a dimension called “distance” which can be used to predict how far off a student’s incorrect answer is from a correct one. This distance metric can be used for recognizing students who may need help, understanding which concepts students struggle with, evaluating assessment questions, and improving multiple-choice answers. This paper discusses the methods, relevant learning scenarios, and applications of the learning analytics system. It features the results and analysis of a usability test conducted on 56 faculty members. 

Online learning has become desirable for many students. In the U.S., more than one-third of all enrolled students participate in at least one online course [13]. The most effective online learning environments allow students to work at their own pace, from any location, at any time, and to receive automated feedback. In light of these benefits and the likely protracted impact of the current public health crisis, the trend toward online learning is likely to increase. 

Object-based development using design-by-contract (DbC) is broadly taught and practiced. Students must be able to read and write symbolic DbC assertions that are sufficiently precise and be able to use these assertions to trace program code. This paper summarizes the results of using an automated tool to pinpoint fine-grain difficulties students face in learning to symbolically trace code involving objects. The pilots were conducted in an undergraduate software engineering course. Quantitative results show that data collected by the tool can help to identify and classify learning obstacles. Qualitative findings help validate student misunderstandings underlying these difficulties. Analysis of exam questions helps understand the persistence of student learning to read and write simple assertions about code behavior. Together, these results provide directions for intervention. 

One aspect of developing correct code, code that functions as specified, is annotating loops with suitable invariants. Loop invariants are useful for human reasoning and are necessary for tool-assisted automated reasoning. Writing loop invariants can be a difficult task for all students, especially beginning software engineering students. In helping students learn to write adequate invariants, we need to understand not only what errors they make, but also why they make them. This poster discusses the use of a Web IDE backed by the RESOLVE verification engine to aid students in developing loop invariants and to collect performance data. In addition to collecting submitted invariant answers, students are asked to provide their steps or thought processes regarding how they arrived at their answers for each submission. The answers and reasons are then analyzed using a mixed-methods approach. Resulting categories of answers indicate that students are able to use formal method concepts with which they are already familiar, such as, pre and post-conditions as a starting place to develop adequate loop invariants. Additionally, some common trouble spots in learning to write invariants are identified. The results will be useful to guide classroom instruction and automated tutoring.