More Like this

1. Witness Feedback for Introductory CS Theory Assignments

   https://doi.org/10.1145/3408877.3439585  

   Bezáková, Ivona; Fluet, Kimberly; Hemaspaandra, Edith; Miller, Hannah; Narváez, David E. (January 2021, 52nd ACM Technical Symposium on Computer Science Education (SIGCSE))

   null (Ed.)

   Computing theory analyzes abstract computational models to rigorously study the computational difficulty of various problems. Introductory computing theory can be challenging for undergraduate students, and the overarching goal of our research is to help students learn these computational models. The most common pedagogical tool for interacting with these models is the Java Formal Languages and Automata Package (JFLAP). We developed a JFLAP server extension, which accepts homework submissions from students, evaluates the submission as correct or incorrect, and provides a witness string when the submission is incorrect. Our extension currently provides witness feedback for deterministic finite automata, nondeterministic finite automata, regular expressions, context-free grammars, and pushdown automata. In Fall 2019, we ran a preliminary investigation on two synchronized sections (Control and Study) of the required undergraduate course Introduction to Computer Science Theory. The Study section (n = 29) used our extension for five targeted homework questions, and the Control section (n = 35) submitted these problems using traditional means. The Study section strongly outperformed the Control section with respect to the percent of perfect homework grades for the targeted homework questions. Our most interesting result was student persistence: with only the short witness string as feedback, students voluntarily persisted in submitting attempts until correct. 

   more » « less
2. Interactive Editing of Circuits in a Step-Based Tutoring System

   Skromme, B; Redshaw, C.; Gupta, A; Gupta, S; Andrei, P; Erives, H; Bailey, A; Thompson, W; Bansal, S. (July 2020, American Society for Engineering Education Annual Conference)

   Step-based tutoring systems are known to be more effective than traditional answer-based systems. They however require that each step in a student’s work be accepted and evaluated automatically to provide effective feedback. In the domain of linear circuit analysis, it is frequently necessary to allow students to draw or edit circuits on their screen to simplify or otherwise transform them. Here, the interface developed to accept such input and provide immediate feedback in the Circuit Tutor system is described, along with systematic assessment data. Advanced simplification methods such as removing circuit sections that are removably hinged, voltage-splittable, or current-splittable are taught to students in an interactive tutorial and then supported in the circuit editor itself. To address the learning curve associated with such an interface, ~70 video tutorials were created to demonstrate exactly how to work the randomly generated problems at each level of each of the tutorials in the system. A complete written record or “transcript” of student’s work in the system is being made available, showing both incorrect and correct steps. Introductory interactive (multiple choice) tutorials are now included on most topics. Assessment of exercises using the interactive editor was carried out by professional evaluators for several institutions, including three that heavily serve underrepresented minorities. Both quantitative and qualitative methods were used, including focus groups, surveys, and interviews. Controlled, randomized, blind evaluations were carried out in three different course sections in Spring and Fall 2019 to evaluate three tutorials using the interactive editor, comparing use of Circuit Tutor to both a commercial answer-based system and to conventional textbook-based paper homework. In Fall 2019, students rated the software a mean of 4.14/5 for being helpful to learn the material vs. 3.05/5 for paper homework (HW), p < 0.001 and effect size d = 1.11σ. On relevant exam questions that semester, students scored significantly (p = 0.014) higher with an effect size of d = 0.64σ when using Circuit Tutor compared to paper HW in one class section, with no significant difference in the other section. 

   more » « less
3. Students’ written homework responses using digital games in Inquiry-Oriented Linear Algebra

   Bettersworth, Z.; Smith, K.; Zandieh, M. (January 2022, Proceedings of the Annual Conference on Research in Undergraduate Mathematics Education)

   Karunakaran, S.; Higgins, A. (Ed.)

   In this report, we characterize seven of twenty-five students’ responses to a single written homework assignment from the Spring 2021 academic semester. The homework was designed to incorporate the Vector Unknown 2D digital game to investigate how students answered questions about span and linear independence after playing various levels of the game. We present our modification of the roles and characteristics framework of Zandieh et al. (2019), our identification of students’ grammatical use of game language and math language, as well as the results of analyzing students’ homework responses using our framework. 

   more » « less
4. Mining SQL Problem Solving Patterns using Advanced Sequence Processing Algorithms

   https://doi.org/10.1145/3596673.3596973  

   Yang, Sophia; Herman, Geoffrey L.; Alawini, Abdussalam (June 2023, DataEd '23: Proceedings of the 2nd International Workshop on Data Systems Education: Bridging education practice with education research)

   SQL is a crucial language for managing relational database systems, and is an essential skill for individuals in roles such as researchers, developers, and business professionals who work with databases. However, learning SQL can be a challenge, presenting an opportunity to study the various methods students use to arrive at semantically equivalent SQL queries. In this study, we examined students’ SQL submissions to homework assignments in the Database Systems course offered to upper-level undergraduate and graduate students at the University of Illinois Urbana-Champaign during the Fall 2022 semester. Our goal was to understand how students arrive at SQL solutions and overcome challenges in the learning process by building on prior research on line chart visualizations that instructors can use to increase visibility on students who are struggling. However, a major limitation of this approach was the difficulty for instructors to sift through a large number of visuals representing each student’s performance on a SQL problem and generate action items at scale, especially when dealing with enrollments of over 700 students. To overcome this limitation, we developed a novel technique to generate textual representations of the student submission sequence using global sequence alignment scores and regular expression algorithms to further compact these submission sequences. This allows instructors to gain insights quickly, on an aggregate level, and in an automated manner, enabling them to identify students who may be struggling with SQL based on their submission sequence characteristics and take appropriate action to improve database education. Our study discovered common textual submission patterns and pattern elements, and we present our recommendations to instructors to improve database education based on these findings. 

   more » « less
5. Analyzing Patterns in Student SQL Solutions via Levenshtein Edit Distance

   https://doi.org/10.1145/3430895.3460979  

   Yang, Sophia; Wei, Ziyuan; Herman, Geoffrey L.; Alawini, Abdussalam (June 2021, Proceedings of the Eighth ACM Conference on Learning @ Scale)

   null (Ed.)

   Structured Query Language (SQL), the standard language for relational database management systems, is an essential skill for software developers, data scientists, and professionals who need to interact with databases. SQL is highly structured and presents diverse ways for learners to acquire this skill. However, despite the significance of SQL to other related fields, little research has been done to understand how students learn SQL as they work on homework assignments. In this paper, we analyze students' SQL submissions to homework problems of the Database Systems course offered at the University of Illinois at Urbana-Champaign. For each student, we compute the Levenshtein Edit Distances between every submission and their final submission to understand how students reached their final solution and how they overcame any obstacles in their learning process. Our system visualizes the edit distances between students' submissions to a SQL problem, enabling instructors to identify interesting learning patterns and approaches. These findings will help instructors target their instruction in difficult SQL areas for the future and help students learn SQL more effectively. 

   more » « less