Background and context. “Explain in Plain English” (EiPE) questions ask students to explain the high-level purpose of code, requiring them to understand the macrostructure of the program’s intent. A lot is known about techniques that experts use to comprehend code, but less is known about how we should teach novices to develop this capability.
Objective. Identify techniques that can be taught to students to assist them in developing their ability to comprehend code and contribute to the body of knowledge of how novices develop their code comprehension skills.
Method. We developed interventions that could be taught to novices motivated by previous research about how experts comprehend code: prompting students to identify beacons, identify the role of variables, tracing, and abstract tracing. We conducted think-aloud interviews of introductory programming students solving EiPE questions, varying which interventions each student was taught. Some participants were interviewed multiple times throughout the semester to observe any changes in behavior over time.
Findings. Identifying beacons and the name of variable roles were rarely helpful, as they did not encourage students to integrate their understanding of that piece in relation to other lines of code. However, prompting students to explain each variable’s purpose helped them focus on useful subsets of the code, which helped manage cognitive load. Tracing was helpful when students incorrectly recognized common programming patterns or made mistakes comprehending syntax (text-surface). Prompting students to pick inputs that potentially contradicted their current understanding of the code was found to be a simple approach to them effectively selecting inputs to trace. Abstract tracing helped students see high-level, functional relationships between variables. In addition, we observed student spontaneously sketching algorithmic visualizations that similarly helped them see relationships between variables.
Implications. Because students can get stuck at many points in the process of code comprehension, there seems to be no silver bullet technique that helps in every circumstance. Instead, effective instruction for code comprehension will likely involve teaching a collection of techniques. In addition to these techniques, meta-knowledge about when to apply each technique will need to be learned, but that is left for future research. At present, we recommend teaching a bottom-up, concrete-to-abstract approach.
more »
« less
On Students' Usage of Tracing for Understanding Code
Explain in Plain English (EiPE) questions evaluate whether students can understand and explain the high-level purpose of code. We conducted a qualitative think-aloud study of introductory programming students solving EiPE questions. In this paper, we focus on how students use tracing (mental execution) to understand code in order to explain it.
We found that, in some cases, tracing can be an effective strategy for novices to understand and explain code. Furthermore, we observed three problems that prevented tracing from being helpful, which are 1) not employing tracing when it could be helpful (some struggling students explained correctly after the interviewer suggested tracing the code), 2) tracing incorrectly due to misunderstandings of the programming language, and 3) tracing with a set of inputs that did not sufficiently expose the code’s behavior (upon interviewer suggesting inputs, students explained correctly). These results suggest that we should teach students to use tracing as a method for understanding code and teach them how to select appropriate inputs to trace.
more »
« less
- Award ID(s):
- 2121424
- NSF-PAR ID:
- 10434209
- Date Published:
- Journal Name:
- SIGCSE 2023: Proceedings of the 54th ACM Technical Symposium on Computer Science Education
- Page Range / eLocation ID:
- 129 to 136
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Novice programmers often struggle with code understanding and debugging. Live Programming environments visualize the runtime values of a program each time it is modified to provide immediate feedback, which help with tracing the program execution. This paper presents the use of a Live Programming tool in a CS1 course to better understand the impact of Live Programming on novices’ learning metrics and their perceptions of the tool. We conducted a within-subjects study at a large public university in a CS1 course in Python (N=237) where students completed tasks in a lab setting, in some cases with a Live Programming environment, and in some cases without. Through post-lab surveys and open-ended feedback, we measured how well students understood the material and how students perceived the programming environment. To understand the impact of Live Programming, we compared the collected data for students who used Live Programming with the data for students who did not. We found that while learning outcomes were the same regardless of whether Live Programming was used or not, students who used the Live Programming tool completed some code tracing tasks faster. Furthermore, students liked the Live Programming environment more, and rated it as more helpful for their learning.more » « less
-
What knowledge does learning programming require? Prior work has focused on theorizing program writing and problem solving skills. We examine program comprehension and propose a formal theory of program tracing knowledge based on control flow paths through an interpreter program's source code. Because novices cannot understand the interpreter's programming language notation, we transform it into causal relationships from code tokens to instructions to machine state changes. To teach this knowledge, we propose a comprehension-first pedagogy based on causal inference, by showing, explaining, and assessing each path by stepping through concrete examples within many example programs. To assess this pedagogy, we built PLTutor, a tutorial system with a fixed curriculum of example programs. We evaluate learning gains among self-selected CS1 students using a block randomized lab study comparing PLTutor with Codecademy, a writing tutorial. In our small study, we find some evidence of improved learning gains on the SCS1, with average learning gains of PLTutor 60% higher than Codecademy (gain of 3.89 vs. 2.42 out of 27 questions). These gains strongly predicted midterms (R2=.64) only for PLTutor participants, whose grades showed less variation and no failures.more » « less
-
Open-ended tasks can be both beneficial and challenging to students learning to program. Such tasks allow students to be more creative and feel ownership over their work, but some students struggle with unstructured tasks and, without proper scaffolds, this can lead to negative learning experiences. Scratch is a widely used coding platform to teach computer science in classrooms and is designed to support learner creativity and expression. With its open-ended nature, Scratch can be used in various ways in the classroom to meet the needs of schools and districts. One challenge of using Scratch in classrooms is supporting learners in exploring their interests and fostering creativity while still meeting the instructional goals of a lesson and ensuring all students are engaged with, and understand, focal concepts and practices. In this paper, we investigate the use of planning sheets to fa- cilitate novice programmers designing and implementing Scratch programs based on open-ended prompts. To evaluate the plan- ning sheets, we look at how closely students’ implemented Scratch projects match their plans and whether the implemented Scratch projects met the technical requirements for the given lesson. We analyzed 303 Scratch projects from 155 middle grade students (ages 10-14) who were introduced to programming via the Scratch Encore Curriculum. Completed Scratch projects that used planning sheets (202) were qualitatively coded to evaluate how closely they matched the initial plan, and Scratch programs (303) were analyzed with an automated grader to check if technical project requirements were met. Our results reveal that students that used planning sheets met significantly more technical project requirements and had more complex structures than those that did not have planning sheets. Results differ based on teacher and type of planning sheet used (physical vs. virtual). This work suggests that planning sheets are a helpful tool for young learners when completing open-ended coding projects.more » « less
-
more » « less
Abstract When students answer test questions incorrectly, we often assume they don't understand the content; instead, they may struggle with certain cognitive skills or with how questions are asked. Our goal was to look beyond content to understand what makes assessment questions most challenging. On the basis of more than 76,000 answers to multiple-choice questions in a large, introductory biology course, we examined three question components—cognitive skills, procedural knowledge, and question forms—and their interactions. We found that the most challenging questions require the students to organize information and make meaning from it—skills that are essential in science. For example, some of the most challenging questions are presented as unstructured word problems and require interpretation; to answer correctly, the students must identify and extract the important information and construct their understanding from it. Our results highlight the importance of teaching students to organize and make meaning from the content we teach.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3545945.3569741
