skip to main content

This content will become publicly available on March 1, 2023

Title: Investigating the Impact of Using a Live Programming Environment in a CS1 Course
Novice programmers often struggle with code understanding and debugging. Live Programming environments visualize the runtime values of a program each time it is modified to provide immediate feedback, which help with tracing the program execution. This paper presents the use of a Live Programming tool in a CS1 course to better understand the impact of Live Programming on novices’ learning metrics and their perceptions of the tool. We conducted a within-subjects study at a large public university in a CS1 course in Python (N=237) where students completed tasks in a lab setting, in some cases with a Live Programming environment, and in some cases without. Through post-lab surveys and open-ended feedback, we measured how well students understood the material and how students perceived the programming environment. To understand the impact of Live Programming, we compared the collected data for students who used Live Programming with the data for students who did not. We found that while learning outcomes were the same regardless of whether Live Programming was used or not, students who used the Live Programming tool completed some code tracing tasks faster. Furthermore, students liked the Live Programming environment more, and rated it as more helpful for their learning.
; ; ; ;
Award ID(s):
Publication Date:
Journal Name:
In Proceedings of the 53rd ACM Technical Symposium on Computer Science Education
Sponsoring Org:
National Science Foundation
More Like this
  1. We analyzed the data collected in a Computer Science I course to quantify the relationship between programming projects, code-tracing assignments and course grade when online tests and closed lab instruction were used in the course. We found that completion of programming projects was positively and moderately correlated with course grade; each completed project contributed nearly one sign grade to the course grade; the grade of students who had completed at least a given number of projects was four sign grades better than of those who had not and the difference was statistically significant; the mean course grade ranged from Fmore »for those who had completed 1 or fewer projects to A- for those who had completed 9 or more projects; and completion of later projects was indicative of higher grade in the course. Similarly, completion of code-tracing assignments was positively, but weakly correlated with course grade; the grade of students who had completed at least a given number of assignments was one letter grade better than of those who had not, and the difference was statistically significant; and the mean course grade ranged from C- for those who had completed 6 or fewer assignments to B+ for those who had completed 11 or more assignments. Concurrence among the course objectives, classroom instruction, assessment techniques, programming projects and assignments may be a pre-requisite for obtaining the results of this study.« less
  2. This Work-in-Progress paper in the Research Category uses a retrospective mixed-methods study to better understand the factors that mediate learning of computational modeling by life scientists. Key stakeholders, including leading scientists, universities and funding agencies, have promoted computational modeling to enable life sciences research and improve the translation of genetic and molecular biology high- throughput data into clinical results. Software platforms to facilitate computational modeling by biologists who lack advanced mathematical or programming skills have had some success, but none has achieved widespread use among life scientists. Because computational modeling is a core engineering skill of value to other STEMmore »fields, it is critical for engineering and computer science educators to consider how we help students from across STEM disciplines learn computational modeling. Currently we lack sufficient research on how best to help life scientists learn computational modeling. To address this gap, in 2017, we observed a short-format summer course designed for life scientists to learn computational modeling. The course used a simulation environment designed to lower programming barriers. We used semi-structured interviews to understand students' experiences while taking the course and in applying computational modeling after the course. We conducted interviews with graduate students and post- doctoral researchers who had completed the course. We also interviewed students who took the course between 2010 and 2013. Among these past attendees, we selected equal numbers of interview subjects who had and had not successfully published journal articles that incorporated computational modeling. This Work-in-Progress paper applies social cognitive theory to analyze the motivations of life scientists who seek training in computational modeling and their attitudes towards computational modeling. Additionally, we identify important social and environmental variables that influence successful application of computational modeling after course completion. The findings from this study may therefore help us educate biomedical and biological engineering students more effectively. Although this study focuses on life scientists, its findings can inform engineering and computer science education more broadly. Insights from this study may be especially useful in aiding incoming engineering and computer science students who do not have advanced mathematical or programming skills and in preparing undergraduate engineering students for collaborative work with life scientists.« less
  3. What knowledge does learning programming require? Prior work has focused on theorizing program writing and problem solving skills. We examine program comprehension and propose a formal theory of program tracing knowledge based on control flow paths through an interpreter program's source code. Because novices cannot understand the interpreter's programming language notation, we transform it into causal relationships from code tokens to instructions to machine state changes. To teach this knowledge, we propose a comprehension-first pedagogy based on causal inference, by showing, explaining, and assessing each path by stepping through concrete examples within many example programs. To assess this pedagogy, wemore »built PLTutor, a tutorial system with a fixed curriculum of example programs. We evaluate learning gains among self-selected CS1 students using a block randomized lab study comparing PLTutor with Codecademy, a writing tutorial. In our small study, we find some evidence of improved learning gains on the SCS1, with average learning gains of PLTutor 60% higher than Codecademy (gain of 3.89 vs. 2.42 out of 27 questions). These gains strongly predicted midterms (R2=.64) only for PLTutor participants, whose grades showed less variation and no failures.« less
  4. A solid understanding of electromagnetic (E&M) theory is key to the education of electrical engineering students. However, these concepts are notoriously challenging for students to learn, due to the difficulty in grasping abstract concepts such as the electric force as an invisible force that is acting at a distance, or how electromagnetic radiation is permeating and propagating in space. Building physical intuition to manipulate these abstractions requires means to visualize them in a three-dimensional space. This project involves the development of 3D visualizations of abstract E&M concepts in Virtual Reality (VR), in an immersive, exploratory, and engaging environment. VR providesmore »the means of exploration, to construct visuals and manipulable objects to represent knowledge. This leads to a constructivist way of learning, in the sense that students are allowed to build their own knowledge from meaningful experiences. In addition, the VR labs replace the cost of hands-on labs, by recreating the experiments and experiences on Virtual Reality platforms. The development of the VR labs for E&M courses involves four distinct phases: (I) Lab Design, (II) Experience Design, (III) Software Development, and (IV) User Testing. During phase I, the learning goals and possible outcomes are clearly defined, to provide context for the VR laboratory experience, and to identify possible technical constraints pertaining to the specific laboratory exercise. During stage II, the environment (the world) the player (user) will experience is designed, along with the foundational elements, such as ways of navigation, key actions, and immersion elements. During stage III, the software is generated as part of the course projects for the Virtual Reality course taught in the Computer Science Department at the same university, or as part of independent research projects involving engineering students. This reflects the strong educational impact of this project, as it allows students to contribute to the educational experiences of their peers. During phase IV, the VR experiences are played by different types of audiences that fit the player type. The team collects feedback and if needed, implements changes. The pilot VR Lab, introduced as an additional instructional tool for the E&M course during the Fall 2019, engaged over 100 students in the program, where in addition to the regular lectures, students attended one hour per week in the E&M VR lab. Student competencies around conceptual understanding of electromagnetism topics are measured via formative and summative assessments. To evaluate the effectiveness of VR learning, each lab is followed by a 10-minute multiple-choice test, designed to measure conceptual understanding of the various topics, rather than the ability to simply manipulate equations. This paper discusses the implementation and the pedagogy of the Virtual Reality laboratory experiences to visualize concepts in E&M, with examples for specific labs, as well as challenges, and student feedback with the new approach. We will also discuss the integration of the 3D visualizations into lab exercises, and the design of the student assessment tools used to assess the knowledge gain when the VR technology is employed.« less
  5. The highly mathematical nature of introductory level vibrations and control theory courses results in students struggling to understand the concepts. Hands-on activity demonstrated in class can help them better understand the concepts. However, there is still an ongoing effort to lower the currently substantial cost of educational laboratory equipment for undergraduate-level engineering courses. Also, with the COVID-19 crisis, the Spring 2020 academic year took an unexpected turn for academics and students all over the world. Engineering faculty who teach laboratories had to move online and instruct from home. Online course preparation takes more time and effort compared to traditionally designedmore »face-to-face courses and was compounded considering the unprecedented situation where many instructors didn't have time to record data from existing lab equipment or record video in their laboratories. In this paper, we present a Matlab Simscape GUI program designed to simulate modeling and control of dynamical systems for vibrations and control theory courses, and their associated laboratories, as one potential solution for online instruction. To complement the simulation program, online classroom and homework activities were designed using a learning sciences approach connecting several critical educational theories which can bolster student motivation, engagement with the material, and overall learning performance. The simulation is presented along with data from 19 students who completed the associated classroom and homework activities. Survey results probing student perceptions about the value of the learning tasks for the simulation were overwhelmingly positive and indicate this approach holds promise in supporting student learning.« less