skip to main content


Title: Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study
Background and Context Lopez and Lister first presented evidence for a skill hierarchy of code reading, tracing, and writing for introductory programming students. Further support for this hierarchy could help computer science educators sequence course content to best build student programming skill. Objective This study aims to replicate a slightly simplified hierarchy of skills in CS1 using a larger body of students (600+ vs. 38) in a non-major introductory Python course with computer-based exams. We also explore the validity of other possible hierarchies. Method We collected student score data on 4 kinds of exam questions. Structural equation modeling was used to derive the hierarchy for each exam. Findings We find multiple best-fitting structural models. The original hierarchy does not appear among the “best” candidates, but similar models do. We also determined that our methods provide us with correlations between skills and do not answer a more fundamental question: what is the ideal teaching order for these skills? Implications This modeling work is valuable for understanding the possible correlations between fundamental code-related skills. However, analyzing student performance on these skills at a moment in time is not sufficient to determine teaching order. We present possible study designs for exploring this more actionable research question.  more » « less
Award ID(s):
2121424
NSF-PAR ID:
10340072
Author(s) / Creator(s):
; ; ; ; ;
Editor(s):
Dorn, Brian; Vahrenhold, Jan
Date Published:
Journal Name:
Computer Science Education
ISSN:
0899-3408
Page Range / eLocation ID:
1 to 29
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Engineers must understand how to build, apply, and adapt various types of models in order to be successful. Throughout undergraduate engineering education, modeling is fundamental for many core concepts, though it is rarely explicitly taught. There are many benefits to explicitly teaching modeling, particularly in the first years of an engineering program. The research questions that drove this study are: (1) How do students’ solutions to a complex, open-ended problem (both written and coded solutions) develop over the course of multiple submissions? and (2) How do these developments compare across groups of students that did and did not participate in a course centered around modeling?. Students’ solutions to an open-ended problem across multiple sections of an introductory programming course were explored. These sections were all divided across two groups: (1) experimental group - these sections discussed and utilized mathematical and computational models explicitly throughout the course, and (2) comparison group - these sections focused on developing algorithms and writing code with a more traditional approach. All sections required students to complete a common open-ended problem that consisted of two versions of the problem (the first version with smaller data set and the other a larger data set). Each version had two submissions – (1) a mathematical model or algorithm (i.e. students’ written solution potentially with tables and figures) and (2) a computational model or program (i.e. students’ MATLAB code). The students’ solutions were graded by student graders after completing two required training sessions that consisted of assessing multiple sample student solutions using the rubrics to ensure consistency across grading. The resulting assessments of students’ works based on the rubrics were analyzed to identify patterns students’ submissions and comparisons across sections. The results identified differences existing in the mathematical and computational model development between students from the experimental and comparison groups. The students in the experimental group were able to better address the complexity of the problem. Most groups demonstrated similar levels and types of change across the submissions for the other dimensions related to the purpose of model components, addressing the users’ anticipated needs, and communicating their solutions. These findings help inform other researchers and instructors how to help students develop mathematical and computational modeling skills, especially in a programming course. This work is part of a larger NSF study about the impact of varying levels of modeling interventions related to different types of models on students’ awareness of different types of models and their applications, as well as their ability to apply and develop different types of models. 
    more » « less
  2. Engineers must understand how to build, apply, and adapt various types of models in order to be successful. Throughout undergraduate engineering education, modeling is fundamental for many core concepts, though it is rarely explicitly taught. There are many benefits to explicitly teaching modeling, particularly in the first years of an engineering program. The research questions that drove this study are: (1) How do students’ solutions to a complex, open-ended problem (both written and coded solutions) develop over the course of multiple submissions? and (2) How do these developments compare across groups of students that did and did not participate in a course centered around modeling?. Students’ solutions to an open-ended problem across multiple sections of an introductory programming course were explored. These sections were all divided across two groups: (1) experimental group - these sections discussed and utilized mathematical and computational models explicitly throughout the course, and (2) comparison group - these sections focused on developing algorithms and writing code with a more traditional approach. All sections required students to complete a common open-ended problem that consisted of two versions of the problem (the first version with smaller data set and the other a larger data set). Each version had two submissions – (1) a mathematical model or algorithm (i.e. students’ written solution potentially with tables and figures) and (2) a computational model or program (i.e. students’ MATLAB code). The students’ solutions were graded by student graders after completing two required training sessions that consisted of assessing multiple sample student solutions using the rubrics to ensure consistency across grading. The resulting assessments of students’ works based on the rubrics were analyzed to identify patterns students’ submissions and comparisons across sections. The results identified differences existing in the mathematical and computational model development between students from the experimental and comparison groups. The students in the experimental group were able to better address the complexity of the problem. Most groups demonstrated similar levels and types of change across the submissions for the other dimensions related to the purpose of model components, addressing the users’ anticipated needs, and communicating their solutions. These findings help inform other researchers and instructors how to help students develop mathematical and computational modeling skills, especially in a programming course. This work is part of a larger NSF study about the impact of varying levels of modeling interventions related to different types of models on students’ awareness of different types of models and their applications, as well as their ability to apply and develop different types of models. 
    more » « less
  3. null (Ed.)
    To meet the rising demand for computer science (CS) courses, K-12 educators need to be prepared to teach introductory concepts and skills in courses such as Computer Science Principles (CSP), which takes a breadth-first approach to CS and includes topics beyond programming such as data, impacts of computing, and networks. Educators are now also being asked to teach more advanced concepts in courses such as the College Board's Advanced Placement Computer Science A (CSA) course, which focuses on advanced programming using Java and includes topics such as objects, inheritance, arrays, and recursion. Traditional CSA curricula have not used content or pedagogy designed to engage a broad range of learners and support their success. Unlike CSP, which is attracting more underrepresented students to computing as it was designed, CSA continues to enroll mostly male, white, and Asian students [College Board 2019, Ericson 2020, Sax 2020]. In order to expand CS education opportunities, it is crucial that students have an engaging experience in CSA similar to CSP. Well-designed differentiated professional development (PD) that focuses on content and pedagogy is necessary to meet individual teacher needs, to successfully build teacher skills and confidence to teach CSA, and to improve engagement with students [Darling-Hammond 2017]. It is critical that as more CS opportunities and courses are developed, teachers remain engaged with their own learning in order to build their content knowledge and refine their teaching practice [CSTA 2020]. CSAwesome, developed and piloted in 2019, offers a College Board endorsed AP CSA curriculum and PD focused on supporting the transition of teachers and students from CSP to CSA. This poster presents preliminary findings aimed at exploring the supports and challenges new-to-CSA high school level educators face when transitioning from teaching an introductory, breadth-first course such as CSP to teaching the more challenging, programming-focused CSA course. Five teachers who completed the online CSAwesome summer 2020 PD completed interviews in spring 2021. The project employed an inductive coding scheme to analyze interview transcriptions and qualitative notes from teachers about their experiences learning, teaching, and implementing CSP and CSA curricula. Initial findings suggest that teachers’ experience in the CSAwesome PD may improve their confidence in teaching CSA, ability to effectively use inclusive teaching practices, ability to empathize with their students, problem-solving skills, and motivation to persist when faced with challenges and difficulties. Teachers noted how the CSAwesome PD provided them with a student perspective and increased feelings of empathy. Participants spoke about the implications of the COVID-19 pandemic on their own learning, student learning, and teaching style. Teachers enter the PD with many different backgrounds, CS experience levels, and strengths, however, new-to-CSA teachers require further PD on content and pedagogy to transition between CSP and CSA. Initial results suggest that the CSAwesome PD may have an impact on long-term teacher development as new-to-CSA teachers who participated indicated a positive impact on their teaching practices, ideologies, and pedagogies. 
    more » « less
  4. Security is a critical aspect in the design, development, and testing of software systems. Due to the increasing need for security-related skills within software systems and engineering, there is a growing demand for these skills to be taught at the university level. A series of 41 security modules was developed to assess the impact of these modules on teaching critical cyber security topics to students. This paper presents the implementation and outcomes of the first set of six security modules in a Freshman level course. This set consists of five modules presented in lectures as well as a sixth module emphasizing encryption and decryption used as the semester project for the course. Each module is a collection of concepts related to cyber security. The individual cyber security concepts are presented with a general description of a security issue to avoid, sample code with the security issue written in the Java programming language, and a second version of the code with an effective solution. The set of these modules was implemented in Computer Science I during the Fall 2019 semester. Incorporating each of the concepts in these modules into lectures depends on both the topic covered and the approach to resolving the related security issue. Students were introduced to computing concepts related to both the security issue and the appropriate solution to fully grasp the overall concept. After presenting the materials to students, continual review with students is also essential. This reviewal process requires exploring use-cases for the programming mechanisms presented as solutions to the security issues discussed. In addition to the security modules presented in lectures, students were given a hands-on approach to understanding the concepts through Model-Eliciting Activities (MEAs). MEAs are open-ended, problem-solving activities in which groups of three to four students work to solve realistic complex problems in a classroom setting. The semester project related to encryption and decryption was implemented into the course as an MEA. To assess the effectiveness of incorporating security modules with the MEA project into the curriculum of Computer Science I, two sections of the course were used as a control group and a treatment group. The treatment group included the security modules in lectures and the MEA project while the control group did not. To measure the overall effectiveness of incorporating security modules with the MEA project, both the instructor’s effectiveness as well as the student’s attitudes and interest were measured. For instructors, the primary question to address was to what extent do instructors change their attitudes towards student learning and their teaching practices because of the implementation of cyber security modules through MEAs. For students, the primary question to address was how the inclusion of security modules with the MEA project improved their understanding of the course materials and their interests in computer science. After implementing security modules with the MEA project, students showed a better understanding of cyber security concepts and a greater interest in broader computer science concepts. The instructor’s beliefs about teaching, learning, and assessment shifted from teacher-centered to student-centered, during his experience with the security modules and MEA. 
    more » « less
  5. This Work-in-Progress paper in the Research Category uses a retrospective mixed-methods study to better understand the factors that mediate learning of computational modeling by life scientists. Key stakeholders, including leading scientists, universities and funding agencies, have promoted computational modeling to enable life sciences research and improve the translation of genetic and molecular biology high- throughput data into clinical results. Software platforms to facilitate computational modeling by biologists who lack advanced mathematical or programming skills have had some success, but none has achieved widespread use among life scientists. Because computational modeling is a core engineering skill of value to other STEM fields, it is critical for engineering and computer science educators to consider how we help students from across STEM disciplines learn computational modeling. Currently we lack sufficient research on how best to help life scientists learn computational modeling. To address this gap, in 2017, we observed a short-format summer course designed for life scientists to learn computational modeling. The course used a simulation environment designed to lower programming barriers. We used semi-structured interviews to understand students' experiences while taking the course and in applying computational modeling after the course. We conducted interviews with graduate students and post- doctoral researchers who had completed the course. We also interviewed students who took the course between 2010 and 2013. Among these past attendees, we selected equal numbers of interview subjects who had and had not successfully published journal articles that incorporated computational modeling. This Work-in-Progress paper applies social cognitive theory to analyze the motivations of life scientists who seek training in computational modeling and their attitudes towards computational modeling. Additionally, we identify important social and environmental variables that influence successful application of computational modeling after course completion. The findings from this study may therefore help us educate biomedical and biological engineering students more effectively. Although this study focuses on life scientists, its findings can inform engineering and computer science education more broadly. Insights from this study may be especially useful in aiding incoming engineering and computer science students who do not have advanced mathematical or programming skills and in preparing undergraduate engineering students for collaborative work with life scientists. 
    more » « less