skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Board 196: A Framework to Assess Debugging Skills for Computational Thinking in Science and Engineering
A rubric is presented to assess debugging skills for students particularly in the natural sciences and engineering. The three categories that are assessed for the cognitive processes in debugging skills are identification, isolation, and iteration. These are defined, and the characteristics of each process are listed. We discuss the method used to develop this rubric that was based on intentional errors in a programming assignment given to students in an introductory physics course. The programming in this assignment was in Python and a visual-based programming platform, called iFlow. We believe that visual-based programming will help elicit weaknesses in debugging because it removes students' familiarity with particular programming languages. Our focus on debugging skills came from a survey of students to self-identify barriers in computational work in an introductory physics course that included engineering majors. This skill was the primary self-identified barrier along with abstraction skills, which will be the focus of another work. We also present the results of this survey. The Python assignment (ntext = 9) was used to create the rubric and the iFlow assignment (ngraphic = 11) was used to test the rubric. Scoring was based on a scale of six levels in each category. Although the sample size was too small to establish rigorous scoring reliability, we discussed how the two researchers attained agreement in scoring the assignments after iterative modifications of the rubric and rescoring. For the Python assignment, the average for identification was 2.75/5, for isolation 2.30/5, and for iteration 3.33/5. For the iFlow assignment, the average for identification was 2.63/5, for isolation 2.23/5, and for iterate 3.32/5. A consistent trend from these assignments showed that students' approach to debugging is mainly to identify and iterate without a full understanding of the error (i.e., isolation). The lack of a full understanding of the error implies that students are prone to repeat the error. Thus, the important outcome of debugging is to understand the source of error by systematically investigating different parts of the computational solution. Our preliminary results led to the hypothesis that students with weak debugging skills are mainly due the isolation process. This hypothesis will be tested in a future experiment. Results from such an experiment will be significant to those who are designing intervention strategies to integrate computational thinking in science and engineering curricula.  more » « less
Award ID(s):
2107104
PAR ID:
10523429
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
ASEE Conferences
Date Published:
Format(s):
Medium: X
Location:
Baltimore , Maryland
Sponsoring Org:
National Science Foundation
More Like this
  1. Blikstein, P; Van_Aalst, J; Kizito, R; Brennan, K (Ed.)
    Developing assessment tools for computational thinking (CT) in STEM education is a precursor for science teachers to effectively integrate intervention strategies for CT practices. One problem to assessing CT skills is students’ varying familiarity with different programming languages and platforms. A text-neutral, open-source platform called iFlow, is capable of addressing this issue. Specifically, this innovative technology has been adopted to elicit underrepresented undergraduate students’ debugging skills. We present how the visual-based coding platform can be applied to bypass programming language bias in assessing CT. In this preliminary study, we discuss design principles of a visual-based platform to effectively assess debugging practices – identification, isolation, and iteration – with the use of iFlow assignments. Our findings suggest how the ability of iFlow to test parts of a program independently, dataflow connectivity, and equity in removing biases from students’ various backgrounds are advantageous over text-based platforms. 
    more » « less
  2. Computational thinking has widely been recognized as a crucial skill for engineers engaged in problem-solving. Multidisciplinary learning environments such as integrated STEM courses are powerful spaces where computational thinking skills can be cultivated. However, it is not clear the best ways to integrate computational thinking instruction or how students develop computational thinking in those spaces. Thus, we wonder: To what extent does engaging students in integrated engineering design and physics labs impact their development of computational thinking? We have incorporated engineering design within a traditional introductory calculus-based physics lab to promote students’ conceptual understanding of physics while fostering scientific inquiry, mathematical modeling, engineering design, and computational thinking. Using a generic qualitative research approach, we explored the development of computational thinking for six teams when completing an engineering design challenge to propose an algorithm to remotely control an autonomous guided vehicle throughout a warehouse. Across five consecutive lab sessions, teams represented their algorithms using a flowchart, completing four iterations of their initial flowchart. 24 flowcharts were open coded for evidence of four computational thinking facets: decomposition, abstraction, algorithms, and debugging. Our results suggest that students’ initial flowcharts focused on decomposing the problem and abstracting aspects that teams initially found to be more relevant. After each iteration, teams refined their flowcharts using pattern recognition, algorithm design, efficiency, and debugging. The teams would benefit from having more feedback about their understanding of the problem, the relevant physics concepts, and the logic and efficiency of the flowcharts 
    more » « less
  3. Students often make mistakes in their introductory programming assignments as part of their learning process. Unfortunately, providing custom repairs for these mistakes can require a substantial amount of time and effort from class instructors. Automated program repair (APR) techniques can be used to synthesize such fixes. Prior work has explored the use of symbolic and neural techniques for APR in the education domain. Both types of approaches require either substantial engineering efforts or large amounts of data and training. We propose to use a large language model trained on code, such as Codex (a version of GPT), to build an APR system -- PyDex -- for introductory Python programming assignments. Our system can fix both syntactic and semantic mistakes by combining multi-modal prompts, iterative querying, test-case-based selection of few-shots, and program chunking. We evaluate PyDex on 286 real student programs and compare to three baselines, including one that combines a state-of-the-art Python syntax repair engine, BIFI, and a state-of-the-art Python semantic repair engine for student assignments, Refactory. We find that PyDex can fix more programs and produce smaller patches on average. 
    more » « less
  4. null (Ed.)
    Computational thinking is understood as the development of skills and knowledge in how to apply computers and technology to systematically solve problems. Computational thinking has been acknowledged as one key aspect in the taxonomy of engineering education and implied in multiple ABET student outcomes. Moreover, many introductory engineering courses worldwide have a component of programming or computational thinking. A preliminary study of enculturation to the engineering profession found that computational thinking was deemed a critical area of development at the early stages of instruction. No existing computational thinking framework was found to fully meet the needs of engineers, based on the expertise of researchers at three different institutions and the aid of a comprehensive literature review. As a result, a revised version of a computational thinking diagnostic was developed and renamed the engineering computational thinking diagnostic (ECTD). The five computational thinking factors of the ECTD are (1) Abstraction, (2) Algorithmic Thinking and Programming, (3) Data Representation, Organization, and Analysis, (4) Decomposition, and (5) Impact of Computing. This paper describes the development and revisions made to the ECTD using data collected from first-year engineering students at a Southwestern public university. The goal of the development of the ECTD is to capture the entry and exit skill levels of engineering students in an engineering program. 
    more » « less
  5. null (Ed.)
    Computational thinking is understood as the development of skills and knowledge in how to apply computers and technology to systematically solve problems. Computational thinking has been acknowledged as one key aspect in the taxonomy of engineering education and implied in multiple ABET student outcomes. Moreover, many introductory engineering courses worldwide have a component of programming or computational thinking. A preliminary study of enculturation to the engineering profession found that computational thinking was deemed a critical area of development at the early stages of instruction (Mendoza Diaz et al., 2018, 2019; Richard et al., 2016; Wickliff et al., 2018). No existing computational thinking framework was found to fully meet the needs of engineers, based on the expertise of researchers at three different institutions and the aid of a comprehensive literature review. As a result, a revised version of a computational thinking diagnostic was developed and renamed the engineering computational thinking diagnostic (ECTD). The five computational thinking factors of the ECTD are (1) Abstraction, (2) Algorithmic Thinking and Programming, (3) Data Representation, Organization, and Analysis, (4) Decomposition, and (5) Impact of Computing. This paper describes the development and revisions made to the ECTD using data collected from first-year engineering students at a Southwestern public university. The goal of the development of the ECTD is to capture the entry and exit skill levels of engineering students in an engineering program. 
    more » « less