skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Design and Pilot Testing of Subgoal Labeled Worked Examples for Five Core Concepts in CS1
Subgoal learning has improved student problem-solving performance in programming, but it has been tested for only one-to-two hours of instruction at a time. Our work pioneers implementing subgoal learning throughout an entire introductory programming course. In this paper we discuss the protocol that we used to identify subgoals for core programming procedures, present the subgoal labels created for the course, and outline the subgoal-labeled instructional materials that were designed for a Java-based course. To examine the effect of subgoal labeled materials on student performance in the course, we compared quiz and exam grades between students who learned using subgoal labels and those who learned using conventional materials. Initial results indicate that learning with subgoals improves performance on early applications of concepts. Moreover, variance in performance was lower and persistence in the course was higher for students who learned with subgoals compared to those who learned with conventional materials, suggesting that learning with subgoal labels may uniquely benefit students who would normally receive low grades or dropout of the course.  more » « less
Award ID(s):
1712231 1927906
PAR ID:
10171487
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education
Page Range / eLocation ID:
548 to 554
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; (, Proceedings of the 49th ACM Technical Symposium on Computer Science Education)
    null (Ed.)
    Subgoal labels are function-based instructional explanations that describe the problem-solving steps to the learner, highlighting the solution process. There is strong evidence that the use of subgoal labels within worked examples improves student learning in other STEM fields. Initial research shows that using subgoal labels within computer science improves student learning, but this has only been tested using a single programming concept (indefinite loops) with text-based programming languages. The proposers are currently expanding subgoal labels to the main programming concepts taught in an introductory programming course using an imperative programming language. In this BOF we seek to uncover tacit knowledge that programming instructors have in order to develop instructional materials that bridge the gap between students, who are CS novices, and instructors, who are CS experts, to improve learning for students who are under-prepared for or struggle in CS1. We will be seeking feedback on the selection of programming topics to be covered, the defined subgoals for those topics and the worked examples created for instructional purposes. 
    more » « less
  2. ; ; ; (, ACM Transactions on Computing Education)
    null (Ed.)
    The subgoal learning framework has improved performance for novice programmers in higher education, but it has only started to be applied and studied in K-12 (primary/secondary). Programming education in K-12 is growing, and many international initiatives are attempting to increase participation, including curricular initiatives like Computer Science Principles and non-profit organizations like Code.org. Given that subgoal learning is designed to help students with no prior knowledge, we designed and implemented subgoals in the introduction to programming unit in Code.org's Computer Science Principles course. The redesigned unit includes subgoal-oriented instruction and subgoal-themed pre-written comments that students could add to their programming activities. To evaluate efficacy, we compared behaviors and performance of students who received the redesigned subgoal unit to those receiving the original unit. We found that students who learned with subgoals performed better on problem-solving questions but not knowledge-based questions and wrote more in open-ended response questions, including a practice Performance Task for the AP exam. Moreover, at least one-third of subgoal students continued to use the subgoal comments after the subgoal-oriented instruction had been faded, suggesting that they found them useful. Survey data from the teachers suggested that students who struggled with the concepts found the subgoals most useful. Implications for future designs are discussed. 
    more » « less
  3. ; ; (, Proceedings of the 2019 ACM Conference on International Computing Education Research)
    This work extends previous research on subgoal labeled instructions by examining their effect across a semester-long, Java-based CS1 course. Across four quizzes, students were asked to explain in plain English the process that they would use to solve a programming problem. In this mixed methods study, we used the SOLO taxonomy to categorize student responses about problem-solving processes and compare students who learned with subgoal labels to those who did not. The use of the SOLO taxonomy classification allows us to look deeper than the mere correctness of answers to focus on the quality of the answers produced in terms of completeness of relevant concepts and explanation of relationships among concepts. Students who learned with subgoals produced higher-rated answers in terms of complexity and quality on three of four quizzes. Also, they were three times more likely to discuss issues of data type on a question about assignments and expressions than students who did not learn with subgoal labeling. This suggests that the use of subgoal labeling enabled students to gain a deeper and more complex understanding of the material presented in the course. 
    more » « less
  4. ; ; (, Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 2 (SIGCSE 2023))
    Subgoal labeling is an instructional design framework for breaking down problems into pieces that are small enough for novices to grasp, and often difficult for instructors (i.e., experts) to articulate. Subgoal labels have been shown to improve student performance during problem solving in many disciplines, including computing. Improved student performance occurs because subgoal labels improve student transfer and retention of knowledge. With support from NSF (DUE-1712025, 1712231, 1927906, 2110156, 2111578), subgoal labels have been previously identified and integrated into a CS1 course (variables, expressions, conditionals, loops, arrays, classes) and an e-book has been created on the Runestone platform to enable students to complete practice problems using the subgoals. The initial implementation focused on Java, but within the past year, the development of subgoals for CS1 courses in Python have been created. Subsequently, course materials have been created as well. This workshop will introduce participants to the new materials (in Python) and demonstrate how the subgoal labels and worked examples are integrated throughout the course. Materials include worked examples and practice problems that increase in complexity and difficulty within each topic. The materials are designed to be integrated into CS1 courses as homework or classroom examples and activities. Assessment of topics using subgoal labels will also be discussed. Participants will also engage in an activity where they create an example for their own course using subgoal labels. 
    more » « less
  5. (, Proceedings of the 2022 ACM Conference on International Computing Education Research V.2 (ICER 2022))
    In our previous research we found that teaching novice programmers introductory programming in Java using subgoal labels led to deeper knowledge [2] and increased persistence for students potentially at risk of dropping out or failing their first undergraduate course in CS [3]. Given the increasing number of universities using Python in introductory programming classes, we have begun the process of defining the subgoals to use in Python-based introductory courses. We have repeated the Task Analysis by Problem Solving (TAPS) development process [1] that we used to create Java-based instructional materials. As with the Java-based instructional materials, we have created subgoals for both evaluating programs written in Python and writing programs in Python. The poster will present an overview of the TAPS development process for identifying the Python subgoals and the proposed experimental design for assessing the effectiveness of the instructional materials. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email