Worked examples are among the most popular types of learning content in programming classes. However, instructors rarely have time to provide line-by-line explanations for a large number of examples typically used in a programming class. In this paper, we explore and assess a human-AI collaboration approach to authoring worked examples for Java programming. We introduce an authoring system for creating Java worked examples that generate a starting version of code explanations and presents it to the instructor to edit if necessary. We also present a study that assesses the quality of explanations created with this approach.
more »
« less
This content will become publicly available on March 18, 2025
Human-AI Co-Creation of Worked Examples for Programming Classes
Worked examples (solutions to typical programming problems presented as a source code in a certain
language and are used to explain the topics from a programming class) are among the most popular types
of learning content in programming classes. Most approaches and tools for presenting these examples to
students are based on line-by-line explanations of the example code. However, instructors rarely have
time to provide line-by-line explanations for a large number of examples typically used in a programming
class. In this paper, we explore and assess a human-AI collaboration approach to authoring worked
examples for Java programming. We introduce an authoring system for creating Java worked examples
that generates a starting version of code explanations and presents it to the instructor to edit if necessary.
We also present a study that assesses the quality of explanations created with this approach.
more »
« less
- Award ID(s):
- 2213789
- NSF-PAR ID:
- 10518251
- Publisher / Repository:
- CEUR
- Date Published:
- Journal Name:
- Proceedings of 5th Workshop on Human-AI Co-Creation with Generative Models (HA-GEN 20224) at IUI 2024
- Subject(s) / Keyword(s):
- Code Examples Authoring Tool Human-AI Collaboration
- Format(s):
- Medium: X
- Location:
- Greenville, SC, USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Worked examples, which present an explained code for solving typical programming problems are among the most popular types of learning content in programming classes. Most approaches and tools for presenting these examples to students are based on line-by-line explanations of the example code. However, instructors rarely have time to provide explanations for many examples typically used in a programming class. In this paper, we assess the feasibility of using LLMs to generate code explanations for passive and active example exploration systems. To achieve this goal, we compare the code explanations generated by chatGPT with the explanations generated by both experts and students.more » « less
-
This paper systematically explores how Large Language Models (LLMs) generate explanations of code examples of the type used in intro-to-programming courses. As we show, the nature of code explanations generated by LLMs varies considerably based on the wording of the prompt, the target code examples being explained, the programming language, the temperature parameter, and the version of the LLM. Nevertheless, they are consistent in two major respects for Java and Python: the readability level, which hovers around 7-8 grade, and lexical density, i.e., the relative size of the meaninful words with respect to the total explanation size. Furthermore, the explanations score very high in correctness but less on three other metrics: completeness, conciseness, and contextualization.more » « less
-
Crossley, Scott ; Popescu, Elvira (Ed.)We present here a novel instructional resource, called DeepCode, to support deep code comprehension and learning in intro-to-programming courses (CS1 and CS2). DeepCode is a set of instructional code examples which we call a codeset and which was annotated by our team with comments (e.g., explaining the logical steps of the underlying problem being solved) and related instructional questions that can play the role of hints meant to help learners think about and articulate explanations of the code. While DeepCode was designed primarily to serve our larger efforts of developing an intelligent tutoring system (ITS) that fosters the monitoring, assessment, and development of code comprehension skills for students learning to program, the codeset can be used for other purposes such as assessment, problem-solving, and in various other learning activities such as studying worked-out code examples with explanations and code visualizations. We present here the underlying principles, theories, and frameworks behind our design process, the annotation guidelines, and summarize the resulting codeset of 98 annotated Java code examples which include 7,157 lines of code (including comments), 260 logical steps, 260 logical step details, 408 statement level comments, and 590 scaffolding questions.more » « less
-
The paper focuses on a new type of interactive learning content for SQL programming - worked examples of SQL code. While worked examples are popular in learning programming, their application for learning SQL is limited. Using a novel tool for presenting interactive worked examples, Database Query Analyzer (DBQA), we performed a large-scale randomized controlled study assessing the value of worked examples as a new type of practice content in a database course. We report the results of the classroom study examining the usage and the impact of DBQA. Among other aspects, we explored the effect of textual step explanations provided by DBQA.more » « less
