Search for: All records

Though the increased availability of Large Language Models (LLMs) presents signi!cant potential for change in the way students learn to program, the text-based nature of the available tools currently preclude block-based languages from much of that innovation. In an attempt to remedy this, we identify the strengths and weaknesses of using a transpiler to leverage the existing learning in commercially available LLMs and Scratch, a visual block-based programming language. Using only prompt engineering, we evaluate an LLM’s performance on two common classroom tasks in a Scratch curriculum. We evaluate the LLM’s ability to: 1) Create project solutions that compile and satisfy project requirements and 2) Analyze student projects’ completion of project requirements using natural language. In both cases, we !nd results indicating that prompt-engineering alone is insu"cient to reliably produce high-quality results. For projects of medium complexity, the LLM-generated solutions con- sistently failed to follow correct syntax or, in the few instances with correct syntax, produce correct solutions. When used for auto- grading, we found a correlation between scores assigned by the official Scratch Encore autograder and those generated by the LLM, nevertheless the discrepancies between the ‘real’ scores and the scores assigned by the LLM remained too great for the tool to be reliable in a classroom setting. 

According to an ecological affordances perspective, any static curriculum has a set of affordances, and differences in teachers, students, and the teaching environment change how those affordances are viewed and used. Therefore, teaching is a relationship between the curriculum, the teacher, and the students. As such, it is not only possible but expected that a teacher will diverge from the details of a lesson plan to better accommodate the needs of themselves as a teacher and their students as learners. In this study, we report on a mixed-methods investigation that explores the different ways upper-elementary and middle-school (7- 13 y.o. students) teachers implement the Scratch-based TIPP&SEE learning strategy and the reasoning for their approaches. As expected, we find that teachers across grade levels often deviate from lesson plan details to cater to their own classrooms. For example, teachers serving younger grades were far more likely to keep scaffolds that lesson plans suggest removing. The varied degree of deviation suggests that the repeated use of a learning strategy, alongside lesson plans that present a variety of scaffolded implementations, is beneficial in enabling teachers to adapt lesson content to serve the needs of their specific classroom.