<?xml-model href='http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng' schematypens='http://relaxng.org/ns/structure/1.0'?><TEI xmlns="http://www.tei-c.org/ns/1.0">
	<teiHeader>
		<fileDesc>
			<titleStmt><title level='a'>Eliciting Student Scratch Script Understandings via Scratch Charades</title></titleStmt>
			<publicationStmt>
				<publisher></publisher>
				<date>02/26/2020</date>
			</publicationStmt>
			<sourceDesc>
				<bibl> 
					<idno type="par_id">10158761</idno>
					<idno type="doi">10.1145/3328778.3366911</idno>
					<title level='j'>SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education</title>
<idno></idno>
<biblScope unit="volume"></biblScope>
<biblScope unit="issue"></biblScope>					

					<author>Diana Franklin</author><author>Jean Salac</author><author>Cathy Thomas</author><author>Zene Sekou</author><author>Sue Krause</author>
				</bibl>
			</sourceDesc>
		</fileDesc>
		<profileDesc>
			<abstract><ab><![CDATA[With many school districts nationwide integrating Computer Science (CS) and Computational Thinking (CT) instruction at the K-8 level, it is crucial researchers closely inspect the relationship between program expression and student understandings. In this study, we propose and report on our use of Scratch Charades, a game in which students act out Scratch scripts while others build them. The purpose of Scratch Charades is to familiarize students with scripts and blocks without the cognitive overhead of the complex user interface. However, in this study, we also used it to elicit student understandings about Scratch blocks and scripts to design mnemonics to help students debug their code. We propose two building and/or debugging strategies based on our observations.]]></ab></abstract>
		</profileDesc>
	</teiHeader>
	<text><body xmlns="http://www.tei-c.org/ns/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink">
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="1">INTRODUCTION</head><p>Momentum has been building for integrating computer science into elementary school classrooms. Providing access to computing curricula is just one part of the solution. It is critical for computer science instruction to be effective for a broad spectrum of students. Diverse learners significantly underperform white peers with higher socio-economic status on important academic markers <ref type="bibr">[4,</ref><ref type="bibr">14,</ref><ref type="bibr">15]</ref>. Recent work <ref type="bibr">[18]</ref> has shown strong correlations between overall school academic performance and learning in a computer science curriculum built on open-ended projects designed using a Constructionist pedagogical approach <ref type="bibr">[7]</ref>.</p><p>Computational Thinking instruction in elementary school often uses Scratch, a visual block-based language with an integrated development environment <ref type="bibr">[11]</ref>. Each command or instruction is called a block, and it can be dragged into code sequences called scripts. Each script begins with an event. For any Scratch learning experience, there are three distinct but related components: the Scratch language, the Scratch development environment, and the curriculum. In this paper, we focus on the Scratch language. In particular, this study has two purposes.</p><p>First, this study introduces our first experience with Scratch Charades, designed to allow students to learn the Scratch programming language outside of the Scratch development environment. While similar to existing methods of demonstrating how code works through human demonstration, it gamifies this to allow students to engage in the activity in a different way than a demonstration.</p><p>Second, observations of students playing the game were used to inform the design of instructional supports for struggling learners. In order to improve instruction for students who need more support, it is important to find out what aspects of Scratch are sources of confusion, defined as a mismatch between what students think a script should do and what a script actually does. If we can elicit students' early impressions of blocks and scripts, this can guide the development of activities, teacher materials, and debugging strategies. Activities can be developed that hone in on specific common issues to highlight how the scripts actually work. Teacher materials can highlight common student mistakes, allowing them to anticipate and recognize the source of specific problems. Finally, debugging strategies can be created that explicitly direct students to look for common mistakes. More precisely, we use Scratch Charades to answer this question: "For what Scratch blocks or code do novice students' interpretations diverge from the Scratch implementation? And in what ways?"</p><p>In this study, we use Scratch Charades, a Scratch-based Charades game, to elicit early interpretation of Scratch blocks and scripts. We focus both on students who act out given scripts and who construct the scripts for code they see acted out. Our experiments identified several common mismatches between student interpretations and Scratch implementation at a block, loop, and script level. And, while experienced instructors will be familiar with these mistakes, this identifies them in a methodical manner and uses those typical mistakes to create building / debugging strategies to help students.</p><p>The contributions of this paper are:</p><p>&#8226; The introduction and refinement of Scratch Charades, a formalized game students can play with each other with specific student roles and materials to introduce new concepts prior to use in the Scratch programming environment. &#8226; Identification of differences in interpretation of the meaning of Scratch blocks between students and Scratch. &#8226; Identification of differences in interpretation of the meaning of Scratch repeat loops between students and Scratch. &#8226; Mistakes that students make in building sequential and loopbased scripts. &#8226; Creation of two building and/or debugging strategies based on the mistakes observed.</p><p>We find that some of the most common sources of confusion are:</p><p>&#8226; What point in direction does &#8226; Which blocks' actions persist and which "reset"</p><p>&#8226; The timing relationship between consecutive blocks &#8226; How to interpret multi-block repeat loops</p><p>The rest of the paper is organized as follows. We next present Scratch Charades itself, followed by background, related work (Section 3) and theoretical framework (Section 4). Section 5 contains the methods. In results, Section 6, we present both the observations and the debugging strategies created from those results. Finally, we describe future work and conclusions in Section 7 and 8.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2">SCRATCH CHARADES</head><p>A version of Scratch Charades has been played in classrooms around the world. As a demonstration, one person acts out a script in order to help them better understand the connection between the blocks and the actions that result from them. The difference between our formalized Scratch Charades game and typical similar demonstrations is that during demonstrations, the audience can see the script being acted out. In this game version, only the actor sees the script -the audience members are active game participants, using blocks to build the scripts they see in order to make explicit any lack of understanding.</p><p>Like in Charades, one person is responsible for acting while others attempt to guess what they are acting out. Two roles were defined: actor, whose goal is to correctly act out a given Scratch script, and builders, whose goal is to recreate the script they saw acted out using LEGO Scratch blocks. Students were placed into groups of three, with the a single actor and two builders each round. The actor role rotated each round.</p><p>In each round, the actor draws the top card from a stack of cards ordered by difficulty (see Figure <ref type="figure">1</ref>). The actor acts out the script on that card. The actor has a "Hello!" say bubble card, a green flag card, and their body for acting out commands. Four signs are attached to the four walls of the classroom, with 0&#176;, 90&#176;, 180&#176;, and -90&#176;hung up to correspond with the four directions on the point in direction blocks.</p><p>The builders have a set of LEGOs, each with a Scratch block sticker on it. They build the script and ask the actor to check it. The builders continue until their script matches the one on the actor's card. Actors can act the script out multiple times in order to assist the builders.</p><p>Blocks chosen for Scratch Charades are shown in Table <ref type="table">2</ref>. These are specifically chosen because they are very useful in simple Scratch projects and they can be acted out (unlike change costume). However, some do involve concepts not yet covered in 4th grade, so we chose what we believed would be the simplest form of each action. For example, turning is very useful. There are two turns to choose from: Turn a number of degrees or point to an absolute location. Because 4th grade students may not have learned degrees yet, we chose point to and placed signs on the wall indicating the location. To make a sprite smaller, there are also two choices: change size by (negative number) and set size to (percentage). Because neither negative numbers nor percentages have necessarily been covered by 4th grade, we chose change size by with light instruction on negative being smaller and positive being larger.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3">BACKGROUND AND RELATED WORK</head><p>Computational Thinking instruction at the elementary-school level often uses Scratch, a visual block-based language with an integrated development environment. Scratch was designed with the philosophy of "low floors, high ceilings, " which is intended to allow students with very little background to create projects quickly (low floors) but have the complexity to provide opportunities for deep learning through very complex projects (high ceilings). Designed to support a Constructionist philosophy [7], students in many different settings and age groups have successfully created projects with Scratch <ref type="bibr">[17,</ref><ref type="bibr">3]</ref>. We also know that some students are able to create projects but not yet understand the underlying code <ref type="bibr">[1]</ref>. For any Scratch learning experience, there are three distinct but related components: the Scratch language, the Scratch development environment, and the curriculum.</p><p>For example, some challenges, like initialization, is affected by the programming environment itself, and the lack of a single "starting point" for a program as well as beginning in the same state as the prior execution ended <ref type="bibr">[5]</ref> or the role of user input <ref type="bibr">[20]</ref>. Some challenges may be conceptual, separate from Scratch, such as loops, sequence, or variables <ref type="bibr">[6,</ref><ref type="bibr">18,</ref><ref type="bibr">20,</ref><ref type="bibr">8]</ref>. Finally, the language itself can be challenging, with some blocks or aspects of the Scratch interface above grade level for students on the younger end of the recommended grade level <ref type="bibr">[9]</ref>.</p><p>In this study, we remove the Scratch programming environment in order to focus on the language itself and capture students' initial understandings through their actions.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4">THEORETICAL FRAMEWORK</head><p>In our theoretical framework, we draw from two bodies of work. First, we view the programming language as the learning interface for the student. Second, we utilize Scratch Charades as a window into students' thinking.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.1">Language as Learning Interface</head><p>Programming language design has increasingly been viewed as an HCI endeavor, especially in the area of computer science learners. As Andy Ko posited in 2014, "Programming languages are the least usable, but most powerful human-computer interfaces ever invented" <ref type="bibr">[10]</ref>. The Quorum programming language was specifically designed for the user, using empirical studies on novice students to choose syntax <ref type="bibr">[19]</ref>.</p><p>This view is supported by Constructivism, which posits that all learners interpret new knowledge through the lens of their own understanding <ref type="bibr">[16]</ref>. This can sometimes be a benefit, utilizing prior knowledge to gain new knowledge, as with analogies. However, this can be a hindrance if the new construct is similar to existing knowledge but with important distinctions.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.2">Manipulatives</head><p>Using manipulatives allows students to explore and express their thinking <ref type="bibr">[13]</ref>. Diverse students may struggle to explain orally or express key ideas and concepts in writing <ref type="bibr">[21,</ref><ref type="bibr">22]</ref>. Manipulatives provide a response mode that is observable <ref type="bibr">[2]</ref>, thereby allowing students with limitations in expressive language, either oral or written, to show what they know. As teachers observe students' engagement with manipulatives, student thinking becomes overt, providing teachers with opportunities to scaffold instruction and enhance student learning. Manipulatives may also support students in recognizing and negotiating their own misconceptions <ref type="bibr">[12]</ref>.  <ref type="table">1</ref>: Participant School Demographics. The student column is the number of students who gave consent to be observed. Other information is at the school level.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Students Grade(s) Hisp</head></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5">METHODS</head><p>In this study, we used students playing Scratch Charades in order to elicit students' understandings or interpretations of Scratch scripts. In this section, we describe the study design, how observations were performed, and the analysis performed on the observational data.  <ref type="table">1</ref>. There was no specific criteria students had to meet in order to be eligible for observation. In all five classes, all students were given the opportunity to participate but only those whose parents had signed consent forms were observed. Before the study took place, none of the students had instruction on Scratch in the classrooms. However, some students may have been introduced to Scratch in other settings.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5.2">Observations</head><p>Researchers served as observer-participants. That is, their main goal was to observe, but they also interacted with the students. Observers focused on one or two groups and observed several types of interesting events such as interpersonal interactions, debugging strategies, and when actions did not match Scratch blocks. Researchers described each observation in prose, noted whether it related to the actor or builder, the script number, and details on the mismatch. Because observers were monitoring multiple groups, the list may not represent all instances of differences between student interpretations and script execution. Observers also explained concepts to students if the students did not self-correct their mistakes.</p><p>In this paper, we focus on two aspects of game play. First, we analyze the instances when the actor's actions did not match the script or when the builders' interpretation of the code did not match actors' actions. Second, we describe the revisions we have made to Scratch Charades as a result of these trials.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5.3">Analysis</head><p>Data was first filtered to retain only instances of mismatches between actions and scripts, removing observations about interpersonal interactions and debugging strategies. The first classroom of data, two researchers open coded the observations to create a coding scheme. A first-level categorization emerged, sorting observations into individual blocks, sequence, loop, and event categories. Block-level observations are misunderstandings about individual blocks, whereas sequence, loops, and event categories refer to misunderstandings about how blocks interact with each other. Instances within each category were further sorted and merged to create sub-categories. Once the coding was set, the two researchers </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6">RESULTS</head><p>We present our results by high-level category -individual blocks, sequence, loop, and event categories. We present the counts to provide insight into how often different misunderstandings were observed. However, our study was not designed to imply that these are the rates at which misunderstandings of this nature occur in large populations. </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Observation</head></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.1">Individual Blocks</head><p>The two blocks that led to confusion were point in direction and change size by. The most common observation, with 14 instances, was that when acting the point in direction block, students pointed in the correct direction but did not turn their whole body to face that direction. The instructions directed the students to turn their whole body to model the behavior of sprites in Scratch.</p><p>Another common observation was that the student acting did not hold the action of the previous block before moving to the next block. For example, in the sequence in Figure <ref type="figure">2</ref>, students would point to 90 &#8226; and then turn back to the front of the room before moving one step. Instead, they should have pointed to 90 &#8226; and taken one step in that direction. This occurred with the change size by block as well, in which students would "shrink" or "grow" as directed but then return to their normal height before the next action. In Scratch, some blocks have the format do X for Y seconds or play __ until done. Some students seemed to have attached this interpretation to the point and change size blocks even though it was not explicitly directed. It is worth noting that this misconception was only seen with these two blocks. For blocks such as move 1 step, there were no observed instances of a student stepping forward and then stepping back.</p><p>In the change size by and point in direction blocks, students also had difficulty interpreting the arguments. point in direction could direct the student to point to 0&#176;, 90&#176;, 180&#176;, or -90&#176;. change size by could direct the student to change size by 10 (grow larger) or change size by -10 (grow smaller).</p><p>Finally, there were two instances of builders using an incorrect block. In one case, the builders used a move 1 step block instead of the correct change size by -10. In the other case, a student used the block to close the repeat loop as a "stop, " placing it at the end of the script.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.1.1">Individual Blocks Discussion</head><p>. There were three points of confusion related to blocks. We now provide suggestions on how instruction or activities that could help students with these concepts.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Meaning of point in direction.</head><p>-in order to make clear that point in direction is rotation of the sprite, rather than pointing, a teacher could have students do a Scratch Charades activity, pointing out that you point with your nose, not with your finger. An alternative would be to use a 2-d picture without arms on a whiteboard instead of humans. They can have a discussion with students about what it means, with the four directions labeled on the four edges of the whiteboard. However, an important question is whether this misconception is because students acted out the blocks, and whether the interpretation would be clear within Scratch. We believe that this shows the imprecision of this command in general; it is not clear what it would mean for a sun sprite or a flower sprite to point in a specific direction. This is an instance in which the game exposed the misunderstanding, not caused it.</p><p>Change size by parameters. -in order to make it easier for students to act out different sizes, the class as a whole could decide on what "small", "medium", and "large" should look like. Then the teacher could explain that for this activity, change size by with a positive number would make the actor one bigger from where they began, and with a negative number would make them one smaller. They could practice this as a group, having the teacher say commands in different orders and having all students act it out together, one block at a time.</p><p>Resetting actions. -Activities could be created with different block sequences that depict blocks that do and do not reset each time, along with questions or observations that students can make. Students could then categorize instructions as those that stop automatically and those that need another instruction to stop (like say __ vs say for __ seconds).</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.2">Sequence</head><p>Sequence relates to placing blocks together to form an ordered script. The majority of sequence errors were made by builders, with many instances of students placing the correct blocks in an  <ref type="figure">3</ref>: A MESS debugging strategy for sequence errors incorrect order or leaving out a block. For scripts that were ordered incorrectly, the most common occurrence was that students switched the ordering of two blocks. For both categories, the errors did not appear to be related to a specific type of block. There were two instances of students building scripts with the correct sequence of blocks but starting from the bottom rather than the top. This could be due to the use of LEGOs for building the scripts as LEGOs are traditionally built from the bottom up.</p><p>Amongst the actors, there were also instances of incorrect ordering and missing blocks, although these occurred less frequently. Like with the building instances, there was no clear pattern to which blocks were involved in these errors. Performing two actions concurrently was the most common actor mistake. There were three observed instances of students making a sound and changing size at the same time. In another instance, a student changed sized while pointing to 180 &#8226; . There was only one instance not involving changing size in which a student lifted the green flag and stepped forward simultaneously.</p><p>6.2.1 Sequence Discussion. These results have identified several different types of errors students can make in sequential code. The biggest misconception was the belief that two instructions could occur at the same time, which could affect how they build programs or expect them to work. The different sequential mistakes could be made explicit to students and presented as a debugging strategy to give students a more methodical approach and more concrete mistakes to look for when they encounter bugs in their own code. We propose A MESS (See Figure <ref type="figure">3</ref>), which provides a checklist of common mistakes to look for in sequential code.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.3">Loops</head><p>While students understood the general concept of loops, they had different interpretations of the loops syntax, as shown in were placed within the loop. If the loop on the left of Figure <ref type="figure">4</ref> was the one given, the most common alternate interpretation by the actor was that the first instruction should be repeated followed by the second instruction being repeated rather than repeating the whole iteration. A similar (but opposite) phenomenon was seen in builders -when the actor acted out the loop correctly (with the two instructions alternated four times), some builders created separate loops.</p><p>Students also got confused about what instructions were inside versus outside the loop. Some students repeated instructions that were after the loop, and others repeated only the first instruction, effectively placing the second instruction out of the loop.</p><p>Builders had a challenge that does not exist in the interface with Scratch -the LEGO top and bottom of the repeat block were separate LEGOs, so they needed to place both of them for a single conceptual instruction. Many students forgot to place the bottom piece or placed it in the wrong location (above the beginning).</p><p>6.3.1 Loops Discussion. These results have identified several different types of errors students can make when constructing loops. Because of the complexity of loops, it might be worth not only presenting a debugging strategy but also having this be a building or planning strategy for loops. We propose Loop BASICs (See Figure <ref type="figure">5</ref>), which provides a checklist of common mistakes to look for.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.4">Events</head><p>Event misconceptions are those involving two Scratch event blocks included in this activity: when green flag clicked and when this sprite clicked. Both actors and builders had cases where a starting event was not included, or where the wrong starting event was used. Instances where an event was not included possibly indicates a lack of understanding among the students that an event is necessary or, more generally, that actions in programs need something to trigger them.</p><p>6.4.1 Events Discussion. The original Scratch Charades rules do not properly handle events. In Scratch, a sprite does not do its own starting event. Instead, a user performs the starting event. In order to make the role of events more clear, we are changing the game play rules.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.5">Scratch Charades challenges</head><p>We now present informal feedback from observers that was outside of the formal protocol but, nonetheless, led to revisions for Scratch Charades for this academic year.</p><p>Dexterity insufficient for LEGOs. Students had great difficulty taking LEGOs apart. Some figured out that if they placed the LE-GOs in a stair-step fashion, it was easier to take them apart. Unfortunately, this counteracts the goal of making the scripts look like Scratch scripts. Scratch uses offsetting the blocks to visually represent placing blocks inside loops and conditionals (like most type-written, commercially-used languages). Using offsetting for sequential scripts removes meaning from offsetting blocks.</p><p>Building script upwards, not downwards. As described in sequence, one group built the script upwards, just as one would build a house in LEGOs. This results in a script that is the opposite order of a Scratch script.</p><p>Confusion about events. As described in the previous section, some students forgot about the event block or used the wrong one.</p><p>Heavy reliance on observers. Students interacted with observers more than anticipated. This most often consisted of helping students put together and take apart LEGOs as well as helping the actor act out blocks correctly.</p><p>As a result of these observations, we have made three major changes to Scratch Charades.</p><p>Magnets, not LEGOs. Instead of using LEGOs, we are using magnets. These are stiff enough to hold up to lots of play, can stick to the tins in which we store the pieces for easy building, and are easy to place and remove. Added User role. There are now three different roles: Sprite, User, and Builder. We suggest groups of 4-5 students. At any given time, there is one sprite, one user, and at least two builders. This is for two reasons. First, this makes the role of the event explicit. Second, this allows every role to have a partner. The user and sprite can discuss the card and how it should be / was acted, and the builders can discuss how to build it. The hope is that less teacher intervention will be necessary if students can discuss.</p><p>Separate play sessions. The scripts for Scratch Charades have been split into two card sets -sequential cards and loop cards. This way, the sequential cards can be used prior to using Scratch the first time, and the loop cards can be saved until just prior to loops. Loops were substantially more challenging than sequential code. Splitting the cards would allow the introduction of loops to be delayed until students' concepts of sequential code were solid.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="7">FUTURE WORK</head><p>There are two major areas for future work. The first is to experiment with Scratch Charades as a learning strategy to understand whether students learn Scratch more quickly or with less confusion if they play Scratch Charades prior to using the Scratch development environment. The second is to evaluate the use of the A MESS and Loop BASICs debugging / building strategies. Do students use these when looking for mistakes? Can they be used as a script to help each other, allowing them to collaboratively problem solve rather than needing individual help from the teacher?</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="8">CONCLUSIONS</head><p>We have found that while students understand the concept of sequential and loop-based code very easily, a few commonly-used Scratch blocks can lead to confusion, and students may not understand the details of loop syntax. In this paper, we presented interpretations observed in student enactments of simple Scratch scripts that diverge from Scratch implementation, along with proposed guidance for mitigating the common mistakes. We presented two debugging strategies that students can use, one for sequential errors and one for loop-based errors. Finally, we introduced Scratch Charades, a game that can be used to introduce students to these concepts.</p><p>Future work could evaluate the effectiveness of these two debugging strategies, as well as investigate the use of Scratch Charades as an educational intervention for getting students started in Scratch.</p></div></body>
		</text>
</TEI>
