More Like this

1. Do Students Use Semantics When Solving Parsons Puzzles? – A Log-Based Investigation

   https://doi.org/10.1007/978-3-030-80421-3_49  

   Kumar, Amruth N. (June 2021, Intelligent Tutoring Systems)

   null (Ed.)

   Parsons puzzles are jigsaw puzzles wherein students are given a program in scrambled order and tasked with reassembling the program in its correct order. Do students use program semantics when solving Parsons puzzles? The answer to this question has implications for the use of Parsons puzzles as a pedagogic tool. In order to answer this question, we considered semantics at the level of statements and control-flow. We analyzed the data collected by a Parsons puzzle tutor over 5 semesters and measured the extent to which students’ puzzle-solving behavior conformed with the use of statement-level and control-flow semantics. We found that students used statement-level semantics to assemble up to 73% of the lines in a puzzle and control-flow semantics to assemble up to 47% of the lines. They used statement-level semantics more than control-flow semantics and more on some puzzles than others. Whenever we found a significant difference between C++ and Java students, C++ students used semantics more than Java students. Finally, we did not find an increase in the use of semantics with in-creased practice. 

   more » « less
2. Mnemonic Variable Names in Parsons Puzzles

   https://doi.org/10.1145/3300115.3309509  

   Kumar, Amruth N. (January 2019, Proceedings of the ACM Conference on Global Computing Education)

   In Parsons Puzzles, students are asked to arrange the lines of a program in their correct order. We investigated the effect of using mnemonic variable names in the program on the ease with which students solved the puzzles - whether students were able to solve puzzles containing mnemonic variable names with fewer actions or in less time than single-character variable names. We conducted a controlled study with cross-over design over four semesters. Much to our surprise, we found no statistically significant difference between students solving puzzles with mnemonic variable names versus single-character variable names - either in terms of the number of actions taken, the grade earned or the time spent per puzzle. In this paper, we will describe the experimental setup and data analysis and present the results of the study. We will discuss some hypotheses as to why the readability of the variable names did not impact students' ability to solve Parsons puzzles. 

   more » « less
3. Using Adaptive Parsons Problems to Scaffold Write-Code Problems

   https://doi.org/10.1145/3501385.3543977  

   Hou, Xinying; Ericson, Barbara Jane; Wang, Xu (August 2022, Proceedings of the 2022 ACM Conference on International Computing Education Research)

   In this paper, we explore using Parsons problems to scaffold novice programmers who are struggling while solving write-code problems. Parsons problems, in which students put mixed-up code blocks in order, can be created quickly and already serve thousands of students while other types of programming support methods are expensive to develop or do not scale. We conducted two studies in which novices were given equivalent Parsons problems as optional scaffolding while solving write-code problems. We investigated when, why, and how students used the Parsons problems as well as their perceptions of the benefits and challenges. A think-aloud observational study with 11 undergraduate students showed that students utilized the Parsons problem before writing a solution to get ideas about where to start; during writing a solution when they were stuck; and after writing a solution to debug errors and look for better strategies. Semi-structured interviews with the same 11 undergraduate students provided evidence that using Parsons problems to scaffold write-code problems helped students to reduce the difficulty, reduce the problem completion time, learn problem-solving strategies, and refine their programming knowledge. However, some students found them less useful if the Parsons solution did not match their approach or if they did not understand the solution. We then conducted a between-subjects classroom study with 81 undergraduate students to investigate the effects on learning. We found that students who received Parsons problems as scaffolding during write-code problems spent significantly less time solving those problems. However, there was no significant learning gain in either condition from pretest to posttest. We also discuss the design implications of our findings. 

   more » « less
4. Understanding the Effects of Using Parsons Problems to Scaffold Code Writing for Students with Varying CS Self-Efficacy Levels

   https://doi.org/10.1145/3631802.3631832  

   Hou, Xinying; Ericson, Barbara Jane; Wang, Xu (November 2023, Proceedings of the 23rd Koli Calling International Conference on Computing Education Research (Koli Calling))

   Introductory programming courses aim to teach students to write code independently. However, transitioning from studying worked examples to generating their own code is often difficult and frustrating for students, especially those with lower CS self-efficacy in general. Therefore, we investigated the impact of using Parsons problems as a code-writing scaffold for students with varying levels of CS self-efficacy. Parsons problems are programming tasks where students arrange mixed-up code blocks in the correct order. We conducted a between-subjects study with undergraduate students (N=89) on a topic where students have limited code-writing expertise. Students were randomly assigned to one of two conditions. Students in one condition practiced writing code without any scaffolding, while students in the other condition were provided with scaffolding in the form of an equivalent Parsons problem. We found that, for students with low CS self-efficacy levels, those who received scaffolding achieved significantly higher practice performance and in-practice problem-solving efficiency compared to those without any scaffolding. Furthermore, when given Parsons problems as scaffolding during practice, students with lower CS selfefficacy were more likely to solve them. In addition, students with higher pre-practice knowledge on the topic were more likely to effectively use the Parsons scaffolding. This study provides evidence for the benefits of using Parsons problems to scaffold students’ write-code activities. It also has implications for optimizing the Parsons scaffolding experience for students, including providing personalized and adaptive Parsons problems based on the student’s current problem-solving status. 

   more » « less
5. Down and Across: Introducing Crossword-Solving as a New NLP Benchmark

   https://doi.org/10.18653/v1/2022.acl-long.189  

   Kulshreshtha, Saurabh; Kovaleva, Olga; Shivagunde, Namrata; Rumshisky, Anna (April 2022, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers))

   Solving crossword puzzles requires diverse reasoning capabilities, access to a vast amount of knowledge about language and the world, and the ability to satisfy the constraints imposed by the structure of the puzzle. In this work, we introduce solving crossword puzzles as a new natural language understanding task. We release a corpus of crossword puzzles collected from the New York Times daily crossword spanning 25 years and comprised of a total of around nine thousand puzzles. These puzzles include a diverse set of clues: historic, factual, word meaning, synonyms/antonyms, fill-in-the-blank, abbreviations, prefixes/suffixes, wordplay, and cross-lingual, as well as clues that depend on the answers to other clues. We separately release the clue-answer pairs from these puzzles as an open-domain question answering dataset containing over half a million unique clue-answer pairs. For the question answering task, our baselines include several sequence-to-sequence and retrieval-based generative models. We also introduce a non-parametric constraint satisfaction baseline for solving the entire crossword puzzle. Finally, we propose an evaluation framework which consists of several complementary performance metrics. 

   more » « less