More Like this

1. Representing and Evaluating Strategies for Solving Parsons Puzzles

   Kumar, Amruth N. (May 2019, Intelligent Tutoring Systems)

   Parsons puzzles are popular for programming education. Identifying the strategies used by students solving Parsons puzzles is of interest because they can be used to determine to what extent students use the strategies typically associated with programming expertise, and to provide feedback and monitor the progress of students in a tutor. We propose solution sequence as an approximation of the student’s strategy for solving Parsons puzzles. It is scalable in terms of both the size of the puzzle and the number of students solving the puzzle. We propose BNF grammar to represent desirable puzzle-solving strategies associated with expert programmers. This representation is extensible and agnostic to the puzzle-solving strategies themselves. Finally, we propose a best match parser that matches a student’s solution sequence against the BNF grammar of a desirable strategy and quantifies the degree to which the student’s solution conforms to the desirable strategy. As a proof of concept, we used the parser to analyze the data collected by a Parsons puzzle tutor on if-else statements over five semesters and found a significant difference between C++ and Java puzzle-solvers in terms of their conformance to one desirable puzzle-solving strategy. Being able to tease out the effects of individual components of a strategy is one benefit of our approach: we found that relaxing shell-first constraint in the strategy resulted in significant improvement in the conformance of both C++ and Java students. 

   more » « less
2. Mnemonic Variable Names in Parsons Puzzles

   https://doi.org/10.1145/3300115.3309509  

   Kumar, Amruth N. (January 2019, Proceedings of the ACM Conference on Global Computing Education)

   In Parsons Puzzles, students are asked to arrange the lines of a program in their correct order. We investigated the effect of using mnemonic variable names in the program on the ease with which students solved the puzzles - whether students were able to solve puzzles containing mnemonic variable names with fewer actions or in less time than single-character variable names. We conducted a controlled study with cross-over design over four semesters. Much to our surprise, we found no statistically significant difference between students solving puzzles with mnemonic variable names versus single-character variable names - either in terms of the number of actions taken, the grade earned or the time spent per puzzle. In this paper, we will describe the experimental setup and data analysis and present the results of the study. We will discuss some hypotheses as to why the readability of the variable names did not impact students' ability to solve Parsons puzzles. 

   more » « less
3. Down and Across: Introducing Crossword-Solving as a New NLP Benchmark

   https://doi.org/10.18653/v1/2022.acl-long.189  

   Kulshreshtha, Saurabh; Kovaleva, Olga; Shivagunde, Namrata; Rumshisky, Anna (April 2022, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers))

   Solving crossword puzzles requires diverse reasoning capabilities, access to a vast amount of knowledge about language and the world, and the ability to satisfy the constraints imposed by the structure of the puzzle. In this work, we introduce solving crossword puzzles as a new natural language understanding task. We release a corpus of crossword puzzles collected from the New York Times daily crossword spanning 25 years and comprised of a total of around nine thousand puzzles. These puzzles include a diverse set of clues: historic, factual, word meaning, synonyms/antonyms, fill-in-the-blank, abbreviations, prefixes/suffixes, wordplay, and cross-lingual, as well as clues that depend on the answers to other clues. We separately release the clue-answer pairs from these puzzles as an open-domain question answering dataset containing over half a million unique clue-answer pairs. For the question answering task, our baselines include several sequence-to-sequence and retrieval-based generative models. We also introduce a non-parametric constraint satisfaction baseline for solving the entire crossword puzzle. Finally, we propose an evaluation framework which consists of several complementary performance metrics. 

   more » « less
4. Integrating Personalized Parsons Problems with Multi-Level Textual Explanations to Scaffold Code Writing

   https://doi.org/10.1145/3626253.3635606  

   Hou, Xinying; Ericson, Barbara J; Wang, Xu (March 2024, Proceedings of the 55th ACM Technical Symposium on Computer Science Education (SIGCSE))

   Novice programmers need to write basic code as part of the learning process, but they often face difficulties. To assist struggling students, we recently implemented personalized Parsons problems, which are code puzzles where students arrange blocks of code to solve them, as pop-up scaffolding. Students found them to be more engaging and preferred them for learning, instead of simply receiving the correct answer, such as the response they might get from generative AI tools like ChatGPT. However, a drawback of using Parsons problems as scaffolding is that students may be able to put the code blocks in the correct order without fully understanding the rationale of the correct solution. As a result, the learning benefits of scaffolding are compromised. Can we improve the understanding of personalized Parsons scaffolding by providing textual code explanations? In this poster, we propose a design that incorporates multiple levels of textual explanations for the Parsons problems. This design will be used for future technical evaluations and classroom experiments. These experiments will explore the effectiveness of adding textual explanations to Parsons problems to improve instructional benefits. 

   more » « less
5. Parsons Problems to Scaffold Code Writing: Impact on Performance and Problem-Solving Efficiency

   https://doi.org/10.1145/3587103.3594182  

   Hou, Xinying; Ericson, Barbara Jane; Wang, Xu (June 2023, Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education (ITiCSE))

   Novice programmers struggle with writing code from scratch. One possible way to help them is by using an equivalent Parsons problem on demand, where learners place mixed-up code blocks in the correct order. In a classroom study with 89 undergraduate students, we examined how using a Parsons problem as scaffolding impacts performance and problem-solving efficiency. Results showed that students in the Parsons as Help group achieved significantly higher practice performance and problem-solving efficiency than students who wrote code without help, while achieving the same level of posttest scores. These results improve the understanding of Parsons problems and contribute to the design of future coding practices. 

   more » « less