skip to main content


Title: PuzzleMe: Leveraging Peer Assessment for In-Class Programming Exercises
Peer assessment, as a form of collaborative learning, can engage students in active learning and improve their learning gains. However, current teaching platforms and programming environments provide little support to integrate peer assessment for in-class programming exercises. We identified challenges in conducting such exercises and adopting peer assessment through formative interviews with instructors of introductory programming courses. To address these challenges, we introduce PuzzleMe, a tool to help Computer Science instructors to conduct engaging in-class programming exercises. PuzzleMe leverages peer assessment to support a collaboration model where students provide timely feedback on their peers' work. We propose two assessment techniques tailored to in-class programming exercises: live peer testing and live peer code review. Live peer testing can improve students' code robustness by allowing them to create and share lightweight tests with peers. Live peer code review can improve code understanding by intelligently grouping students to maximize meaningful code reviews. A two-week deployment study revealed that PuzzleMe encourages students to write useful test cases, identify code problems, correct misunderstandings, and learn a diverse set of problem-solving approaches from peers.  more » « less
Award ID(s):
1915515
NSF-PAR ID:
10360850
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Proceedings of the ACM on Human-Computer Interaction
Volume:
5
Issue:
CSCW2
ISSN:
2573-0142
Page Range / eLocation ID:
1 to 24
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Evidence has shown that facilitating student-centered learning (SCL) in STEM classrooms enhances student learning and satisfaction [1]–[3]. However, despite increased support from educational and government bodies to incorporate SCL practices [1], minimal changes have been made in undergraduate STEM curriculum [4]. Faculty often teach as they were taught, relying heavily on traditional lecture-based teaching to disseminate knowledge [4]. Though some faculty express the desire to improve their teaching strategies, they feel limited by a lack of time, training, and incentives [4], [5]. To maximize student learning while minimizing instructor effort to change content, courses can be designed to incorporate simpler, less time-consuming SCL strategies that still have a positive impact on student experience. In this paper, we present one example of utilizing a variety of simple SCL strategies throughout the design and implementation of a 4-week long module. This module focused on introductory tissue engineering concepts and was designed to help students learn foundational knowledge within the field as well as develop critical technical skills. Further, the module sought to develop important professional skills such as problem-solving, teamwork, and communication. During module design and implementation, evidence-based SCL teaching strategies were applied to ensure students developed important knowledge and skills within the short timeframe. Lectures featured discussion-based active learning exercises to encourage student engagement and peer collaboration [6]–[8]. The module was designed using a situated perspective, acknowledging that knowing is inseparable from doing [9], and therefore each week, the material taught in the two lecture sessions was directly applied to that week’s lab to reinforce students’ conceptual knowledge through hands-on activities and experimental outcomes. Additionally, the majority of assignments served as formative assessments to motivate student performance while providing instructors with feedback to identify misconceptions and make real-time module improvements [10]–[12]. Students anonymously responded to pre- and post-module surveys, which focused on topics such as student motivation for enrolling in the module, module expectations, and prior experience. Students were also surveyed for student satisfaction, learning gains, and graduate student teaching team (GSTT) performance. Data suggests a high level of student satisfaction, as most students’ expectations were met, and often exceeded. Students reported developing a deeper understanding of the field of tissue engineering and learning many of the targeted basic lab skills. In addition to hands-on skills, students gained confidence to participate in research and an appreciation for interacting with and learning from peers. Finally, responses with respect to GSTT performance indicated a perceived emphasis on a learner-centered and knowledge/community-centered approaches over assessment-centeredness [13]. Overall, student feedback indicated that SCL teaching strategies can enhance student learning outcomes and experience, even over the short timeframe of this module. Student recommendations for module improvement focused primarily on modifying the lecture content and laboratory component of the module, and not on changing the teaching strategies employed. The success of this module exemplifies how instructors can implement similar strategies to increase student engagement and encourage in-depth discussions without drastically increasing instructor effort to re-format course content. Introduction. 
    more » « less
  2. The 2021 return to face-to-face teaching and proctored exams revealed significant gaps in student learning during remote instruction. The challenge of supporting underperforming students is not expected to abate in the next 5-10 years as COVID-19-related learning losses compound structural inequalities in K-12 education. More recently, anecdotal evidence across courses shows declines in classroom attendance and student engagement. Lack of engagement indicates emotional barriers rather than intellectual deficiencies, and its growth coincides with the ongoing mental health epidemic. Regardless of the underlying reasons, professors are now faced with the unappealing choice of awarding failing grades to an uncomfortably large fraction of classes or awarding passing grades to students who do not seem prepared for the workforce or adult life in general. Faculty training, if it exists, addresses neither the scale of this situation nor the emotional/identity aspects of the problem. There is an urgent need for pedagogical remediation tools that can be applied without additional TA or staff resources, without training in psychiatry, and with only five or eight weeks remaining in the semester. This work presents two work-in-progress interventions for engineering faculty who face the challenges described above. In the first intervention, students can improve their exam score by submitting videos of reworked exams. The requirement of voiceover forces students to understand the thought process behind problems, even if they have copied the answers from a friend. Incorporating peer review into the assignment reduces the workload for instructor grading. This intervention has been successfully implemented in sophomore- and senior-level courses with positive feedback from both faculty and students. In the second intervention, students who fail the midterm are offered an automatic passing exam grade (typically 51%) in exchange for submitting a knowledge inventory and remediation plan. Students create a glossary of terms and concepts from the class and rank them by their level of understanding. Recent iterations of the remediation plan also include reflections on emotions and support networks. In February 2023, the project team will scale the interventions to freshman-level Introductory Programming, which has 400 students and the highest fail/withdrawal rate in the college. The large sample size will enable more robust statistics to correlate exam scores, intervention rubric items, and surveys on assignment effectiveness. Piloting interventions in a variety of environments and classes will establish best pedagogical practices that minimize instructors’ workload and decision fatigue. The ultimate goal of this project is to benefit students and faculty through well-defined and systematic interventions across the curriculum. 
    more » « less
  3. Purpose In response to the evolving COVID-19 pandemic, many universities have transitioned to online instruction. With learning promising to be online, at least in part, for the near future, instructors may be thinking of providing online collaborative learning opportunities to their students who are increasingly isolated from their peers because of social distancing guidelines. This paper aims to provide design recommendations for online collaborative project-based learning exercises based on this research in a software engineering course at the university level. Design/methodology/approach Through joint work between learning scientists, course instructors and software engineering practitioners, instructional design best practices of alignment between the context of the learners, the learning objectives, the task and the assessment are actualized in the design of collaborative programming projects for supporting learning. The design, first segments a short real-time collaborative exercise into tasks, each with a problem-solving phase where students participate in collaborative programming, and a reflection phase for reflecting on what they learned in the task. Within these phases, a role-assignment paradigm scaffolds collaboration by assigning groups of four students to four complementary roles that rotate after each task. Findings By aligning each task with granular learning objectives, significant pre- to post-test learning from the exercise as well as each task is observed. Originality/value The roles used in the paradigm discourage divide-and-conquer tendencies often associated with collaborative projects. By requiring students to discuss conflicting ideas to arrive at a consensus implementation, their ideas are made explicit, thus providing opportunities for clarifying misconceptions through discussion and learning from the collaboration. 
    more » « less
  4. Security is a critical aspect in the design, development, and testing of software systems. Due to the increasing need for security-related skills within software systems and engineering, there is a growing demand for these skills to be taught at the university level. A series of 41 security modules was developed to assess the impact of these modules on teaching critical cyber security topics to students. This paper presents the implementation and outcomes of the first set of six security modules in a Freshman level course. This set consists of five modules presented in lectures as well as a sixth module emphasizing encryption and decryption used as the semester project for the course. Each module is a collection of concepts related to cyber security. The individual cyber security concepts are presented with a general description of a security issue to avoid, sample code with the security issue written in the Java programming language, and a second version of the code with an effective solution. The set of these modules was implemented in Computer Science I during the Fall 2019 semester. Incorporating each of the concepts in these modules into lectures depends on both the topic covered and the approach to resolving the related security issue. Students were introduced to computing concepts related to both the security issue and the appropriate solution to fully grasp the overall concept. After presenting the materials to students, continual review with students is also essential. This reviewal process requires exploring use-cases for the programming mechanisms presented as solutions to the security issues discussed. In addition to the security modules presented in lectures, students were given a hands-on approach to understanding the concepts through Model-Eliciting Activities (MEAs). MEAs are open-ended, problem-solving activities in which groups of three to four students work to solve realistic complex problems in a classroom setting. The semester project related to encryption and decryption was implemented into the course as an MEA. To assess the effectiveness of incorporating security modules with the MEA project into the curriculum of Computer Science I, two sections of the course were used as a control group and a treatment group. The treatment group included the security modules in lectures and the MEA project while the control group did not. To measure the overall effectiveness of incorporating security modules with the MEA project, both the instructor’s effectiveness as well as the student’s attitudes and interest were measured. For instructors, the primary question to address was to what extent do instructors change their attitudes towards student learning and their teaching practices because of the implementation of cyber security modules through MEAs. For students, the primary question to address was how the inclusion of security modules with the MEA project improved their understanding of the course materials and their interests in computer science. After implementing security modules with the MEA project, students showed a better understanding of cyber security concepts and a greater interest in broader computer science concepts. The instructor’s beliefs about teaching, learning, and assessment shifted from teacher-centered to student-centered, during his experience with the security modules and MEA. 
    more » « less
  5. Background/Context: After-school programs that focus on integrating computer programming and mathematics in authentic environments are seldomly accessible to students from culturally and linguistically diverse backgrounds, particularly bilingual Latina students in rural contexts. Providing a context that broadens Latina students’ participation in mathematics and computer programming requires educators to carefully examine how verbal and nonverbal language is used to interact and to position students as they learn new concepts in middle school. This is also an important stage for adolescents because they are likely to make decisions about their future careers in STEM. Having access to discourse and teaching practices that invite students to participate in mathematics and computer programming affords them opportunities to engage with these fields. Purpose/Focus of Study: This case study analyzes how small-group interactions mediated the positionings of Cindy, a bilingual Latina, as she learned binary numbers in an after-school program that integrated computer programming and mathematics (CPM). Setting: The Advancing Out-of-School Learning in Mathematics and Engineering (AOLME) program was held in a rural bilingual (Spanish and English) middle school in the Southwest. The after-school program was designed to provide experiences for primarily Latinx students to learn how to integrate mathematics with computer programming using Raspberry Pi and Python as a platform. Our case study explores how Cindy was positioned as she interacted with two undergraduate engineering students who served as facilitators while learning binary numbers with a group of three middle school students. Research Design: This single intrinsic case focused on exploring how small-group interactions among four students mediated Cindy’s positionings as she learned binary numbers through her participation in AOLME. Data sources included twelve 90-minute video sessions and Cindy’s journal and curriculum binder. Video logs were created, and transcripts were coded to describe verbal and nonverbal interactions among the facilitators and Cindy. Analysis of select episodes was conducted using systemic functional linguistics (SFL), specifically language modality, to identify how positioning took place. These episodes and positioning analysis describe how Cindy, with others, navigated the process of learning binary numbers under the stereotype that female students are not as good at mathematics as male students. Findings: From our analysis, three themes that emerged from the data portray Cindy’s experiences learning binary numbers. The major themes are: (1) Cindy’s struggle to reveal her understanding of binary numbers in a competitive context, (2) Cindy’s use of “fake it until you make it” to hide her cognitive dissonance, and (3) the use of Spanish and peers’ support to resolve Cindy’s understanding of binary numbers. The positioning patterns observed help us learn how, when Cindy’s bilingualism was viewed and promoted as an asset, this social context worked as a generative axis that addressed the challenges of learning binary numbers. The contrasting episodes highlight the facilitators’ productive teaching strategies and relations that nurtured Cindy’s social and intellectual participation in CPM. Conclusions/Recommendations: Cindy’s case demonstrates how the facilitator’s teaching, and participants’ interactions and discourse practices contributed to her qualitatively different positionings while she learned binary numbers, and how she persevered in this process. Analysis of communication acts supported our understanding of how Cindy’s positionings underpinned the discourse; how the facilitators’ and students’ discourse formed, shaped, or shifted Cindy’s positioning; and how discourse was larger than gender storylines that went beyond classroom interactions. Cindy’s case reveals the danger of placing students in “struggle” instead of a “productive struggle.” The findings illustrated that when Cindy was placed in struggle when confronting responding moves by the facilitator, her “safe” reaction was hiding and avoiding. In contrast, we also learned about the importance of empathetic, nurturing supporting responses that encourage students’ productive struggle to do better. We invite instructors to notice students’ hiding or avoiding and consider Cindy’s case. Furthermore, we recommend that teachers notice their choice of language because this is important in terms of positioning students. We also highlight Cindy’s agency as she chose to take up her friend’s suggestion to “fake it” rather than give up. 
    more » « less