More Like this

1. Learning to Cheat: Quantifying Changes in Score Advantage of Unproctored Assessments Over Time

   https://doi.org/10.1145/3386527.3405925  

   Chen, Binglin; Azad, Sushmita; Fowler, Max; West, Matthew; Zilles, Craig (August 2020, Proceedings of the Seventh ACM Conference on Learning @ Scale)

   null (Ed.)

   Proctoring educational assessments (e.g., quizzes and exams) has a cost, be it in faculty (and/or course staff) time or in money to pay for proctoring services. Previous estimates of the utility of proctoring (generally by estimating the score advantage of taking an exam without proctoring) vary widely and have mostly been implemented using an across subjects experimental designs and sometimes with low statistical power. We investigated the score advantage of unproctored exams versus proctored exams using a within-subjects design for N = 510 students in an on-campus introductory programming course with 5 proctored exams and 4 unproctored exams. We found that students scored 3.32 percentage points higher on questions on unproctored exams than on proctored exams (p < 0.001). More interestingly, however, we discovered that this score advantage on unproctored exams grew steadily as the semester progressed, from around 0 percentage points at the start of semester to around 7 percentage points by the end. As the most obvious explanation for this advantage is cheating, we refer to this behavior as the student population "learning to cheat". The data suggests that both more individuals are cheating and the average benefit of cheating is increasing over the course of the semester. Furthermore, we observed that studying for unproctored exams decreased over the course of the semester while studying for proctored exams stayed constant. Lastly, we estimated the score advantage by question type and found that our long-form programming questions had the highest score advantage on unproctored exams, but there are multiple possible explanations for this finding. 

   more » « less
2. Learning is Quick: Evidence for the Impact of Learning Beliefs on Self-Regulated Learning

   Morphew, Jason; Shafer, Devyn; Zhang, Muxin; Girotti-Hernandez, Daniela; Stelzer, Tim (April 2023, American Educational Research Association)

   Exam preparation in introductory science courses is self-regulated. Practice testing has been shown to produce better learning then other strategies. However, many students do not use practice tests effectively when studying. This mixed-methods study examines two experiments aimed at improving examining student predictions about learning and studying. We found that scores on a mock exam impact students’ intentions for studying but not study habits. We also found that many underperforming students initially increase the use of ineffective study strategies rather than adopt a strategy change. Students who distribute studying throughout the semester and engage with course concepts more deeply demonstrate improvement and increased satisfaction. These results suggest that exam preparation interventions may need to include study strategy and metacognitive instruction. 

   more » « less
3. Evolving study strategies and support structures of introductory physics students

   https://doi.org/10.1103/PhysRevPhysEducRes.20.020114  

   Shafer, Devyn; Girotti-Hernandez, Daniela; Stelzer, Tim (September 2024, Physical Review Physics Education Research)

   Henderson, Charles (Ed.)

   Students often struggle in the transition from high school to college. One component of this struggle is adjusting study habits to navigate new academic expectations. Another is establishing new relationships in college that can support their emotional well-being and academic success. We administered surveys consisting primarily of open-ended questions to students taking an introductory physics course in order to gain insight into students’ study habits and support structures and how they change from high school to college. We find that many students learn that they need to dedicate more time outside of class to studying in order to succeed in college. Many students trying to improve their performance report practicing more, but our results suggest that more practice alone is not sufficient; rather, students were able to increase their performance and satisfaction in the course by engaging more deeply with the material. Regarding support structures, we find that in high school, students find their teachers highly supportive and accessible, but they are less likely to approach their college professors for help. Meanwhile, many students find peers to be an important source of support in college as the amount of support they receive from their families diminishes with distance from home. Gaining a better understanding of students’ study habits, support structures, and how they conceptualize them can help us design course structures and messaging that can more effectively help students develop strong learning strategies and social networks. Published by the American Physical Society2024 

   more » « less
4. Students' Perceptions and Behavior Related to Second-Chance Testing

   https://doi.org/10.1109/FIE49875.2021.9637173  

   Emeka, Chinedu; Bretl, Timothy; Herman, Geoffrey; West, Matthew; Zilles, Craig (October 2021, 2021 IEEE Frontiers in Education Conference (FIE))

   This full research paper explores students’ attitudes toward second-chance testing and how second-chance testing influences students’ behavior. Second-chance testing refers to giving students the opportunity to take a second instance of each exam for some sort of grade replacement. Previous work has demonstrated that second-chance testing can lead to improved student outcomes in courses, but how to best structure second-chance testing to maximize its benefits remains an open question. We complement previous work by interviewing a diverse group of 23 students that have taken courses that use second-chance testing. From the interviews, we sought to gain insight into students’ views and use of second-chance testing. We found that second-chance testing was almost universally viewed positively by the students and was frequently cited as helping to reduce test takers’ anxiety and boost their confidence. Overall, we find that the majority of students prepare for second-chance exams in desirable ways, but we also note ways in which second-chance testing can potentially lead to undesirable behaviors including procrastination, over-reliance on memorization, and attempts to game the system. We identified emergent themes pertaining to various facets of second-chance test-taking, including: 1) concerns about the time commitment required for second-chance exams; 2) a belief that second-chance exams promoted fairness; and 3) how second-chance testing incentivized learning. This paper will provide instructors and other stakeholders with detailed insights into students’ behavior regarding second-chance testing, enabling instructors to develop better policies and avoid unintended consequences. 

   more » « less
5. Student Perceptions of the Complete Online Transition of Two CS Courses in Response to the COVID-19 Pandemic

   Farghally, M. F. (July 2021, 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference)

   null (Ed.)

   Student perceptions of the complete online transition of two CS courses in response to the COVID-19 pandemic Due to the COVID-19 pandemic, universities across the globe switched from traditional Face-to-Face (F2F) course delivery to completely online. Our university declared during our Spring break that students would not return to campus, and that all courses must be delivered fully online starting two weeks later. This was challenging to both students and instructors. In this evidence-based practice paper, we present results of end-of-semester student surveys from two Spring 2020 CS courses: a programming intensive CS2 course, and a senior theory course in Formal Languages and Automata (FLA). Students indicated course components they perceived as most beneficial to their learning, before and then after the online transition, and preferences for each regarding online vs. F2F. By comparing student reactions across courses, we gain insights on which components are easily adapted to online delivery, and which require further innovation. COVID was unfortunate, but gave a rare opportunity to compare students’ reflections on F2F instruction with online instructional materials for half a semester vs. entirely online delivery of the same course during the second half. The circumstances are unique, but we were able to acquire insights for future instruction. Some course components were perceived to be more useful either before or after the transition, and preferences were not the same in the two courses, possibly due to differences in the courses. Students in both courses found prerecorded asynchronous lectures significantly less useful than in-person lectures. For CS2, online office hours were significantly less useful than in-person office hours, but we found no significant difference in FLA. CS2 students felt less supported by their instructor after the online transition, but no significant difference was indicated by FLA students. FLA students found unproctored online exams offered through Canvas more stressful than in-person proctored exams, but the opposite was indicated by CS2 students. CS2 students indicated that visual materials from an eTextbook were more useful to them after going online than before, but FLA students indicated no significant difference. Overall, students in FLA significantly preferred the traditional F2F version of the course, while no significant difference was detected for CS2 students. We did not find significant effects from gender on the preference of one mode over the other. A serendipitous outcome was learning that some changes forced by circumstance should be considered for long term adoption. Offering online lab sessions and online exams where the questions are primarily multiple choice are possible candidates. However, we found that students need to feel the presence of their instructor to feel properly supported. To determine what course components need further improvement before transitioning to fully online mode, we computed a logistic regression model. The dependent variable is the student's preference for F2F or fully online. The independent variables are the course components before and after the online transition. For both courses, in-person lectures were a significant factor negatively affecting students' preferences of the fully online mode. Similarly, for CS2, in-person labs and in-person office hours were significant factors pushing students’ preferences toward F2F mode. 

   more » « less