ABSTRACT The article documents students’ experiences with the shift online at the onset of the COVID-19 pandemic and provides informed recommendations to STEM instructors regarding academic integrity and student stress. Over 500 students were surveyed on these topics, including an open-ended question. Students experienced more stress and perceived a greater workload in online courses and therefore preferred in-person courses overall. Personal awareness of cheating during online exams is positively correlated with the proportion of cheating a student perceives. Fear of getting caught is the best cheating deterrent while getting a better grade makes cheating most enticing. Randomization of questions and answer choices is perceived as a highly effective tool to reduce cheating and is reported as the least stress-inducing method. Inability to backtrack and time limits cause students the most stress. Students report that multiple choice questions are the least effective question type to discourage cheating and oral exam questions cause the most stress. Use of camera and lockdown browser or being video- and audio- recorded caused the majority of student stress. Yet, nearly 60% agree that the combination of camera and lockdown browser is an effective deterrent. Recommendations: (i) Be transparent regarding academic dishonesty detection methods and penalties. (ii) Use online invigilating tools. (iii) Synchronize exams and (iv) randomize exam questions. (v) Allow backtracking. (vi) Avoid converting in-person exams to online exams; instead, explore new ways of designing exams for the online environment.
more »
« less
Data-Driven Investigation into Variants of Code Writing Questions
To defend against collaborative cheating in code writing questions, instructors of courses with online, asynchronous exams can use the strategy of question variants. These question variants are manually written questions to be selected at random during exam time to assess the same learning goal. In order to create these variants, currently the instructors have to rely on intuition to accomplish the competing goals of ensuring that variants are different enough to defend against collaborative cheating, and yet similar enough where students are assessed fairly. In this paper, we propose data-driven investigation into these variants. We apply our data-driven investigation into a dataset of three midterm exams from a large introductory programming course. Our results show that (1) observable inequalities of student performance exist between variants and (2) these differences are not just limited to score. Our results also show that the information gathered from our data-driven investigation can be used to provide recommendations for improving design of future variants.
more »
« less
- Award ID(s):
- 1816615
- NSF-PAR ID:
- 10299904
- Date Published:
- Journal Name:
- 2020 IEEE 32nd Conference on Software Engineering Education and Training
- Page Range / eLocation ID:
- 1 to 10
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Proctoring educational assessments (e.g., quizzes and exams) has a cost, be it in faculty (and/or course staff) time or in money to pay for proctoring services. Previous estimates of the utility of proctoring (generally by estimating the score advantage of taking an exam without proctoring) vary widely and have mostly been implemented using an across subjects experimental designs and sometimes with low statistical power. We investigated the score advantage of unproctored exams versus proctored exams using a within-subjects design for N = 510 students in an on-campus introductory programming course with 5 proctored exams and 4 unproctored exams. We found that students scored 3.32 percentage points higher on questions on unproctored exams than on proctored exams (p < 0.001). More interestingly, however, we discovered that this score advantage on unproctored exams grew steadily as the semester progressed, from around 0 percentage points at the start of semester to around 7 percentage points by the end. As the most obvious explanation for this advantage is cheating, we refer to this behavior as the student population "learning to cheat". The data suggests that both more individuals are cheating and the average benefit of cheating is increasing over the course of the semester. Furthermore, we observed that studying for unproctored exams decreased over the course of the semester while studying for proctored exams stayed constant. Lastly, we estimated the score advantage by question type and found that our long-form programming questions had the highest score advantage on unproctored exams, but there are multiple possible explanations for this finding.more » « less
-
Parks, Samantha T. (Ed.)ABSTRACT The article documents faculty experiences with the shift online due to the pandemic and provides recommendations to science, technology, engineering, and mathematics (STEM) instructors. Over 100 faculty members were surveyed on these topics and contrasted with previously reported student experiences. The online shift changed how faculty administered exams, ran courses, and acted to ensure academic integrity. For example, when exams went online, 73% of faculty reported spending more time preventing cheating. Concerning academic integrity and stress, faculty and students agreed with the exception of a few notable disconnects. Students reported greater workloads in online classes, while faculty maintained that the shift online did not change student workloads. Students perceived more online cheating than faculty. Overall, there seems to be a significant disconnect regarding faculty not realizing how much their actions may encourage or discourage cheating. Few faculty (<15%) indicated that being a tough grader or having test times too short is a motivating factor, but over 55% of students reported that these motivate students to cheat. Conversely, over 60% of students reported respect for their professors discourages them from cheating, while only 37% of faculty indicated the same. Over 70% of faculty and students indicated that fear of getting caught is a deterrent to cheating. Recommendations to reconnect include (i) faculty should use the finding that the number one deterrent of cheating is fear of getting caught; and (ii) faculty should maintain students’ respect by being clear or overestimating workload requirements, carefully adjusting time for online exams, and setting clear expectations with uncomplicated exam questions consistent with the material taught.more » « less
-
null (Ed.)In the spring of 2020, universities across America, and the world, abruptly transitioned to online learning. The online transition required faculty to find novel ways to administer assessments and in some cases, for students to utilize novel ways of cheating in their classes. The purpose of this paper is to provide a retrospective on cheating during online exams in the spring of 2020. It specifically looks at honor code violations in a sophomore level engineering course that enrolled more than 200 students. In this particular course, four pre-COVID assessments were given in class and six mid-COVID assessments were given online. This paper examines the increasing rate of cheating on these assessments and the profiles of the students who were engaged in cheating. It compares students who were engaged in violations of the honor code by uploading exam questions vs. those who those who looked at solutions to uploaded questions. This paper also looks at the abuse of Chegg during exams and the responsiveness of Chegg’s honor code team. It discusses the effectiveness of Chegg’s user account data in pursuing academic integrity cases. Information is also provided on the question response times for Chegg tutors in answering exam questions and the actual efficacy of cheating in this fashion.more » « less
-
While engineering grows as a part of elementary education, important questions arise about the skills and practices we ask of students. Both collaboration and decision making are complex and critical to the engineering design process, but come with social and emotional work that can be difficult for elementary students to navigate. Productive engagement in collaborative teams has been seen to be highly variable; for some teams, interpersonal conflicts move the design process forward, while for others they stall the process. In this work in progress, we are investigating the research question, what is the nature of students’ disciplinary talk during scaffolded decision making? We explore this research question via a case study of one student group in a 4th-grade classroom enrolled in an outreach program run by a private university in a Northeastern city. This program sends pairs of university students into local elementary schools to facilitate engineering in the classroom for one hour per week. This is the only engineering instruction the elementary students receive and the engineering curriculum is planned by the university students. For the implementation examined in this study, the curriculum was designed by two researchers to scaffold collaborative groupwork and decision making. The instruction was provided by an undergraduate and one of the researchers, a graduate student. The scaffolds designed for this semester of outreach include a set of groupwork norms and a decision matrix. The groupwork norms were introduced on the first day of instruction; the instructors read them aloud, proposed groupwork scenarios to facilitate a whole class discussion about whether or not the norms were followed and how the students could act to follow the norms, and provided time for students to practice the norms in their engineering design groups for the first project. For the rest of the semester, an anchor chart of the norms was displayed in the classroom and referenced to encourage consensus. The researchers designed the decision matrix scaffold to encourage design decisions between multiple prototypes based on problem criteria and test results. Instructors modeled the use of this decision matrix on the third day of instruction, and students utilized the matrix in both design projects of the semester. Data sources for this descriptive study include students’ written artifacts, photos of their design constructions, and video records of whole-class and team discourse. We employ qualitative case study and microethnographic analysis techniques to explore the influence of the intentional discourse scaffolds on students’ collaborative and decision-making practices. Our analysis allowed us to characterize the linguistic resources (including the decision matrix) that the students used to complete four social acts during decision making: design evaluation, disagreeing with a teammate, arguing for a novel idea, and sympathizing with a design. This research has implications for the design of instructional scaffolds for engineering curricula at the elementary school level, whether taking place in an outreach program or in regular classroom instruction.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/CSEET49119.2020.9206195
