skip to main content


Title: Oral exams for large-enrollment engineering courses to promote academic integrity and student engagement during remote instruction
This work-in-progress paper presents an innovative practice of using oral exams to maintain academic integrity and promote student engagement in large-enrollment engineering courses during remote instruction. With the abrupt and widespread transition to distance learning and assessment brought on by the COVID-19 pandemic, there has been a registered upsurge in academic integrity violations globally. To address the challenge of compromised integrity, in the winter quarter of 2021 we have implemented oral exams across six mostly high-enrollment mechanical and electrical engineering undergraduate courses. We present our oral exam design parameters in each of the courses and discuss how oral exams relate to academic integrity, student engagement, stress, and implicit bias. We also address the challenge of scalability, as most of our oral exams were implemented in large classes, where academic integrity and student-instructor disconnection have generally gotten disproportionately worse during remote learning. Our survey results indicate that oral exams have positively contributed to academic integrity in our courses. Based on our preliminary study and experiences, we expect oral exams can be effectively leveraged to hinder cheating and foster academic honesty in students, even when in-person instruction and assessment resumes.  more » « less
Award ID(s):
2044472
PAR ID:
10311799
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ;
Date Published:
Journal Name:
2021 IEEE Frontiers in Education Conference (FIE)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. This project aims to enhance students’ learning in foundational engineering courses through oral exams based on the research conducted at the University of California San Diego. The adaptive dialogic nature of oral exams provides instructors an opportunity to better understand students’ thought processes, thus holding promise for improving both assessments of conceptual mastery and students’ learning attitudes and strategies. However, the issues of oral exam reliability, validity, and scalability have not been fully addressed. As with any assessment format, careful design is needed to maximize the benefits of oral exams to student learning and minimize the potential concerns. Compared to traditional written exams, oral exams have a unique design space, which involves a large range of parameters, including the type of oral assessment questions, grading criteria, how oral exams are administered, how questions are communicated and presented to the students, how feedback were provided, and other logistical perspectives such as weight of oral exam in overall course grade, frequency of oral assessment, etc. In order to address the scalability for high enrollment classes, key elements of the project are the involvement of the entire instructional team (instructors and teaching assistants). Thus the project will create a new training program to prepare faculty and teaching assistants to administer oral exams that include considerations of issues such as bias and students with disabilities. The purpose of this study is to create a framework to integrate oral exams in core undergraduate engineering courses, complementing existing assessment strategies by (1) creating a guideline to optimize the oral exam design parameters for the best students learning outcomes; and (2) Create a new training program to prepare faculty and teaching assistants to administer oral exams. The project will implement an iterative design strategy using an evidence-based approach of evaluation. The effectiveness of the oral exams will be evaluated by tracking student improvements on conceptual questions across consecutive oral exams in a single course, as well as across other courses. Since its start in January 2021, the project is well underway. In this poster, we will present a summary of the results from year 1: (1) exploration of the oral exam design parameters, and its impact in students’ engagement and perception of oral exams towards learning; (2) the effectiveness of the newly developed instructor and teaching assistants training programs (3) The development of the evaluation instruments to gauge the project success; (4) instructors and teaching assistants experience and perceptions. 
    more » « less
  2. While studies have shown that oral exams are a valuable method of assessment, their use has been limited due to concerns about scalability, examiner bias, and student anxiety. This paper presents preliminary results on incorporating oral exams into two large undergraduate engineering courses, examining the potential viability of these assessment strategies. This work was done when the courses were offered remotely due to COVID-19, but the results offer valuable insights that could carry over to in-person instruction as well. 
    more » « less
  3. Bucks County Community College (Bucks) in collaboration with Drexel University (Drexel) is committed to increasing the number of workforce ready engineers and engineering technicians and to creating a blueprint for 2+2 engineering education programs nationally. Recently, educational reform took an unexpected turn to remote teaching due to the world-wide COVID-19 pandemic. Within our NSF ATE grant to enhance our present engineering technology curriculum we modified and enhanced instructional and student engagement methods to assure workforce readiness of our students in a remote world. Curriculum enhancements within the engineering technology (ET) occupational major at Bucks and the B.S. in ET degree program at Drexel, modifications to delivery of workforce development certification programs through the Bucks Center for Workforce Development (CWD), and college-wide student engagement strategies were implemented to assure quality education and student engagement. Modifications to credit courses included asynchronous online courses, synchronous remote courses, and hybrid courses, which combined remote and on campus laboratory instruction. Our CWD implemented hybrid instruction that included necessary resources for students such as tool kits and borrowed laptop computers. In addition, a college wide program called Bucks+ was implemented through the Bucks Business and Innovation Department to increase enrollment, retention, and workforce readiness of students. The Bucks+ program focuses on student engagement through competition within curriculum, and extracurricular endeavors that prepare students for industry. We will share our successes and challenges within our call to action to engage students in a remote world and to enhance their educational experience through innovative instructional techniques. 
    more » « less
  4. This theory paper focuses on understanding how mastery learning has been implemented in undergraduate engineering courses through a systematic review. Academic environments that promote learning, mastery, and continuous improvement rather than inherent ability can promote performance and persistence. Scholarship has argued that students could achieve mastery of the course material when the time available to master concepts and the quality of instruction was made appropriate to each learner. Increasing time to demonstrate mastery involves a course structure that allows for repeated attempts on learning assessments (i.e., homework, quizzes, projects, exams). Students are not penalized for failed attempts but are rewarded for achieving eventual mastery. The mastery learning approach recognizes that mastery is not always achieved on first attempts and learning from mistakes and persisting is fundamental to how we learn. This singular concept has potentially the greatest impact on students’ mindset in terms of their belief they can be successful in learning the course material. A significant amount of attention has been given to mastery learning courses in secondary education and mastery learning has shown an exceptionally positive effect on student achievement. However, implementing mastery learning in an undergraduate course can be a cumbersome process as it requires instructors to significantly restructure their assignments and exams, evaluation process, and grading practices. In light of these challenges, it is unclear the extent to which mastery learning has been implemented in undergraduate engineering courses or if similar positive effects can be found. Therefore, we conducted a systematic review to elucidate, how in the U.S., (1) has mastery learning been implemented in undergraduate engineering courses from 1990 to the present time and (2) the student outcomes that have been reported for these implementations. Using the systematic process outlined by Borrego et al. (2014), we surveyed seven databases and a total of 584 articles consisting of engineering and non-engineering courses were identified. We focused our review on studies that were centered on applying the mastery learning pedagogical method in undergraduate engineering courses. All peer-reviewed and practitioner articles and conference proceedings that were within our scope were included in the synthetization phase of the review. Most articles were excluded based on our inclusion and exclusion criteria. Twelve studies focused on applying mastery learning to undergraduate engineering courses. The mastery learning method was mainly applied on midterm exams, few studies used the method on homework assignments, and no study applied the method to the final exam. Students reported an increase in learning as a result of applying mastery learning. Several studies reported that students’ grades in a traditional final exam were not affected by mastery learning. Students’ self-reported evaluation of the course suggests that students prefer the mastery learning approach over traditional methods. Although a clear consensus on the effect of the mastery learning approach could not be achieved as each article applied different survey instruments to capture students’ perspectives. Responses to open-ended questions have mixed results. Two studies report more positive student comments on opened-ended questions, while one study report receiving more negative comments regarding the implementation of the mastery learning method. In the full paper we more thoroughly describe the ways in which mastery learning was implemented along with clear examples of common and divergent student outcomes across the twelve studies. 
    more » « less
  5. ABSTRACT The article documents students’ experiences with the shift online at the onset of the COVID-19 pandemic and provides informed recommendations to STEM instructors regarding academic integrity and student stress. Over 500 students were surveyed on these topics, including an open-ended question. Students experienced more stress and perceived a greater workload in online courses and therefore preferred in-person courses overall. Personal awareness of cheating during online exams is positively correlated with the proportion of cheating a student perceives. Fear of getting caught is the best cheating deterrent while getting a better grade makes cheating most enticing. Randomization of questions and answer choices is perceived as a highly effective tool to reduce cheating and is reported as the least stress-inducing method. Inability to backtrack and time limits cause students the most stress. Students report that multiple choice questions are the least effective question type to discourage cheating and oral exam questions cause the most stress. Use of camera and lockdown browser or being video- and audio- recorded caused the majority of student stress. Yet, nearly 60% agree that the combination of camera and lockdown browser is an effective deterrent. Recommendations: (i) Be transparent regarding academic dishonesty detection methods and penalties. (ii) Use online invigilating tools. (iii) Synchronize exams and (iv) randomize exam questions. (v) Allow backtracking. (vi) Avoid converting in-person exams to online exams; instead, explore new ways of designing exams for the online environment. 
    more » « less