- Award ID(s):
- 2122941
- NSF-PAR ID:
- 10357640
- Date Published:
- Journal Name:
- Zone 1 Conference of the American Society for Engineering Education
- ISSN:
- 2332-368X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
This project aims to enhance students’ learning in foundational engineering courses through oral exams based on the research conducted at the University of California San Diego. The adaptive dialogic nature of oral exams provides instructors an opportunity to better understand students’ thought processes, thus holding promise for improving both assessments of conceptual mastery and students’ learning attitudes and strategies. However, the issues of oral exam reliability, validity, and scalability have not been fully addressed. As with any assessment format, careful design is needed to maximize the benefits of oral exams to student learning and minimize the potential concerns. Compared to traditional written exams, oral exams have a unique design space, which involves a large range of parameters, including the type of oral assessment questions, grading criteria, how oral exams are administered, how questions are communicated and presented to the students, how feedback were provided, and other logistical perspectives such as weight of oral exam in overall course grade, frequency of oral assessment, etc. In order to address the scalability for high enrollment classes, key elements of the project are the involvement of the entire instructional team (instructors and teaching assistants). Thus the project will create a new training program to prepare faculty and teaching assistants to administer oral exams that include considerations of issues such as bias and students with disabilities. The purpose of this study is to create a framework to integrate oral exams in core undergraduate engineering courses, complementing existing assessment strategies by (1) creating a guideline to optimize the oral exam design parameters for the best students learning outcomes; and (2) Create a new training program to prepare faculty and teaching assistants to administer oral exams. The project will implement an iterative design strategy using an evidence-based approach of evaluation. The effectiveness of the oral exams will be evaluated by tracking student improvements on conceptual questions across consecutive oral exams in a single course, as well as across other courses. Since its start in January 2021, the project is well underway. In this poster, we will present a summary of the results from year 1: (1) exploration of the oral exam design parameters, and its impact in students’ engagement and perception of oral exams towards learning; (2) the effectiveness of the newly developed instructor and teaching assistants training programs (3) The development of the evaluation instruments to gauge the project success; (4) instructors and teaching assistants experience and perceptions.more » « less
-
Successive relearning improves performance on a high‐stakes exam in a difficult biopsychology course
Summary Successive relearning combines two effective learning techniques (retrieval practice and spaced practice) and involves practicing retrieval until some level of mastery has been reached (i.e., at least one correct retrieval attempt) in each of multiple sessions. Several laboratory studies have demonstrated the promise of successive relearning for enhancing student learning, but attempts to evaluate its effectiveness in authentic educational contexts are limited. In the current research, we implemented successive relearning in a difficult biopsychology course to examine its effectiveness at enhancing student learning on a high‐stakes exam. Students successively relearned some course content via a virtual flashcard program, whereas other course content was learned with business‐as‐usual activities. In two experiments, successive relearning boosted students' learning of course content by at least 10% (withd s ranging from 0.54 to 1.10). These findings suggest that implementing successive relearning in courses can be an effective method of promoting student achievement. -
The 2021 return to face-to-face teaching and proctored exams revealed significant gaps in student learning during remote instruction. The challenge of supporting underperforming students is not expected to abate in the next 5-10 years as COVID-19-related learning losses compound structural inequalities in K-12 education. More recently, anecdotal evidence across courses shows declines in classroom attendance and student engagement. Lack of engagement indicates emotional barriers rather than intellectual deficiencies, and its growth coincides with the ongoing mental health epidemic. Regardless of the underlying reasons, professors are now faced with the unappealing choice of awarding failing grades to an uncomfortably large fraction of classes or awarding passing grades to students who do not seem prepared for the workforce or adult life in general. Faculty training, if it exists, addresses neither the scale of this situation nor the emotional/identity aspects of the problem. There is an urgent need for pedagogical remediation tools that can be applied without additional TA or staff resources, without training in psychiatry, and with only five or eight weeks remaining in the semester. This work presents two work-in-progress interventions for engineering faculty who face the challenges described above. In the first intervention, students can improve their exam score by submitting videos of reworked exams. The requirement of voiceover forces students to understand the thought process behind problems, even if they have copied the answers from a friend. Incorporating peer review into the assignment reduces the workload for instructor grading. This intervention has been successfully implemented in sophomore- and senior-level courses with positive feedback from both faculty and students. In the second intervention, students who fail the midterm are offered an automatic passing exam grade (typically 51%) in exchange for submitting a knowledge inventory and remediation plan. Students create a glossary of terms and concepts from the class and rank them by their level of understanding. Recent iterations of the remediation plan also include reflections on emotions and support networks. In February 2023, the project team will scale the interventions to freshman-level Introductory Programming, which has 400 students and the highest fail/withdrawal rate in the college. The large sample size will enable more robust statistics to correlate exam scores, intervention rubric items, and surveys on assignment effectiveness. Piloting interventions in a variety of environments and classes will establish best pedagogical practices that minimize instructors’ workload and decision fatigue. The ultimate goal of this project is to benefit students and faculty through well-defined and systematic interventions across the curriculum.more » « less
-
This work-in-progress paper presents an innovative practice of using oral exams to maintain academic integrity and promote student engagement in large-enrollment engineering courses during remote instruction. With the abrupt and widespread transition to distance learning and assessment brought on by the COVID-19 pandemic, there has been a registered upsurge in academic integrity violations globally. To address the challenge of compromised integrity, in the winter quarter of 2021 we have implemented oral exams across six mostly high-enrollment mechanical and electrical engineering undergraduate courses. We present our oral exam design parameters in each of the courses and discuss how oral exams relate to academic integrity, student engagement, stress, and implicit bias. We also address the challenge of scalability, as most of our oral exams were implemented in large classes, where academic integrity and student-instructor disconnection have generally gotten disproportionately worse during remote learning. Our survey results indicate that oral exams have positively contributed to academic integrity in our courses. Based on our preliminary study and experiences, we expect oral exams can be effectively leveraged to hinder cheating and foster academic honesty in students, even when in-person instruction and assessment resumes.more » « less
-
This complete evidence-based practice paper discusses the strategies and results of an introduction to mechanics course, designed to prepare students for introductory-level physics and other fundamental courses in engineering, such as statics, strength of materials, and dynamics. The course was developed to address historically high failure (DFW) rates in the physics courses and is part of a set of interventions implemented to support student success in a college of engineering and computer science. The course focuses on providing in-depth understanding of Newton’s Laws of motion, free-body diagrams, and linear and projectile motion. Because it focuses on a limited number of competencies, it is possible to spend more time on inquiry-based activities and in-class discussions. The course framework was designed considering the Ebbinghaus’ Forgetting Curve, to provide students with learning opportunities in 6-day cycles: (i) day 1: a pre-class learning activity (reading or video) and a quiz; (ii) day 2: in-class Kahoot low-stakes quiz with discussion, a short lecture with embedded time for problem-solving and discussion, and in-class activities (labs, group projects); (iii) day 4: homework due two days after the class; (iv) day 6: homework self-reflection (autopsy based on provided solutions) two days after homework is due. The assessment of course performance is based on the well-characterized force concept inventory (FCI) exam that is administered before the intro to mechanics course and both before and after the Physics I course; and on student performance (grades) in Physics and Statics courses. Results from the FCI pre-test show that students who took the introduction to mechanics course (treatment group) started the physics course with a much better understanding of force concepts than other students in the course. The FCI post-test shows better normalized gain for the treatment group, compared to other students, which is also aligned with student performance in the course. Additionally, student performance is significantly better in statics, with 25% DWF rate compared to 50% for the other students. In summary, the framework of the course, which focuses on providing students with in-depth understanding of force concepts, has led to better learning and performance in Physics I, but importantly it has also helped students achieve better performance in the Statics course, the first fundamental course in civil and mechanical engineering programs.more » « less