This theory paper focuses on understanding how mastery learning has been implemented in undergraduate engineering courses through a systematic review. Academic environments that promote learning, mastery, and continuous improvement rather than inherent ability can promote performance and persistence. Scholarship has argued that students could achieve mastery of the course material when the time available to master concepts and the quality of instruction was made appropriate to each learner. Increasing time to demonstrate mastery involves a course structure that allows for repeated attempts on learning assessments (i.e., homework, quizzes, projects, exams). Students are not penalized for failed attempts but are rewarded for achieving eventual mastery. The mastery learning approach recognizes that mastery is not always achieved on first attempts and learning from mistakes and persisting is fundamental to how we learn. This singular concept has potentially the greatest impact on students’ mindset in terms of their belief they can be successful in learning the course material. A significant amount of attention has been given to mastery learning courses in secondary education and mastery learning has shown an exceptionally positive effect on student achievement. However, implementing mastery learning in an undergraduate course can be a cumbersome process as it requires instructors to significantly restructure their assignments and exams, evaluation process, and grading practices. In light of these challenges, it is unclear the extent to which mastery learning has been implemented in undergraduate engineering courses or if similar positive effects can be found.
Therefore, we conducted a systematic review to elucidate, how in the U.S., (1) has mastery learning been implemented in undergraduate engineering courses from 1990 to the present time and (2) the student outcomes that have been reported for these implementations. Using the systematic process outlined by Borrego et al. (2014), we surveyed seven databases and a total of 584 articles consisting of engineering and non-engineering courses were identified. We focused our review on studies that were centered on applying the mastery learning pedagogical method in undergraduate engineering courses. All peer-reviewed and practitioner articles and conference proceedings that were within our scope were included in the synthetization phase of the review. Most articles were excluded based on our inclusion and exclusion criteria.
Twelve studies focused on applying mastery learning to undergraduate engineering courses. The mastery learning method was mainly applied on midterm exams, few studies used the method on homework assignments, and no study applied the method to the final exam. Students reported an increase in learning as a result of applying mastery learning. Several studies reported that students’ grades in a traditional final exam were not affected by mastery learning. Students’ self-reported evaluation of the course suggests that students prefer the mastery learning approach over traditional methods. Although a clear consensus on the effect of the mastery learning approach could not be achieved as each article applied different survey instruments to capture students’ perspectives. Responses to open-ended questions have mixed results. Two studies report more positive student comments on opened-ended questions, while one study report receiving more negative comments regarding the implementation of the mastery learning method. In the full paper we more thoroughly describe the ways in which mastery learning was implemented along with clear examples of common and divergent student outcomes across the twelve studies.
more »
« less
Comparison of Grade Replacement and Weighted Averages for Second-Chance Exams
We explore how course policies affect students' studying and learning when a second-chance exam is offered. High-stakes, one-off exams remain a de facto standard for assessing student knowledge in STEM, despite compelling evidence that other assessment paradigms such as mastery learning can improve student learning. Unfortunately, mastery learning can be costly to implement. We explore the use of optional second-chance testing to sustainably reap the benefits of mastery-based learning at scale. Prior work has shown that course policies affect students' studying and learning but have not compared these effects within the same course context. We conducted a quasi-experimental study in a single course to compare the effect of two grading policies for second-chance exams and the effect of increasing the size of the range of dates for students taking asynchronous exams. The first grading policy, called 90-cap, allowed students to optionally take a second-chance exam that would fully replace their score on a first-chance exam except the second-chance exam would be capped at 90% credit. The second grading policy, called 90-10, combined students' first- and second-chance exam scores as a weighted average (90% max score + 10% min score). The 90-10 policy significantly increased the likelihood that marginally competent students would take the second-chance exam. Further, our data suggests that students learned more under the 90-10 policy, providing improved student learning outcomes at no cost to the instructor. Most students took exams on the last day an exam was available, regardless of how many days the exam was available.
more »
« less
- Award ID(s):
- 1915257
- NSF-PAR ID:
- 10200288
- Date Published:
- Journal Name:
- Proceedings of the 2020 ACM Conference on International Computing Education Research
- Page Range / eLocation ID:
- 56 to 66
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This Research Full paper focuses on perceptions and experiences of freshman and sophomore engineering students when playing an online serious engineering game that was designed to improve engineering intuition and knowledge of statics. Use of serious educational engineering games has increased in engineering education to help students increase technical competencies in engineering disciplines. However, few have investigated how these engineering games are experienced by the students; how games influence students' perceptions of learning, or how these factors may lead to inequitable perspectives among diverse populations of students. Purpose/Hypothesis: The purpose of this study was to explore the perceptions, appeal, and opinions about the efficacy of educational online games among a diverse population of students in an engineering mechanics statics course. It was hypothesized that compared to majority groups (e.g., men, White), women of color who are engineering students would experience less connections to the online educational game in terms of ease of use and level of frustration while playing. It is believed that these discordant views may negatively influence the game's appeal and efficacy towards learning engineering in this population of students. Design/Method: The Technology Acceptance Model (TAM) is expanded in this study, where the perspectives of women of colour (Latinx, Asian and African American) engineering students are explored. The research approach employed in this study is a mixed-method sequential exploratory design, where students first played the online engineering educational game, then completed a questionnaire, followed by participation in a focus group. Responses were initially analyzed through open and magnitude coding approaches to understand whether students thought these educational games reflected their personal culture. Results: Preliminary results indicate that though the majority of the students were receptive to using the online engineering software for their engineering education, merely a few intimated that they would use this software for engineering exam or technical job interview preparation. A level-one categorical analysis identified a few themes that comprised unintended preservation of inequality in favor of students who enjoyed contest-based education and game technology. Competition-based valuation of presumed mastery of course content fostered anxiety and intimidation among students, which caused some to "game the game" instead of studying the material, to meet grade goals. Some students indicated that they spent more time (than necessary) to learn the goals of the game than engineering content itself, suggesting a need to better integrate course material while minimizing cognitive effort in learning to navigate the game. Conclusions: Preliminary results indicate that engineering software's design and the way is coupled with course grading and assessment of learning outcomes, affect student perceptions of the technology's acceptance, usefulness, and ease of use as a "learning tool." Students were found to have different expectations of serious games juxtaposed software/apps designed for entertainment. Conclusions also indicate that acceptance of inquiry-based educational games in a classroom among diverse populations of students should clearly articulate and connect the game goals/objectives with class curriculum content. Findings also indicate that a multifaceted schema of tools, such as feedback on game challenges, and explanations for predictions of the game should be included in game/app designs.more » « less
-
This project aims to enhance students’ learning in foundational engineering courses through oral exams based on the research conducted at the University of California San Diego. The adaptive dialogic nature of oral exams provides instructors an opportunity to better understand students’ thought processes, thus holding promise for improving both assessments of conceptual mastery and students’ learning attitudes and strategies. However, the issues of oral exam reliability, validity, and scalability have not been fully addressed. As with any assessment format, careful design is needed to maximize the benefits of oral exams to student learning and minimize the potential concerns. Compared to traditional written exams, oral exams have a unique design space, which involves a large range of parameters, including the type of oral assessment questions, grading criteria, how oral exams are administered, how questions are communicated and presented to the students, how feedback were provided, and other logistical perspectives such as weight of oral exam in overall course grade, frequency of oral assessment, etc. In order to address the scalability for high enrollment classes, key elements of the project are the involvement of the entire instructional team (instructors and teaching assistants). Thus the project will create a new training program to prepare faculty and teaching assistants to administer oral exams that include considerations of issues such as bias and students with disabilities. The purpose of this study is to create a framework to integrate oral exams in core undergraduate engineering courses, complementing existing assessment strategies by (1) creating a guideline to optimize the oral exam design parameters for the best students learning outcomes; and (2) Create a new training program to prepare faculty and teaching assistants to administer oral exams. The project will implement an iterative design strategy using an evidence-based approach of evaluation. The effectiveness of the oral exams will be evaluated by tracking student improvements on conceptual questions across consecutive oral exams in a single course, as well as across other courses. Since its start in January 2021, the project is well underway. In this poster, we will present a summary of the results from year 1: (1) exploration of the oral exam design parameters, and its impact in students’ engagement and perception of oral exams towards learning; (2) the effectiveness of the newly developed instructor and teaching assistants training programs (3) The development of the evaluation instruments to gauge the project success; (4) instructors and teaching assistants experience and perceptions.more » « less
-
This full research paper explores students’ attitudes toward second-chance testing and how second-chance testing influences students’ behavior. Second-chance testing refers to giving students the opportunity to take a second instance of each exam for some sort of grade replacement. Previous work has demonstrated that second-chance testing can lead to improved student outcomes in courses, but how to best structure second-chance testing to maximize its benefits remains an open question. We complement previous work by interviewing a diverse group of 23 students that have taken courses that use second-chance testing. From the interviews, we sought to gain insight into students’ views and use of second-chance testing. We found that second-chance testing was almost universally viewed positively by the students and was frequently cited as helping to reduce test takers’ anxiety and boost their confidence. Overall, we find that the majority of students prepare for second-chance exams in desirable ways, but we also note ways in which second-chance testing can potentially lead to undesirable behaviors including procrastination, over-reliance on memorization, and attempts to game the system. We identified emergent themes pertaining to various facets of second-chance test-taking, including: 1) concerns about the time commitment required for second-chance exams; 2) a belief that second-chance exams promoted fairness; and 3) how second-chance testing incentivized learning. This paper will provide instructors and other stakeholders with detailed insights into students’ behavior regarding second-chance testing, enabling instructors to develop better policies and avoid unintended consequences.more » « less
-
The 2021 return to face-to-face teaching and proctored exams revealed significant gaps in student learning during remote instruction. The challenge of supporting underperforming students is not expected to abate in the next 5-10 years as COVID-19-related learning losses compound structural inequalities in K-12 education. More recently, anecdotal evidence across courses shows declines in classroom attendance and student engagement. Lack of engagement indicates emotional barriers rather than intellectual deficiencies, and its growth coincides with the ongoing mental health epidemic. Regardless of the underlying reasons, professors are now faced with the unappealing choice of awarding failing grades to an uncomfortably large fraction of classes or awarding passing grades to students who do not seem prepared for the workforce or adult life in general. Faculty training, if it exists, addresses neither the scale of this situation nor the emotional/identity aspects of the problem. There is an urgent need for pedagogical remediation tools that can be applied without additional TA or staff resources, without training in psychiatry, and with only five or eight weeks remaining in the semester. This work presents two work-in-progress interventions for engineering faculty who face the challenges described above. In the first intervention, students can improve their exam score by submitting videos of reworked exams. The requirement of voiceover forces students to understand the thought process behind problems, even if they have copied the answers from a friend. Incorporating peer review into the assignment reduces the workload for instructor grading. This intervention has been successfully implemented in sophomore- and senior-level courses with positive feedback from both faculty and students. In the second intervention, students who fail the midterm are offered an automatic passing exam grade (typically 51%) in exchange for submitting a knowledge inventory and remediation plan. Students create a glossary of terms and concepts from the class and rank them by their level of understanding. Recent iterations of the remediation plan also include reflections on emotions and support networks. In February 2023, the project team will scale the interventions to freshman-level Introductory Programming, which has 400 students and the highest fail/withdrawal rate in the college. The large sample size will enable more robust statistics to correlate exam scores, intervention rubric items, and surveys on assignment effectiveness. Piloting interventions in a variety of environments and classes will establish best pedagogical practices that minimize instructors’ workload and decision fatigue. The ultimate goal of this project is to benefit students and faculty through well-defined and systematic interventions across the curriculum.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3372782.3406260
