An official website of the United States government
The Department of Electrical and Computer Engineering at a large Midwestern University is seeking to enhance undergraduate engineering education through a combination of programmatic efforts to create departmental change. Three distinct programs aim to transform ECE education through collaborative course design, enhancements to the department climate, and increases in the opportunities for underrepresented undergraduate engineering students. Due to the integrative and corresponding programmatic goals, it was vital to develop a unified evaluation in line with the program evaluation standards (Yarbrough, Shulha, Hopson, & Caruthers, 2011). Further, the interaction of multiple programs necessitated evaluating goal attainment at both the programmatic and departmental levels to determine not only the effects of individual programs but also to examine the broader effect of the interaction of multiple ongoing programmatic efforts to enhance engineering education. To facilitate this process, program team members developed comprehensive lists of ongoing activities designed to create change in the department within each program. Evaluators worked with the program teams to theme and cluster activities into similar groups. To understand how each cluster of activities was positioned to create departmental change and revolutionize engineering education, the evaluators and team members then attempted to identify how each cluster of activities worked as change strategies within the model by Henderson, Beach, and Finkelstein (2011). Thus, evaluators were able to identify over twenty distinct clusters of change activities working as change strategies within the four pillars of the change model: Curriculum and pedagogy, reflective teachers, policy, and shared vision. Positioning activities within this model allowed the evaluators and team members to 1) Better understand the broad scope of departmental activities and change strategies, 2) Identify strengths and challenges associated with their current efforts to transform engineering education within the department, and 3) Develop and integrate ongoing evaluation efforts to further understand both the programmatic and interactive effects of having multiple programs designed at facilitating departmental change and enhancing engineering education. The model for understanding department change and the approaches within that model that are being used to transform ECE education will be presented. We will further explain how the change model approach facilitated evaluating each program and the interactive effects of the combined programmatic efforts within the program evaluation standards of utility, feasibility, propriety, and accuracy (Yarbrough et al., 2011). Specific programmatic and interactive evaluation approaches will be discussed. References Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48(8), 952-984. Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Sage.
more »
« less
Evaluating the extent of a large-scale transformation in gateway science courses
We evaluate the impact of an institutional effort to transform undergraduate science courses using an approach based on course assessments. The approach is guided by A Framework for K-12 Science Education and focuses on scientific and engineering practices, crosscutting concepts, and core ideas, together called three-dimensional learning. To evaluate the extent of change, we applied the Three-dimensional Learning Assessment Protocol to 4 years of chemistry, physics, and biology course exams. Changes in exams differed by discipline and even by course, apparently depending on an interplay between departmental culture, course organization, and perceived course ownership, demonstrating the complex nature of transformation in higher education. We conclude that while transformation must be supported at all organizational levels, ultimately, change is controlled by factors at the course and departmental levels.
more »
« less
- PAR ID:
- 10105038
- Date Published:
- Journal Name:
- Science Advances
- Volume:
- 4
- Issue:
- 10
- ISSN:
- 2375-2548
- Page Range / eLocation ID:
- eaau0554
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
In the United States, the onset of COVID-19 triggered a nationwide lockdown, which forced many universities to move their primary assessments from invigilated in-person exams to unproctored online exams. This abrupt change occurred midway through the Spring 2020 semester, providing an unprecedented opportunity to investigate whether online exams can provide meaningful assessments of learning relative to in-person exams on a per-student basis. Here, we present data from nearly 2,000 students across 18 courses at a large Midwestern University. Using a meta-analytic approach in which we treated each course as a separate study, we showed that online exams produced scores that highly resembled those from in-person exams at an individual level despite the online exams being unproctored—as demonstrated by a robust correlation between online and in-person exam scores. Moreover, our data showed that cheating was either not widespread or ineffective at boosting scores, and the strong assessment value of online exams was observed regardless of the type of questions asked on the exam, the course level, academic discipline, or class size. We conclude that online exams, even when unproctored, are a viable assessment tool.more » « less
-
Specifications and mastery grading schemes have been growing in popularity in higher education over the past several years, and reports of specifications grading and other alternative grading systems are emerging in the chemistry education literature. The general goal of these alternative grading approaches is to reduce the reliance on high-stakes exams and give students a more transparent pathway to achieving the course learning outcomes. More importantly, relying less on infrequent high-stakes exams may help reduce historical equity gaps in introductory gateway STEM courses. Herein, we describe the implementation of two versions of mastery grading systems in large enrollment general chemistry courses at a public R1 institution. Class-wide course outcomes, equity gaps in performance on a common final exam, and student feedback on their experience navigating these grading schemes are presented. We show that combining mastery grading with interactive courseware tools improved the average performance on a common final assessment for under-represented minority (URM) students by 7.1 percentage points relative to an active control course that used infrequent high-stakes exams.more » « less
-
Despite recent interest and pressing need, we lack a clear model of culturally relevant, responsive, sensitive teaching in university STEM departments. Most culturally relevant efforts within STEM education address actions individual professors can take within their own classrooms and mentoring, rather than describing how to go about enacting cultural transformation at the departmental level. In this article, we propose the application of the Ladson-Billings model of culturally relevant pedagogy to promote an inclusive culture within undergraduate STEM departments. The model consists of three components: academic success, cultural competence and integrity, and critical consciousness. We define each component and describe what it looks like and how it can be used to guide departmental transformation, including examples in biology, physics, mathematics, and computer science departments at our own institution. This model can help guide faculty committed to creating departments where all kinds of STEM students can thrive, provided they are willing to work hard.more » « less
-
As part of our Noyce Capacity Building grant, we examined how existing STEM course content is preparing students for the Missouri Content Assessment (MoCA) exam that is required for state certification. We also examined this through the lens of the recent curriculum designed on the state of Missouri model, which is Unified Science: Biology. Our most recent graduates in STEM Education exhibited difficulty in passing the MoCA exam in their specific STEM area of interest. Data from 2014-2018 showed that only half of our teacher candidates passed their STEM field MoCA exam on the first attempt. Some candidates took as many as four attempts to pass their exams while others were unable to achieve a passing score. In order to increase the number of teacher candidates as well as their success rates on the MoCA exam, we planned targeted revisions to our curriculum and coursework at Webster. One approach we used was to have content experts (faculty from Webster and our partner institution, St. Louis Community College) take the MoCA exams in their respective STEM disciplines. The qualitative survey results, coded using NVivo, and feedback from these experts allowed us to map learning outcomes and knowledge areas found on the exam to information on the MoCA website as well as our existing curriculum and course content. In this presentation, we will report the results of our survey, the impact this had on our curriculum redesign and articulation agreements, and our future strategies for our Track 1 application that were informed by the results of our capacity building activity. We also include use of study guides, practice tests and review sessions for MoCA exams to impact/improve pass rates of STEM Education majors on the content exam.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1126/sciadv.aau0554
