Search for: All records

The Engineering Research Centers (ERCs), funded by the National Science Foundation (NSF), play an important role in improving engineering education, bridging engineering academia and broad communities, and promoting a culture of diversity and inclusion. Each ERC must partner with an independent evaluation team to annually assess their performance and impact on progressing education, connecting community, and building diversified culture. This evaluation is currently performed independently (and in isolation), which leads to inconsistent evaluations and a redundant investment of ERCs’ resources into such tasks (e.g. developing evaluation instruments). These isolated efforts by ERCs to quantitatively evaluate their education programs also typically lack adequate sample size within a single center, which limits the validity and reliability of the quantitative analyses. Three ERCs, all associated with a large southwest university in the United States, worked collaboratively to overcome sample size and measure inconsistency concerns by developing a common quantitative instrument that is capable of evaluating any ERC’s education and diversity impacts. The instrument is the result of a systematic process with comparing and contrasting each ERC’s existing evaluation tools, including surveys and interview protocols. This new, streamlined tool captures participants’ overall experience as part of the ERC by measuring various constructs including skillset development, perception of diversity and inclusion, future plans after participating in the ERC, and mentorship received from the ERC. Scales and embedded items were designed broadly for possible use with both yearlong (e.g. graduate and undergraduate student, and postdoctoral scholars) and summer program (Research Experience for Undergraduates, Research Experience for Teachers, and Young Scholar Program) participants. The instrument was distributed and tested during Summer 2019 with participants in the summer programs from all three ERCs. The forthcoming paper will present the new common cross-ERC evaluation instrument, demonstrate the effort of collecting data across all three ERCs, present preliminary findings, and discuss collaborative processes and challenges. The preliminary implication for this work is the ability to directly compare educational programs across ERCs. The authors also believe that this tool can provide a fast start for new ERCs on how to evaluate their educational programs. 

This Innovative Practice Work in Progress paper presents the collaborative efforts made by three NSF-funded Engineering Research Centers (ERCs) to synthesize common tools for educational program evaluation. The aim of the NSF ERCs is to achieve transformative changes by integrating engineering research and education with technological innovation within areas at the frontiers of science and engineering (e.g., NSF's 10 Big Ideas). Such centers across the nation study and innovate within their technical area using similar structures and implementation strategies, including the coordination of educational endeavors. Independent partners are enlisted as part of these centers to evaluate education and diversity impacts annually. Each center typically performs this task in isolation from other such centers. The effort required to create resources for such evaluation outcome can result in redundancy and an inability for psychometric analysis due to small available populations within a single center. This paper elaborates on the ongoing efforts of this collaborative research aimed at addressing these issues by creating a streamlined, customizable, and standardized set of evaluation instruments that can be applied to any ERC evaluation.