More Like this

1. Evidence for validity and reliability of a research-based assessment instrument on measurement uncertainty

   https://doi.org/10.1103/PhysRevPhysEducRes.20.020125  

   Geschwind, Gayle; Vignal, Michael; Caballero, Marcos D; Lewandowski, H J (October 2024, Physical Review Physics Education Research)

   The Survey of Physics Reasoning on Uncertainty Concepts in Experiments (SPRUCE) was designed to measure students’ proficiency with measurement uncertainty concepts and practices across ten different assessment objectives to help facilitate the improvement of laboratory instruction focused on this important topic. To ensure the reliability and validity of this assessment, we conducted a comprehensive statistical analysis using classical test theory. This analysis includes an evaluation of the test as a whole, as well as an in-depth examination of individual items and assessment objectives. We make use of a previously reported on scoring scheme involving pairing items with assessment objectives, creating a new unit for statistical analysis referred to as a “couplet.” The findings from our analysis provide evidence for the reliability and validity of SPRUCE as an assessment tool for undergraduate physics labs. This increases both instructors’ and researchers’ confidence in using SPRUCE for measuring students’ proficiency with measurement uncertainty concepts and practices to ultimately improve laboratory instruction. Additionally, our results using couplets and assessment objectives demonstrate how these can be used with traditional classic test theory analysis. Published by the American Physical Society2024 

   more » « less
2. Lifelong and lifewide learning for the perpetual development of expertise in engineering

   https://doi.org/10.1080/03043797.2021.1944064  

   Johri, Aditya (June 2021, European Journal of Engineering Education)

   Increasing digitisation of engineering and social practices has altered the relationship between formal schooling and development of expertise for professional engineering work. What does the development of expertise look like when knowledge is generated and shared at an accelerated pace due to shifts in technology? In this paper, I present case studies of two early career software engineers. Using methodological insights from digital ethnography, I trace their professional journeys over two decades. I empirically demonstrate how the development of engineering expertise is a continuous and perpetual endeavour and engineers learn throughout their lives (lifelong) and across all the different spaces they inhabit at any given time (lifewide). I argue for extending engineering work practices research and research in engineering education more broadly to take larger timescales of learning into account to build a comprehensive understanding of engineering expertise development. 

   more » « less
3. Who is the best engineer?: Identity theory as a framework to reflect on our role in the construction of beliefs about the value of social responsibility in engineering

   Dringenberg, E (December 2024, SEFI Annual Conference)

   The 2024 SEFI conference posed the question, “How can we ensure the highest quality of technical competence while at the same time ensuring that social and environmental responsibility is core to the identity of engineering graduates?” Identity formation is a complex process that has been theorized in many ways. In this workshop, I invited participants to consider Holland and colleagues’ theory of identity as a useful framework for reflecting on our how our participation in engineering education contributes to beliefs about what makes a “real” or the “best” engineer. This theory posits that within our classrooms, students are participating in a complex cultural practice through which they ultimately learn to identify (and be identified) as more or less of an engineer than others. Our everyday classroom practices ultimately function to co-construct 1) shared beliefs about what makes a “good” engineer, and 2) everyone’s relative position in a social hierarchy. Furthermore, identify development is theorized to include both social forces (i.e., rules and guidelines that influence how people behave in a social space) and individual agency (i.e., we are not just carbon copies of culture or norms because our actions shape the culture and norms). Understanding identity development as such empowers us to be intentional with our own participation in identity construction by providing theoretical entry points for conveying the value of social responsibility. The usefulness of this particular identity theory to ideate strategies for integrating social responsibility into students’ engineering identities has been corroborated by the empirical findings of our U.S.-based engineering education research. During this workshop, we utilized the theory to draw out existing or future concrete practices that each of us, given our unique global and institutional contexts, are motivated to enact in support of social responsibility as core to engineering. Specifically, our interactions culminated with answering the following question: What is one concrete way I can be intentional in how I participate in identity co-construction? Participant responses to this prompt are presented directly. 

   more » « less
4. Measures of mathematics teachers’ behavior and affect: An examination of the assessment landscape

   Gallagher, M.; Bardelli, E.; Folger, T.; Neely, A.; Bostic, J.; Walkowiak, T.; Wilhelm, A.; Zelkowski, J. (April 2022, Annual meeting program American Educational Research Association)

   Although the paradigm wars between quantitative and qualitative research methods and the associated epistemologies may have settled down in recent years within the mathematics education research community, the high value placed on quantitative methods and randomized control trials remain as the gold standard at the policy-making level (USDOE, 2008). Although diverse methods are valued in the mathematics education community, if mathematics educators hope to influence policy to cultivate more equitable education systems, then we must engage in rigorous quantitative research. However, quantitative research is limited in what it can measure by the quantitative tools that exist. In mathematics education, it seems as though the development of quantitative tools and studying their associated validity and reliability evidence has lagged behind the important constructs that rich qualitative research has uncovered. The purpose of this study is to describe quantitative instruments related to mathematics teacher behavior and affect in order to better understand what currently exists in the field, what validity and reliability evidence has been published for such instruments, and what constructs each measure. 1. How many and what types of instruments of mathematics teacher behavior and affect exist? 2. What types of validity and reliability evidence are published for these instruments? 3. What constructs do these instruments measure? 4. To what extent have issues of equity been the focus of the instruments found? 

   more » « less
5. Impact of the Emerging Engineering Education Research and Innovation Community

   https://doi.org/10.18260/1-2--34759  

   Fentiman, Audeen; Riley, Donna; Litzler, Elizabeth; London, Jeremi; Williams, Julia; Case, Jennifer (June 2020, ASEE Annual Conference and Exposition)

   The Engineering Education departments at three large public universities are collaborating on an NSF-funded program to document the impact of the emerging EER&I community. This paper is a report on what has been learned to date. Goals of the program include (1) identifying the broader EER&I network, (2) identifying examples of EER&I impact, (3) organizing and hosting a summit of EER&I leaders to develop a systematic process for documenting the impact of EER&I, (4) piloting the process, and (5) compiling and disseminating best practices. Members of the community have been identified, including many who are conducting engineering education research without being part of a formal engineering education program, and some examples of the impact of engineering education research have been gathered. The summit has been held, and a process for documenting the impact of EER&I has been proposed. Results of the summit include a range of possible metrics that can be used to document EER&I impact and ways to communicate that impact. Some pilots have been conducted at the three collaborating schools and several other sites, and a few institutions are now preparing documentation. Results of the summit and the pilots will be shared. In their pilots, engineering education programs have been able to collect and analyze data that describe their efforts to impact how engineering is taught at the university level. Quantitative metrics include research expenditures, publications, number of graduates, positions graduates hold, faculty leadership in groups that influence engineering education policy, and so on. It has proven to be more difficult to demonstrate a direct causal relationship between those efforts and actual changes in the way engineering is taught in the traditional disciplines. The path to each change seems to be unique, and the most effective way to convey the impact is through telling each individual story. Thus, ongoing work focuses on generating a range of qualitative approaches that can be used to document and analyze these change processes. Collaborators on the NSF program are currently piloting ways to convey those stories to the many audiences interested in the results. 

   more » « less