skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Evidence for validity and reliability of a research-based assessment instrument on measurement uncertainty
The Survey of Physics Reasoning on Uncertainty Concepts in Experiments (SPRUCE) was designed to measure students’ proficiency with measurement uncertainty concepts and practices across ten different assessment objectives to help facilitate the improvement of laboratory instruction focused on this important topic. To ensure the reliability and validity of this assessment, we conducted a comprehensive statistical analysis using classical test theory. This analysis includes an evaluation of the test as a whole, as well as an in-depth examination of individual items and assessment objectives. We make use of a previously reported on scoring scheme involving pairing items with assessment objectives, creating a new unit for statistical analysis referred to as a “couplet.” The findings from our analysis provide evidence for the reliability and validity of SPRUCE as an assessment tool for undergraduate physics labs. This increases both instructors’ and researchers’ confidence in using SPRUCE for measuring students’ proficiency with measurement uncertainty concepts and practices to ultimately improve laboratory instruction. Additionally, our results using couplets and assessment objectives demonstrate how these can be used with traditional classic test theory analysis. Published by the American Physical Society2024  more » « less
Award ID(s):
2317149
PAR ID:
10591363
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
American Physical Society
Date Published:
Journal Name:
Physical Review Physics Education Research
Volume:
20
Issue:
2
ISSN:
2469-9896
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Physical Review Physics Education Research)
    Concepts and practices surrounding measurement uncertainty are vital knowledge for physicists and are often emphasized in undergraduate physics laboratory courses. We have previously developed a research-based assessment instrument—the Survey of Physics Reasoning on Uncertainty Concepts in Experiments (SPRUCE)—to examine student proficiency with measurement uncertainty along a variety of axes, including sources of uncertainty, handling of uncertainty, and distributions and repeated measurements. We present here initial results from the assessment representing over 1500 students from 20 institutions. We analyze students’ performance pre- and postinstruction in lab courses and examine how instruction impacts students with different majors and gender. We find that students typically excel in certain areas, such as reporting the mean of a distribution as their result, while they struggle in other areas, such as comparing measurements with uncertainty and correctly propagating errors using formulas. Additionally, we find that the importance that an instructor places in certain areas of measurement uncertainty is uncorrelated with student performance in those areas. Published by the American Physical Society2024 
    more » « less
  2. ; ; ; ; (, Physical Review Physics Education Research)
    In introductory physics laboratory instruction, students often expect to confirm or demonstrate textbook physics concepts. This expectation is largely undesirable: labs that emphasize confirmation of textbook physics concepts are generally unsuccessful at teaching those concepts and even in contexts that do not emphasize confirmation, such expectations can lead to students disregarding or manipulating their data in order to obtain the expected result. In other words, when students expect their lab activities to confirm a known result, they may relinquish epistemic agency and violate disciplinary practices. We present a contrasting case where, we claim, confirmatory expectations can actually support productive disciplinary engagement. In this case study, we analyze the complex dynamics of students’ epistemological framing in a lab where students’ confirmatory expectations support and even generate epistemic agency and disciplinary practices, including developing original ideas, measures, and apparatuses to apply to the material world. Published by the American Physical Society2024 
    more » « less
  3. ; ; (, Physical Review Physics Education Research)
    [This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] One of the greatest weaknesses of physics education research is the paucity of research on graduate education. While there are a growing number of investigations of graduate student degree progress and admissions, there are very few investigations of at the graduate level. Additionally, existing studies of learning in physics graduate programs frequently focus on content knowledge rather than professional skills such as problem solving. Given that over 90% of physics Ph.D. graduates report solving technical problems regularly in the workplace, we sought to develop an assessment to measure how well graduate programs are training students to solve problems. Using a framework that characterizes expert-like problem-solving skills as a set of decisions to be made, we developed and validated such an assessment in graduate quantum mechanics (QM) following recently developed design frameworks for measuring problem solving and best practices for assessment validation. We collected validity evidence through think-aloud interviews with practicing physicists and physics graduate students, as well as written solutions provided by physics graduate and undergraduate students. The assessment shows strong potential in differentiating novice and expert problem solving in QM and showed reliability in repeated testing with similar populations. These results show the promise of measuring expert decision making in graduate QM and provide baseline measurements for future educational interventions to more effectively teach these skills. Published by the American Physical Society2025 
    more » « less
  4. ; (, Physical Review Physics Education Research)
    [This paper is part of the Focused Collection in Investigating and Improving Quantum Education through Research.] Research-based multiple-choice questions implemented in class with peer instruction have been shown to be an effective tool for improving students’ engagement and learning outcomes. Moreover, multiple-choice questions that are carefully sequenced to build on each other can be particularly helpful for students to develop a systematic understanding of concepts pertaining to a theme. Here, we discuss the development, validation, and implementation of a multiple-choice question sequence (MQS) on the topic of quantum measurement in the context of wave functions in the infinite-dimensional Hilbert space. This MQS was developed using students’ common difficulties with quantum measurements as a guide and was implemented in a junior-/senior-level quantum mechanics course at a large research university in the U.S. We compare student performance on assessment tasks focusing on quantum measurement before and after the implementation of the MQS and discuss how different difficulties were reduced and how to further improve students’ conceptual understanding of quantum measurement in infinite-dimensional Hilbert space. Published by the American Physical Society2025 
    more » « less
  5. ; ; (, Physical Review Physics Education Research)
    We examine students’ challenges in determining the number of distinct many-particle stationary states for a system of noninteracting identical particles, focusing on how these insights guided the design, validation, and evaluation of a quantum interactive learning tutorial (QuILT) to aid students’ understanding. Specifically, we focus on systems with a fixed number of available single-particle states and particles, where the total energy is not fixed. The QuILT is designed to provide scaffolding support to help students learn these complex concepts more effectively. This study was conducted in advanced quantum mechanics courses, where written questions were administered to students in class following traditional instruction on the relevant concepts. Additionally, individual interviews were conducted with students to gain deeper insights. Our findings reveal that both upper-level undergraduate and graduate students face similar challenges in understanding these concepts. Additionally, difficulty with basic concepts in combinatorics that are necessary to answer the questions correctly was also found. The QuILT offers scaffolding support to help undergraduate and graduate students systematically reason through these concepts. Published by the American Physical Society2024 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email