Search for: All records

Abstract Validity is a fundamental consideration of test development and test evaluation. The purpose of this study is to define and reify three key aspects of validity and validation, namely test‐score interpretation, test‐score use, and the claims supporting interpretation and use. This study employed a Delphi methodology to explore how experts in validity and validation conceptualize test‐score interpretation, use, and claims. Definitions were developed through multiple iterations of data collection and analysis. By clarifying the language used when conducting validation, validation may be more accessible to a broader audience, including but not limited to test developers, test users, and test consumers. 

Quantitative measures in mathematics education have informed policies and practices for over a century. Thus, it is critical that such measures in mathematics education have sufficient validity evidence to improve mathematics experiences for students. This article provides a systematic review of the validity evidence related to measures used in elementary mathematics education. The review includes measures that focus on elementary students as the unit of analyses and attends to validity as defined by current conceptions of measurement. Findings suggest that one in ten measures in mathematics education include rigorous evidence to support intended uses. Recommendations are made to support mathematics education researchers to continue to take steps to improve validity evidence in the design and use of quantitative measures. 

Assessment continues to be an important conversation point within Science, Technology, Engineering, and Mathematics (STEM) education scholarship and practice (Krupa et al., 2019; National Research Council, 2001). There are guidelines for developing and evaluating assess- ments (e.g., AERA et al., 2014; Carney et al., 2022; Lavery et al., 2019; Wilson & Wilmot, 2019). There are also Standards for Educational & Psychological Testing (Standards; AERA et al., 2014) that discuss important rele- vant frameworks and information about using assessment results and interpretations. Quantitative assessments are used as part of daily STEM instruction, STEM research, and STEM evaluation; therefore, having robust assess- ments is necessary (National Research Council, 2001). An aim of this editorial is to give readers a few relevant ideas about modern assessment research, some guidance for the use of quantitative assessments, and framing validation and assessment research as equity-forward work. 

Recent calls to action focus on using educational tools that promote mathematics learning through evidence-based and equity-forward practices (NCTM, 2018). These practices may be derived from scholarship that examines factors related to mathematics teaching and learning using quantitative measures. A purpose of this presentation is to highlight areas of strength and opportunity related to the use of quantitative measures in scholarship examining K-12 mathematics settings. One outcome from this research-in progress is that scholars may become more aware of quantitative assessments for use in their research. A second outcome from this research is to foster conversations among colleagues around collaborative scholarship as well as areas for growth within mathematics education assessment. As a result, scholars may be better equipped to engage in quantitative research within mathematics contexts. Recognizing what is available and relevant to a desired area of study has potential to address contexts connected to topics described in Catalyzing Change (NCTM, 2018, 2020, 2020). That is, scholars cannot quantitatively measure constructs described in Catalyzing Change until it is known what measures are available and what they assess. This research-in progress aims to engage researchers in ongoing research and promote discussions across attendees. 

The Universal Screeners for Number Sense (USNS) measure the construct of number sense, which has been characterized in numerous ways over the last 50 years. A purpose of this study is to provide validity evidence regarding the USNS assessments, which may support scholarship on numeracy and practical work in K-12 schools. Findings indicate that the USNS assessments may be used confidently to measure students’ growth in number sense. 

The paper reports on the developments of a repository of quantitative assessments used in mathematics education contexts. This repository centralizes assessments and the associated validity evidence. The repository is public and freely available and has potential to inform future quantitative mathematics education scholarship. 

Problem solving is a central focus of mathematics teaching and learning. If teachers are expected to support students' problem-solving development, then it reasons that teachers should also be able to solve problems aligned to grade level content standards. The purpose of this validation study is twofold: (1) to present evidence supporting the use of the Problem Solving Measures Grades 3–5 with preservice teachers (PSTs), and (2) to examine PSTs' abilities to solve problems aligned to grades 3–5 academic content standards. This study used Rasch measurement techniques to support psychometric analysis of the Problem Solving Measures when used with PSTs. Results indicate the Problem Solving Measures are appropriate for use with PSTs, and PSTs' performance on the Problem Solving Measures differed between first-year PSTs and end-of-program PSTs. Implications include program evaluation and the potential benefits of using K-12 student-level assessments as measures of PSTs' content knowledge. 

Validity and validation is central to conducting high quality quantitative mathematics education scholarship. This presentation aims to support scholars engaged in quantitative research by providing information about the degrees to which validity evidence related to their instrument use or interpretation, were found in mathematics education scholarship. Findings have potential to steer future quantitatively focused scholarship and support equity aims. 

The COVID-19 pandemic disrupted many school accountability systems that rely on student-level achievement data. Many states encountered uncertainty about how to meet federal accountability requirements without typical school data. Prior research provides evidence that student achievement is correlated to students’ social background, which raises concerns about the predictive bias of accountability systems. This mixed-methods study (a) examines the predictive ability of non-achievement-based variables (i.e., students’ social background) on school districts’ report card letter grade in Ohio, and (b) explores educators’ perceptions of report card grades. Results suggest that social background and community demographic variables have a significant impact on measures of school accountability.