More Like this

1. GATHERING VALIDITY EVIDENCE TO SUPPORT MATHEMATICS EDUCATION SCHOLARSHIP

   Bostic, J.; Krupa, E.; Folger, T.; Bentley, B.; Stokes, D. (October 2022, Psychology of Mathematics Education North America)

   A. Lischka, E. Dyer (Ed.)

   Validity and validation is central to conducting high quality quantitative mathematics education scholarship. This presentation aims to support scholars engaged in quantitative research by providing information about the degrees to which validity evidence related to their instrument use or interpretation, were found in mathematics education scholarship. Findings have potential to steer future quantitatively focused scholarship and support equity aims. 

   more » « less
2. Content validity evidence for new problem-solving measures (PSM3, PSM4, and PSM5)

   Bostic, J. (January 2018, Proceedings for the 40th Annual Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education)

   Instrument development should adhere to the Standards (AERA et al., 2014). “Content oriented evidence of validation is at the heart of the [validation] process” (AERA et al., 2014, p.15) and is one of the five sources of validity evidence. The research question for this study is: What is the evidence related to test content for the three instruments called the PSM3, PSM4, and PSM5? The study’s purpose is to describe content validity evidence related to new problem-solving measures currently under development. We have previously published validity evidence for problem-solving measures (PSM6, PSM7, and PSM8) that address middle grades math standards (see Bostic & Sondergeld, 2015; Bostic, Sondergeld, Folger, & Kruse, 2017). 

   more » « less
3. Work In Progress: An Object Assembly Test of Sketching in Undergraduate Engineering

   https://doi.org/10.1109/FIE56618.2022.9962634  

   Merzdorf, Hillary E.; Jaison, Donna; Weaver, Morgan B.; Linsey, Julie; Hammond, Tracy; Douglas, Kerrie A. (October 2022, 2022 IEEE Frontiers in Education Conference (FIE))

   This Research Work-In-Progress reports the implementation of an Object Assembly Test for sketching skills in an undergraduate mechanical engineering graphics course. Sketching is essential for generating and refining ideas, and for communication among team members. Design thinking is supported through sketching as a means of translating between internal and external representations, and creating shared representations of collaborative thinking. While many spatial tests exist in engineering education, these tests have not directly used sketching or tested sketching skill. The Object Assembly Test is used to evaluate sketching skills on 3-dimensional mental imagery and mental rotation tasks in 1- and 2-point perspective. We describe revisions to the Object Assembly Test skills and grading rubric since its pilot test, and implement the test in an undergraduate mechanical engineering course for further validation. We summarize inter-rater reliability for each sketching exercise and for each grading metric for a sample of sketches, with discussion of score use and interpretation. 

   more » « less
4. Validity in a different context: Exploring relations to other variables evidence.

   Sondergeld, T. (July 2019, Proceedings of the 43rd Meeting of the International Group for the Psychology of Mathematics Education.)

   Multiple forms of validity evidence should be reviewed to produce assessments with valid and reliable results (AERA, APA, NCME, 2014). Most mathematics validation studies do not, however, investigate beyond content and internal structure (Bostic, Krupa, Carney, & Shih, in press). The purpose of this study is to examine the less commonly reviewed validity evidence of "relationships to other variables" (RTOV) using mathematics problem-solving assessments (PSM3-5) as an example. RTOV explores how test scores may be related to other variables. When RTOV has been examined in mathematics validation studies, it was at the overall test level (see Bostic, Sondergeld, Folger, & Kruse, 2017 for an example). As such, the research question guiding our study is: What information is present when examining RTOV at both the overall test and individual item-levels? 

   more » « less
5. The Development and Validation of a Survey Measuring Opportunity to Learn Spatial Reasoning Skills at Home

   https://doi.org/10.7275/pare.1885  

   Wellberg, Sarah; Sparks, Anthony; Ketterlin-Geller, Leanne (October 2023, Practical assessment research evaluation)

   The early development of spatial reasoning skills has been linked to future success in mathematics (Wai, Lubinski, & Benbow, 2009), but research to date has mainly focused on the development of these skills within classroom settings rather than at home. The home environment is often the first place students are exposed to, and develop, early mathematics skills, including spatial reasoning (Blevins-Knabe, 2016; Hart, Ganley, & Purpura, 2016). The purpose of the current study is to develop a survey instrument to better understand Kindergarten through Grade 2 students’ opportunities to learn spatial reasoning skills at home. Using an argument-based approach to validation (Kane, 2013), we collected multiple sources of validity evidence, including expert review of item wording and content and pilot data from 201 parent respondents. This manuscript outlines the interpretation/use argument that guides our validation study and presents evidence collected to evaluate the scoring inferences for using the survey to measure students’ opportunities to learn spatial reasoning skills at home. 

   more » « less