This Brief Report presents an example of assessment validation using an argument-based approach. The instrument we developed is a Brief Assessment of Students’ Mature Number Sense, which measures a central goal in mathematics education. We chose to develop this assessment to provide an efficient way to measure the effect of instructional practices designed to improve students’ number sense. Using an argument-based framework, we first identify our proposed interpretations and uses of student scores. We then outline our argument with three claims that provide evidence connecting students’ responses on the assessment with its intended uses. Finally, we highlight why using argument-based validation benefits measure developers as well as the broader mathematics education community.
A validation argument for the Priorities for Mathematics Instruction (PMI) survey.
Mathematics education needs measures that can be used to research and/or evaluate the impact of professional development for constructs that are broadly relevant to the field. To address this need we developed the Priorities for Mathematics Instruction (PMI) survey consisting of two scales focused on the constructs of Explicit Attention to Concepts (EAC) and Student Opportunities to Struggle (SOS) – which have been linked to increased student understanding
and achievement. We identified the most critical assumptions that underlie the proposed interpretation and use of the scale scores and then examined the related validity evidence. We found the evidence for each assumption supports the proposed interpretation and use of the scale scores.
more »
« less
- Award ID(s):
- 1907840
- NSF-PAR ID:
- 10336663
- Editor(s):
- Olanoff, D.; Johnson, K.; Spitzer, S. M.
- Date Published:
- Journal Name:
- The 43rd Meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education, Virtual/Philadelphia, USA.
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
-
The purpose of this work-in-progress paper is to share insights from current efforts to develop and test the validity of an instrument to measure undergraduate students’ perceived support in science, technology, engineering, and mathematics (STEM). The development and refinement of our survey instrument ultimately functions to extend, operationalize, and empirically test the Model of Co-curricular Support (MCCS). The MCCS is a conceptual framework of student support that demonstrates the breadth of assistance currently used to support undergraduate students in STEM, particularly those from underrepresented groups. We are currently gathering validity evidence for an instrument that evaluates the extent to which colleges of engineering and science offer supportive environments. To date, exploratory factor analysis and correlation for construct validity have helped us develop 14 constructs for student support in STEM. Future work will focus on modeling relationships between these constructs and student outcomes, providing the explanatory power needed to explain empirically how co-curricular supports contribute to different forms of student success in STEM. We hope that operationalizing the MCCS through this survey will shift how we conceptualize and offer student support, enabling college administrators and student support practitioners to evaluate their portfolio of student support efforts.more » « less
-
Undergraduate science, technology, engineering, and mathematics (STEM) students’ motivations have a strong influence on whether and how they will persist through challenging coursework and into STEM careers. Proper conceptualization and measurement of motivation constructs, such as students’ expectancies and per- ceptions of value and cost (i.e., expectancy value theory [EVT]) and their goals (i.e., achievement goal theory [AGT]), are necessary to understand and enhance STEM persistence and success. Research findings suggest the importance of exploring multiple measurement models for motivation constructs, including traditional con- firmatory factor analysis, exploratory structural equation models (ESEM), and bifactor models, but more research is needed to determine whether the same model fits best across time and context. As such, we mea- sured undergraduate biology students’ EVT and AGT motivations and investigated which measurement model best fit the data, and whether measurement invariance held, across three semesters. Having determined the best- fitting measurement model and type of invariance, we used scores from the best performing model to predict biology achievement. Measurement results indicated a bifactor-ESEM model had the best data-model fit for EVT and an ESEM model had the best data-model fit for AGT, with evidence of measurement invariance across semesters. Motivation factors, in particular attainment value and subjective task value, predicted small yet statistically significant amounts of variance in biology course outcomes each semester. Our findings provide support for using modern measurement models to capture students’ STEM motivations and potentially refine conceptualizations of them. Such future research will enhance educators’ ability to benevolently monitor and support students’ motivation, and enhance STEM performance and career success.more » « less
-
more » « less
Abstract Greenspace in schools might enhance students' academic performance. However, the literature—dominated by ecological studies at the school level in countries from the Northern Hemisphere—presents mixed evidence of a beneficial association. We evaluated the association between school greenness and student‐level academic performance in Santiago, Chile, a capital city of the Global South. This cross‐sectional study included 281,695 fourth‐grade students attending 1,498 public, charter, and private schools in Santiago city between 2014 and 2018. Student‐level academic performance was assessed using standardized test scores and indicators of attainment of learning standards in mathematics and reading. School greenness was estimated using Normalized Difference Vegetation Index (NDVI). Linear and generalized linear mixed‐effects models were fit to evaluate associations, adjusting for individual‐ and school‐level sociodemographic factors. Analyses were stratified by school type. In fully adjusted models, a 0.1 increase in school greenness was associated with higher test scores in mathematics (36.9 points, 95% CI: 2.49; 4.88) and in reading (1.84 points, 95% CI: 0.73; 2.95); as well as with higher odds of attaining learning standards in mathematics (OR: 1.20, 95% CI: 1.12; 1.28) and reading (OR: 1.07, 95% CI: 1.02; 1.13). Stratified analysis showed differences by school type, with associations of greater magnitude and strength for students attending public schools. No significant associations were detected for students in private schools. Higher school greenness was associated with improved individual‐level academic outcomes among elementary‐aged students in a capital city in South America. Our results highlight the potential of greenness in the school environment to moderate educational and environmental inequalities in urban areas.
-
High levels of stress and anxiety are common amongst college students, particularly engineering students. Students report lack of sleep, grades, competition, change in lifestyle, and other significant stressors throughout their undergraduate education (1, 2). Stress and anxiety have been shown to negatively impact student experience (3-6), academic performance (6-8), and retention (9). Previous studies have focused on identifying factors that cause individual students stress while completing undergraduate engineering degree programs (1). However, it not well-understood how a culture of stress is perceived and is propagated in engineering programs or how this culture impacts student levels of identification with engineering. Further, the impact of student stress has not been directly considered in engineering regarding recruitment, retention, and success. Therefore, our guiding research question is: Does the engineering culture create stress for students that hinder their engineering identity development? To answer our research question, we designed a sequential mixed methods study with equal priority of quantitative survey data and qualitative individual interviews. Our study participants are undergraduate engineering students across all levels and majors at a large, public university. Our sample goal is 2000 engineering student respondents. We combined three published surveys to build our quantitative data collection instrument, including the Depression Anxiety Stress Scales (DASS), Identification with engineering subscale, and Engineering Department Inclusion Level subscale. The objective of the quantitative instrument is to illuminate individual perceptions of the existence of an engineering stress culture (ESC) and create an efficient tool to measure the impact ESC on engineering identity development. Specifically, we seek to understand the relationships among the following constructs; 1) identification with engineering, 2) stress and anxiety, and 3) feelings of inclusion within their department. The focus of this paper presents the results of the pilot of the proposed instrument with 20 participants and a detailed data collection and analysis process. In an effort to validate our instrument, we conducted a pilot study to refine our data collection process and the results will guide the data collection for the larger study. In addition to identifying relationships among construct, the survey data will be further analyzed to specify which demographics are mediating or moderating factors of these relationships. For example, does a student’s 1st generation status influence their perception of stress or engineering identity development? Our analysis may identify discipline-specific stressors and characterize culture components that promote student anxiety and stress. Our objective is to validate our survey instrument and use it to inform the protocol for the follow-up interviews to gain a deeper understanding of the responses to the survey instrument. Understanding what students view as stressful and how students identify stress as an element of program culture will support the development of interventions to mitigate student stress. References 1. Schneider L (2007) Perceived stress among engineering students. A Paper Presented at St. Lawrence Section Conference. Toronto, Canada. Retrieved from: www. asee. morrisville. edu. 2. Ross SE, Niebling BC, & Heckert TM (1999) Sources of stress among college students. Social psychology 61(5):841-846. 3. Goldman CS & Wong EH (1997) Stress and the college student. Education 117(4):604-611. 4. Hudd SS, et al. (2000) Stress at college: Effects on health habits, health status and self-esteem. College Student Journal 34(2):217-228. 5. Macgeorge EL, Samter W, & Gillihan SJ (2005) Academic Stress, Supportive Communication, and Health A version of this paper was presented at the 2005 International Communication Association convention in New York City. Communication Education 54(4):365-372. 6. Burt KB & Paysnick AA (2014) Identity, stress, and behavioral and emotional problems in undergraduates: Evidence for interaction effects. Journal of college student development 55(4):368-384. 7. Felsten G & Wilcox K (1992) Influences of stress and situation-specific mastery beliefs and satisfaction with social support on well-being and academic performance. Psychological Reports 70(1):291-303. 8. Pritchard ME & Wilson GS (2003) Using emotional and social factors to predict student success. Journal of college student development 44(1):18-28. 9. Zhang Z & RiCharde RS (1998) Prediction and Analysis of Freshman Retention. AIR 1998 Annual Forum Paper.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
