An official website of the United States government
Education researchers often compare performance across race and gender on research-based assessments of physics knowledge to investigate the impacts of racism and sexism on physics student learning. These investigations' claims rely on research-based assessments providing reliable, unbiased measures of student knowledge across social identity groups. We used classical test theory and differential item functioning (DIF) analysis to examine whether the items on the Force Concept Inventory (FCI) provided unbiased data across social identifiers for race, gender, and their intersections. The data was accessed through the Learning About STEM Student Outcomes platform and included responses from 4,848 students posttests in 152 calculus-based introductory physics courses from 16 institutions. The results indicated that the majority of items (22) on the FCI were biased towards a group. These results point to the need for instrument validation to account for item bias and the identification or development of fair research-based assessments.
more »
« less
This content will become publicly available on April 1, 2026
Is the Force Concept Inventory biased across the intersections of gender and race?
Research-based assessments (RBAs) allow researchers and practitioners to compare student performance across different contexts and institutions. In recent years, research attention has focused on the student populations these RBAs were initially developed with because much of that research was done with “samples of convenience” that were predominantly white men. Prior research has found that the Force Concept Inventory (FCI) behaved differently for men and women using differential item functioning (DIF) analysis. We extend this research in two ways. First, we test the FCI for DIF across the intersection of gender and race for Asian, Black, Hispanic, White, and White Hispanic men and women. Second, we apply the Eaton and Willoughby five-factor model of the FCI to interpret the results of the DIF analysis. We found large DIF on a large number of FCI items. The patterns of items with large DIF follows the five-factor model. The alignment of DIF with this factor structure, along with the measurement invariance of this factor structure across these ten social identities, indicates that the items on the FCI are likely not biased but are instead measuring real differences in physics knowledge among these groups. We frame these differences as educational debts that society owes to these marginalized groups that physics instruction needs to actively repay.
more »
« less
- PAR ID:
- 10639650
- Publisher / Repository:
- American Physical Society
- Date Published:
- Journal Name:
- Physical Review Physics Education Research
- Volume:
- 21
- Issue:
- 1
- ISSN:
- 2469-9896
- Subject(s) / Keyword(s):
- Physics
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract This paper estimates the long-run impacts of banning affirmative action on men and women from under-represented minority (URM) racial and ethnic groups in the United States. Using data from the US Census and American Community Survey, we use a difference-in-differences framework to compare the college degree completion, graduate degree completion, earnings, and employment of URM individuals to non-URM individuals before and after affirmative action bans went into effect across several US states. We also employ event study analyses and alternative estimators to confirm the validity of our approach and discuss the generalizability of the findings. Results suggest that banning affirmative action results in a decline in URM women’s college degree completion, earnings, and employment relative to non-Hispanic White women, driven largely by impacts on Hispanic women. Thus, affirmative action bans resulted in an increase in racial/ethnic disparities in both college degree completion and earnings among women. Effects on URM men are more ambiguous and indicate significant heterogeneity across states, with some estimates pointing to a possible positive impact on labour market outcomes of Black men. These results suggest that the relative magnitude of college quality versus mismatch effects vary for URM men and women and highlight the importance of disaggregating results by gender, race, and ethnicity. We conclude by discussing how our results compare with others in the literature and directions for future research.more » « less
-
It is well established that access to social supports is essential for engineering students’ persistence and yet access to supports varies across groups. Understanding the differential supports inherent in students’ social networks and then working to provide additional needed supports can help the field of engineering education become more inclusive of all students. Our work contributes to this effort by examing the reliability and fairness of a social capital instrument, the Undergraduate Supports Survey (USS). We examined the extent to which two scales were reliable across ability levels (level of social capital), gender groups and year-in-school. We conducted two item response theory (IRT) models using a graded response model and performed differential item functioning (DIF) tests to detect item differences in gender and year-in-school. Our results indicate that most items have acceptable to good item discrimination and difficulty. DIF analysis shows that multiple items report DIF across gender groups in the Expressive Support scale in favor of women and nonbinary engineering students. DIF analysis shows that year-in-school has little to no effect on items, with only one DIF item. Therefore, engineering educators can use the USS confidently to examine expressive and instrumental social capital in undergraduates across year-in-school. Our work can be used by the engineering education research community to identify and address differences in students’ access to support. We recommend that the engineering education community works to be explicit in their expressive and instrumental support. Future work will explore the measurement invariance in Expressive Support items across gender.more » « less
-
It is well established that access to social supports is essential for engineering students’ persistence and yet access to supports varies across groups. Understanding the differential supports inherent in students’ social networks and then working to provide additional needed supports can help the field of engineering education become more inclusive of all students. Our work contributes to this effort by examing the reliability and fairness of a social capital instrument, the Undergraduate Supports Survey (USS). We examined the extent to which two scales were reliable across ability levels (level of social capital), gender groups and year-in-school. We conducted two item response theory (IRT) models using a graded response model and performed differential item functioning (DIF) tests to detect item differences in gender and year-in-school. Our results indicate that most items have acceptable to good item discrimination and difficulty. DIF analysis shows that multiple items report DIF across gender groups in the Expressive Support scale in favor of women and nonbinary engineering students. DIF analysis shows that year-in-school has little to no effect on items, with only one DIF item. Therefore, engineering educators can use the USS confidently to examine expressive and instrumental social capital in undergraduates across year-in-school. Our work can be used by the engineering education research community to identify and address differences in students’ access to support. We recommend that the engineering education community works to be explicit in their expressive and instrumental support. Future work will explore the measurement invariance in Expressive Support items across gender.more » « less
-
The American Physical Society calls on its members to improve the diversity of physics by sup- porting an inclusive culture that encourages women and people of color to become physicists. Becoming a physicist demands a set of beliefs about what it means to learn and do physics. Rather than physics courses and degree programs supporting students in developing these beliefs, evidence shows that physics education filters out students without sufficient beliefs. To better understand the role of beliefs in the lack of diversity in physics, we investigated the intersectional nature of race/racism and gender/sexism in inequities in student beliefs towards learning and doing physics using a critical quantitative intersectionality framework. The analyses used hierarchical linear models to examine students’ beliefs as measured by the Colorado learning attitudes about science survey. The data came from the LASSO database and included 1248 students in 29 calculus-based mechanics courses. Like prior studies, we found that beliefs either did not change or slightly decreased for most groups. Results identified large differences across intersecting race and gender groups. White students, particularly White men, tended to have more expert-like beliefs than any other group of students. Physics instruction must address these educational debts to move toward an inclusive culture supportive of diverse students and professionals.more » « less
