skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Evaluating The Fairness Of The Undergraduate Supports Survey: A DIF Analysis Of Gender And Year-In-School
It is well established that access to social supports is essential for engineering students’ persistence and yet access to supports varies across groups. Understanding the differential supports inherent in students’ social networks and then working to provide additional needed supports can help the field of engineering education become more inclusive of all students. Our work contributes to this effort by examing the reliability and fairness of a social capital instrument, the Undergraduate Supports Survey (USS). We examined the extent to which two scales were reliable across ability levels (level of social capital), gender groups and year-in-school. We conducted two item response theory (IRT) models using a graded response model and performed differential item functioning (DIF) tests to detect item differences in gender and year-in-school. Our results indicate that most items have acceptable to good item discrimination and difficulty. DIF analysis shows that multiple items report DIF across gender groups in the Expressive Support scale in favor of women and nonbinary engineering students. DIF analysis shows that year-in-school has little to no effect on items, with only one DIF item. Therefore, engineering educators can use the USS confidently to examine expressive and instrumental social capital in undergraduates across year-in-school. Our work can be used by the engineering education research community to identify and address differences in students’ access to support. We recommend that the engineering education community works to be explicit in their expressive and instrumental support. Future work will explore the measurement invariance in Expressive Support items across gender.  more » « less
Award ID(s):
2406960 2129282
PAR ID:
10528575
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
European Society for Engineering Education (SEFI)
Date Published:
Subject(s) / Keyword(s):
item response theory differential item analysis social capital social support assessment
Format(s):
Medium: X
Location:
Dublin, Ireland
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, SEFI Annual Conference, TU Dublin, IE)
    It is well established that access to social supports is essential for engineering students’ persistence and yet access to supports varies across groups. Understanding the differential supports inherent in students’ social networks and then working to provide additional needed supports can help the field of engineering education become more inclusive of all students. Our work contributes to this effort by examing the reliability and fairness of a social capital instrument, the Undergraduate Supports Survey (USS). We examined the extent to which two scales were reliable across ability levels (level of social capital), gender groups and year-in-school. We conducted two item response theory (IRT) models using a graded response model and performed differential item functioning (DIF) tests to detect item differences in gender and year-in-school. Our results indicate that most items have acceptable to good item discrimination and difficulty. DIF analysis shows that multiple items report DIF across gender groups in the Expressive Support scale in favor of women and nonbinary engineering students. DIF analysis shows that year-in-school has little to no effect on items, with only one DIF item. Therefore, engineering educators can use the USS confidently to examine expressive and instrumental social capital in undergraduates across year-in-school. Our work can be used by the engineering education research community to identify and address differences in students’ access to support. We recommend that the engineering education community works to be explicit in their expressive and instrumental support. Future work will explore the measurement invariance in Expressive Support items across gender. 
    more » « less
  2. ; ; ; ; (, ASEE PEER)
    Background Nonbinary engineering students (i.e., engineering students that identify as outside of the gender binary) must navigate a cis- and heteronormative society, in addition to a male dominated engineering culture. Nonbinary students in higher education report high levels of minority threat, lower levels of persistence than cisgender peers and lessened feelings of belonging. One avenue for supporting nonbinary students’ persistence in engineering is to increase understanding of the types of individuals that support nonbinary students and the supports available to nonbinary engineering students. For this study, we are utilizing social capital theory and Devor’s witnessing and mirroring framework to explore the supports nonbinary engineering students access through their social networks and how those supports impact persistence. Social capital, the resources embedded into social relationships, has been used as both an indicator and outcome in relation to students’ well-being, belonging, academic success, and persistence. Devor’s witnessing and mirroring framework brings greater meaning to the value of transgender and cisgender alters in a nonbinary engineers’ network. Specifically, how cisgender alters witness a nonbinary person’s identity as an outsider to gender nonconformity and how transgender alters mirror a nonbinary person’s experience as an insider to gender nonconformity. Purpose: The purpose of this work in progress is to identify supports that are impactful on nonbinary engineering students’ persistence in their majors and careers. Specifically, we ask the questions: 1) What supports do alters, of differing genders, provide that witness and mirror nonbinary engineers; 2) to what extent are supports provided by alters impactful on students' success in their majors and careers? Methodology/approach: We purposely selected 4 nonbinary engineering students to interview from a larger study on engineering students’ social capital and professional skills. Nonbinary engineering students were asked about their experiences of being nonbinary in engineering spaces, the support they received from cisgender and transgender alters and the impact of that support on their persistence in engineering. We analyzed the themes in the interviews by coding the alters mentioned, the types of supports provided (mirroring, witnessing, expressive and instrumental) and the impact of supports on persistence. Future Work and Implications: The finding of this study will contribute to a greater understanding of the social capital and social networks of nonbinary students, as there is dearth of research on their experiences in engineering spaces. This work can guide our understanding of the supports that nonbinary engineering students have available to them and perhaps more importantly, the supports they do not have available to them. The findings from this study will be used to inform organizational and institutional policies to support engineering students in accessing more social capital. 
    more » « less
  3. ; ; ; ; (, ASEE PEER)
    Background Nonbinary engineering students (i.e., engineering students that identify as outside of the gender binary) must navigate a cis- and heteronormative society, in addition to a male dominated engineering culture. Nonbinary students in higher education report high levels of minority threat, lower levels of persistence than cisgender peers and lessened feelings of belonging. One avenue for supporting nonbinary students’ persistence in engineering is to increase understanding of the types of individuals that support nonbinary students and the supports available to nonbinary engineering students. For this study, we are utilizing social capital theory and Devor’s witnessing and mirroring framework to explore the supports nonbinary engineering students access through their social networks and how those supports impact persistence. Social capital, the resources embedded into social relationships, has been used as both an indicator and outcome in relation to students’ well-being, belonging, academic success, and persistence. Devor’s witnessing and mirroring framework brings greater meaning to the value of transgender and cisgender alters in a nonbinary engineers’ network. Specifically, how cisgender alters witness a nonbinary person’s identity as an outsider to gender nonconformity and how transgender alters mirror a nonbinary person’s experience as an insider to gender nonconformity. Purpose: The purpose of this work in progress is to identify supports that are impactful on nonbinary engineering students’ persistence in their majors and careers. Specifically, we ask the questions: 1) What supports do alters, of differing genders, provide that witness and mirror nonbinary engineers; 2) to what extent are supports provided by alters impactful on students' success in their majors and careers? Methodology/approach: We purposely selected 4 nonbinary engineering students to interview from a larger study on engineering students’ social capital and professional skills. Nonbinary engineering students were asked about their experiences of being nonbinary in engineering spaces, the support they received from cisgender and transgender alters and the impact of that support on their persistence in engineering. We analyzed the themes in the interviews by coding the alters mentioned, the types of supports provided (mirroring, witnessing, expressive and instrumental) and the impact of supports on persistence. Future Work and Implications: The finding of this study will contribute to a greater understanding of the social capital and social networks of nonbinary students, as there is dearth of research on their experiences in engineering spaces. This work can guide our understanding of the supports that nonbinary engineering students have available to them and perhaps more importantly, the supports they do not have available to them. The findings from this study will be used to inform organizational and institutional policies to support engineering students in accessing more social capital. 
    more » « less
  4. ; ; ; ; (, ASEE Peer)
    Background Nonbinary engineering students (i.e., engineering students that identify as outside of the gender binary) must navigate a cis- and heteronormative society, in addition to a male dominated engineering culture. Nonbinary students in higher education report high levels of minority threat, lower levels of persistence than cisgender peers and lessened feelings of belonging. One avenue for supporting nonbinary students’ persistence in engineering is to increase understanding of the types of individuals that support nonbinary students and the supports available to nonbinary engineering students. For this study, we are utilizing social capital theory and Devor’s witnessing and mirroring framework to explore the supports nonbinary engineering students access through their social networks and how those supports impact persistence. Social capital, the resources embedded into social relationships, has been used as both an indicator and outcome in relation to students’ well-being, belonging, academic success, and persistence. Devor’s witnessing and mirroring framework brings greater meaning to the value of transgender and cisgender alters in a nonbinary engineers’ network. Specifically, how cisgender alters witness a nonbinary person’s identity as an outsider to gender nonconformity and how transgender alters mirror a nonbinary person’s experience as an insider to gender nonconformity. Purpose The purpose of this work in progress is to identify supports that are impactful on nonbinary engineering students’ persistence in their majors and careers. Specifically, we ask the questions: 1) What supports do alters, of differing genders, provide that witness and mirror nonbinary engineers; 2) to what extent are supports provided by alters impactful on students' success in their majors and careers? Methodology/approach We purposely selected 4 nonbinary engineering students to interview from a larger study on engineering students’ social capital and professional skills. Nonbinary engineering students were asked about their experiences of being nonbinary in engineering spaces, the support they received from cisgender and transgender alters and the impact of that support on their persistence in engineering. We analyzed the themes in the interviews by coding the alters mentioned, the types of supports provided (mirroring, witnessing, expressive and instrumental) and the impact of supports on persistence. Future Work & Implications The finding of this study will contribute to a greater understanding of the social capital and social networks of nonbinary students, as there is dearth of research on their experiences in engineering spaces. This work can guide our understanding of the supports that nonbinary engineering students have available to them and perhaps more importantly, the supports they do not have available to them. The findings from this study will be used to inform organizational and institutional policies to support engineering students in accessing more social capital. 
    more » « less
  5. ; ; ; ; (, 2021 Physics Education Research Conference Proceedings)
    Education researchers often compare performance across race and gender on research-based assessments of physics knowledge to investigate the impacts of racism and sexism on physics student learning. These investigations' claims rely on research-based assessments providing reliable, unbiased measures of student knowledge across social identity groups. We used classical test theory and differential item functioning (DIF) analysis to examine whether the items on the Force Concept Inventory (FCI) provided unbiased data across social identifiers for race, gender, and their intersections. The data was accessed through the Learning About STEM Student Outcomes platform and included responses from 4,848 students posttests in 152 calculus-based introductory physics courses from 16 institutions. The results indicated that the majority of items (22) on the FCI were biased towards a group. These results point to the need for instrument validation to account for item bias and the identification or development of fair research-based assessments. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email