Many colleges and universities now require faculty search committees to use rubrics when evaluating faculty job candidates, as proponents believe these “decision-support tools” can reduce the impact of bias in candidate evaluation. That is, rubrics are intended to ensure that candidates are evaluated more fairly, which is then thought to contribute to the enhanced hiring of candidates from minoritized groups. However, there is scant — and even contradictory — evidence to support this claim. This study used a multiple case study methodology to explore how five faculty search committees used rubrics in candidate evaluation, and the extent to which using a rubric seemed to perpetuate or mitigate bias in committee decision-making. Results showed that the use of rubrics can improve searches by clarifying criteria, encouraging criteria use in evaluation, calibrating the application of criteria to evidence, and in some cases, bringing diversity, equity, and inclusion work (DEI) into consideration. However, search committees also created and implemented rubrics in ways that seem to perpetuate bias, undermine effectiveness, and potentially contribute to the hiring of fewer minoritized candidates. We conclude by providing stakeholders with practical recommendations on using rubrics and actualizing DEI in faculty hiring.
more »
« less
Can rubrics combat gender bias in faculty hiring?
Research has documented the presence of bias against women in hiring, including in academic science, technology, engineering, and mathematics (STEM). Hiring rubrics (also called criterion checklists, decision support tools, and evaluation tools) are widely recommended as a precise, cost-effective remedy to counteract hiring bias, despite a paucity of evidence that they actually work (see table S8). Our in-depth case study of rubric usage in faculty hiring in an academic engineering department in a very research-active university found that the rate of hiring women increased after the department deployed rubrics and used them to guide holistic discussions. Yet we also found evidence of substantial gender bias persisting in some rubric scoring categories and evaluators’ written comments. We do not recommend abandoning rubrics. Instead, we recommend a strategic and sociologically astute use of rubrics as a department self-study tool within the context of a holistic evaluation of semifinalist candidates.
more »
« less
- Award ID(s):
- 1661306
- PAR ID:
- 10440698
- Date Published:
- Journal Name:
- Science
- Volume:
- 377
- Issue:
- 6601
- ISSN:
- 0036-8075
- Page Range / eLocation ID:
- 35 to 37
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Constructed responses can be used to assess the complexity of student thinking and can be evaluated using rubrics. The two most typical rubric types used are holistic and analytic. Holistic rubrics may be difficult to use with expert-level reasoning that has additive or overlapping language. In an attempt to unpack complexity in holistic rubrics at a large scale, we have developed a systematic approach called deconstruction. We define deconstruction as the process of converting a holistic rubric into defining individual conceptual components that can be used for analytic rubric development and application. These individual components can then be recombined into the holistic score which keeps true to the holistic rubric purpose, while maximizing the benefits and minimizing the shortcomings of each rubric type. This paper outlines the deconstruction process and presents a case study that shows defined concept definitions for a hierarchical holistic rubric developed for an undergraduate physiology-content reasoning context. These methods can be used as one way for assessment developers to unpack complex student reasoning, which may ultimately improve reliability and validation of assessments that are targeted at uncovering large-scale complex scientific reasoning.more » « less
-
Constructed responses can be used to assess the complexity of student thinking and can be evaluated using rubrics. The two most typical rubric types used are holistic and analytic. Holistic rubrics may be difficult to use with expert-level reasoning that has additive or overlapping language. In an attempt to unpack complexity in holistic rubrics at a large scale, we have developed a systematic approach called deconstruction. We define deconstruction as the process of converting a holistic rubric into defining individual conceptual components that can be used for analytic rubric development and application. These individual components can then be recombined into the holistic score which keeps true to the holistic rubric purpose, while maximizing the benefits and minimizing the shortcomings of each rubric type. This paper outlines the deconstruction process and presents a case study that shows defined concept definitions for a hierarchical holistic rubric developed for an undergraduate physiology-content reasoning context. These methods can be used as one way for assessment developers to unpack complex student reasoning, which may ultimately improve reliability and validation of assessments that are targeted at uncovering large-scale complex scientific reasoning.more » « less
-
more » « less
Professional cultures—distinct systems of meanings, rituals, and hierarchies—can reinforce social closure processes and can be a potent source of social inequality. This paper examines gender inequality in the faculty hiring process in academic engineering. This is a theoretically useful case in which a strong professional culture with meritocratic ideals exists alongside the underrepresentation of women. Previous studies of faculty hiring often find that women are devalued, yet few studies examine this process in real job searches nested within professional communities. We analyze the introductions of 175 finalist candidates by department faculty hosts, just before the candidates begin their high stakes job talks on their original research. We find that, compared to statements introducing men candidates, those introducing women are less likely to acknowledge and proclaim their research excellence, and they are more likely to contain irrelevant and inappropriate content. We argue that the professional cultural schema of scientific excellence is culturally masculine and can either highlight or obscure the introducers’ framing of candidates’ research competence, potentially anchoring audience expectations. The job talk ritual also helps socialize early career engineers into professional norms and understandings. We develop theoretical and policy implications for the study of gender barriers to employment in and beyond academic STEM.
-
more » « less
Abstract Background The lack of racial diversity in science, technology, engineering, and mathematics (STEM) disciplines is perhaps one of the most challenging issues in the United States higher education system. The issue is not only concerning diverse students, but also diverse faculty members. One important contributing factor is the faculty hiring process. To make progress toward equity in hiring decisions, it is necessary to better understand how applicants are considered and evaluated. In this paper, we describe and present our study based on a survey of current STEM faculty members and administrators who examined applicant qualifications and characteristics in STEM faculty hiring decisions.
Results There are three key findings of the present research. First, we found that faculty members placed different levels of importance on characteristics and qualifications for tenure track hiring and non-tenure track hiring. For example, items related to research were more important when evaluating tenure track applicants, whereas items related to teaching and diversity were more important when evaluating non-tenure track applicants. Second, faculty members’ institutional classification, position, and personal identities (e.g., gender, race/ethnicity) had an impact on their evaluation criteria. For instance, we found men considered some diversity-related items more important than women. Third, faculty members rated the importance of qualifications with diversity, equity, and inclusion (DEI)-related constructs significantly lower than qualifications that did not specify DEI-related constructs, and this trend held for both tenure track and non-tenure track faculty hiring.
Conclusions This study was an attempt to address the issue of diversity in STEM faculty hiring at institutions of higher education by examining how applicant characteristics are considered and evaluated in faculty hiring practices. Emphasizing research reputation and postdoctoral reputation while neglecting institutional diversity and equitable and inclusive teaching, research, and service stunt progress toward racial diversity because biases—both implicit and explicit, both positive and negative—still exist. Our results were consistent with research on bias in recruitment, revealing that affinity bias, confirmation bias, and halo bias exist in the faculty hiring process. These biases contribute to inequities in hiring, and need to be addressed before we can reach, sustain, and grow desired levels of diversity.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1126/science.abm2329
