An official website of the United States government
This study aims to understand the effectiveness of typical admissions criteria in identifying students who will complete the Physics Ph.D. Multivariate statistical analysis of roughly one in eight physics Ph.D. students from 2000 to 2010 indicates that the traditional admissions metrics of undergraduate grade point average (GPA) and the Graduate Record Examination (GRE) Quantitative, Verbal, and Physics Subject Tests do not predict completion as effectively admissions committees presume. Significant associations with completion were found for undergraduate GPA in all models and for GRE Quantitative in two of four studied models; GRE Physics and GRE Verbal were not significant in any model. It is notable that completion changed by less than 10% for U.S. physics major test takers scoring in the 10th versus 90th percentile on the Quantitative test. Aside from these limitations in predicting Ph.D. completion overall, overreliance on GRE scores in admissions processes also selects against underrepresented groups.
more »
« less
Instructional Set Moderates the Effect of GRE on Faculty Appraisals of Applicant Competence: A Vignette Study With Implications for Holistic Review
While there is movement to create more equitable and holistic admission review processes, faculty continue to place strong emphasis on a single piece of information when making admissions decisions: standardized test scores. This study used an experimental design to test whether instructions provided to faculty prior to assessing doctoral applicants could support holistic review by reducing the weight of the general record examination (GRE) in faculty appraisals of competence and merit for graduate study. Tenured and/or tenure-track faculty ( N =271) were randomly assigned to one of three instructional conditions: Control (no instruction), “Diamond in the Rough,” and “Weed Out.” In addition, faculty participants were randomly assigned to read one of two vignettes of a prospective first-generation student who either received high or average GRE scores. Faculty then rated the applicant’s competence using a three-item survey. As expected, faculty who read the vignette describing the candidate with the high GRE rated him as more competent than faculty who read the average GRE vignette. In addition, being instructed to seek out diamonds in the rough buffered the effect of the GRE score on competence. Faculty were also asked to indicate whether they would need additional information to make an admissions decision. They were more likely to ask about grades and research skills than about psychosocial factors that might contextualize the candidate’s performance and perceived competence. The results of this study have implications for creating more equitable doctoral admissions processes that center equity, diversity, and inclusion in decision making.
more »
« less
- Award ID(s):
- 1742065
- PAR ID:
- 10388723
- Date Published:
- Journal Name:
- Frontiers in Psychology
- Volume:
- 12
- ISSN:
- 1664-1078
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
A growing number of college applications has presented an annual challenge for college admissions in the United States. Admission offices have historically relied on standardized test scores to organize large applicant pools into viable subsets for review. However, this approach may be subject to bias in test scores and selection bias in test-taking with recent trends toward test-optional admission. We explore a machine learning-based approach to replace the role of standardized tests in subset generation while taking into account a wide range of factors extracted from student applications to support a more holistic review. We evaluate the approach on data from an undergraduate admission office at a selective US institution (13,248 applications). We find that a prediction model trained on past admission data outperforms an SAT-based heuristic and matches the demographic composition of the last admitted class. We discuss the risks and opportunities for how such a learned model could be leveraged to support human decision-making in college admissions.more » « less
-
The use of the Physics GRE in graduate admissions has gained considerable attention in recent years. While studies have shown the problematic nature of the exam quantitatively, it is time that student experiences were also included in the discussion. In this qualitative study, we interviewed 69 current graduate physics and astronomy students about their process of deciding where to apply to graduate school. Physics GRE requirements played a substantial role in this decision, with 48 students mentioning the test as part of their process to narrow programs down to a personalized short-list. Participants discussed potential barriers (e.g., financial and travel) that affected some students, but not others, which created an unequal playing field for grad school applicants. Secondly, while the participants noticed a shift towards more departments having "optional'' GRE language, female student participants still felt the need to take and submit their Physics GRE scores, while male student participants truly saw "optional'' as optional. These results suggest that graduate programs requiring the Physics GRE are doing more than asking prospective graduate students to simply take a physics test, and that the use of 'optional' requirement language may be inadvertently disadvantaging the very students that they are trying to recruit.more » « less
-
Abstract This qualitative study examines the information collected about applicants to mathematics or science teacher preparation programs (MSTPPs) and how university faculty perceive and value this information in admissions decisions. Based on document review and interviews with MSTPP faculty and admissions directors, we found that broad measures of mathematics and science content background (e.g., achievement test scores, past mathematics and science courses taken) were used more frequently than information on applicants‐specific mathematics and science content knowledge and dispositions. In many cases, application components (such as interviews and personal essay statements) were perceived by faculty to be conducive to surfacing applicants‐content knowledge and dispositions; however, they were not constructed or employed in a way that afforded the obtainment of this information. We highlight salient examples of MSTPPs‐collection and use of information related to mathematics and science and discuss implications for TPP admissions processes.more » « less
-
Using analysis of variance on a sample consisting of 1,499 US students across 21 US PhD programs, we show that there is no significant difference in the time it takes US male and female physics PhD students to complete their degree programs. This result comes in spite of a statistically significant 18 percentile point gap in median GRE-P scores between genders. Additional analyses reveal that there is no statistical difference between US students reported as White, Black/Hispanic/Multiracial/Native American, and Asian. Expanding our sample to also include 1,143 Non-US students, we find a small but significant effect of citizenship status on time to PhD completion where the average time for Non-US students to complete a physics PhD is about two months less than their US student counterparts. These results show that in spite of known gaps in standardized admissions exams between genders, these differences are not reflected in subsequent graduate school performance. Our findings reinforce the need for graduate admissions committees to go beyond quantitative metrics and conduct a holistic assessment of an applicant's potential to perform research effectively and to earn a PhD.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3389/fpsyg.2021.749621
