An official website of the United States government
We study a game theoretic model of standardized testing for college admissions. Students are of two types; High and Low. There is a college that would like to admit the High type students. Students take a potentially costly standardized exam which provides a noisy signal of their type. The students come from two populations, which are identical in talent (i.e. the type distribution is the same), but differ in their access to resources: the higher resourced population can at their option take the exam multiple times, whereas the lower resourced population can only take the exam once. We study two models of score reporting, which capture existing policies used by colleges. The first policy (sometimes known as "super-scoring") allows students to report the max of the scores they achieve. The other policy requires that all scores be reported. We find in our model that requiring that all scores be reported results in superior outcomes in equilibrium, both from the perspective of the college (the admissions rule is more accurate), and from the perspective of equity across populations: a student's probability of admission is independent of their population, conditional on their type. In particular, the false positive rates and false negative rates are identical in this setting, across the highly and poorly resourced student populations. This is the case despite the fact that the more highly resourced students can -- at their option -- either report a more accurate signal of their type, or pool with the lower resourced population under this policy.
more »
« less
Downstream Effects of Affirmative Action
We study a two-stage model, in which students are 1) admitted to college on the basis of an entrance exam which is a noisy signal about their qualifications (type), and then 2) those students who were admitted to college can be hired by an employer as a function of their college grades, which are an independently drawn noisy signal of their type. Students are drawn from one of two populations, which might have different type distributions. We assume that the employer at the end of the pipeline is rational, in the sense that it computes a posterior distribution on student type conditional on all information that it has available (college admissions, grades, and group membership), and makes a decision based on posterior expectation. We then study what kinds of fairness goals can be achieved by the college by setting its admissions rule and grading policy. For example, the college might have the goal of guaranteeing equal opportunity across populations: that the probability of passing through the pipeline and being hired by the employer should be independent of group membership, conditioned on type. Alternately, the college might have the goal of incentivizing the employer to have a group blind hiring rule. We show that both goals can be achieved when the college does not report grades. On the other hand, we show that under reasonable conditions, these goals are impossible to achieve even in isolation when the college uses an (even minimally) informative grading policy
more »
« less
- Award ID(s):
- 1763307
- PAR ID:
- 10100410
- Date Published:
- Journal Name:
- ACM FAT* 2019
- Page Range / eLocation ID:
- 240 to 248
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The Greater Equity, Access, and Readiness for Engineering and Technology (GEARSET) Program, an NSF funded S-STEM program was developed GEARSET to address several institutional needs at the university. The original target population for the GEARSET program was identified as a subset of the students who applied to the College of Engineering and do not meet all the admissions requirements and are admitted to an Exploratory Studies major in the university’s University College. Historical data indicates that approximately 170 students per year with a high school GPA of 3.00 or higher are admitted to Exploratory Studies because they do not meet the College of Engineering admissions criteria. Of these, roughly 78 students remain at the University after one year. Of those 78, only about 45 students per year transition to college of Engineering majors by the end of their first year. These numbers do not accurately reflect the ability of these students, but rather are due in part to curricular bottlenecks, lack of institutional support, and lack of significant relevant exposure of students to material meant to engage their engineering future selves. This data motivated the creation of the GEARSET program. Specifically, the program was designed to 1. Increase recruitment, retention, student success, and transfer rates into engineering of students who are not admitted directly to engineering but who are instead admitted to the university’s University College. 2. Increase meaningfulness and engineering relevance of pre-engineering curriculum. 3. Increase diversity within the student population of various engineering departments in the College of Engineering. 4. Remove bottlenecks in curriculum and improve access to engineering and decrease length to degree. A key aspect of the program is a curated curriculum. All students in the GEARSET program are enrolled in multiple courses historically proven to promote better understanding of the key areas of Math, Chemistry and Physics needed to be successful engineers. All students have access to advisors within the COE to help them better understand the programs, curriculum and professional outcomes of each discipline of Engineering. Another key component of the program is that low income students in the GEARSET cohort who successfully transfer to a major within the COE after one year receive scholarship support. Here we describe the Program, the results to date, and the impact of the recent global pandemic and the subsequent transition to test optional admissions criteria on the definition of the GEARSET cohort, program implementation, and student participation.more » « less
-
Chaudhuri, Kamalika; Jegelka, Stefanie; Song, Le; Szepesvari, Csaba; Niu, Gang; Sabato, Sivan (Ed.)Recent work highlights the role of causality in designing equitable decision-making algorithms. It is not immediately clear, however, how existing causal conceptions of fairness relate to one another, or what the consequences are of using these definitions as design principles. Here, we first assemble and categorize popular causal definitions of algorithmic fairness into two broad families: (1) those that constrain the effects of decisions on counterfactual disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions almost always—in a measure theoretic sense—result in strongly Pareto dominated decision policies, meaning there is an alternative, unconstrained policy favored by every stakeholder with preferences drawn from a large, natural class. For example, in the case of college admissions decisions, policies constrained to satisfy causal fairness definitions would be disfavored by every stakeholder with neutral or positive preferences for both academic preparedness and diversity. Indeed, under a prominent definition of causal fairness, we prove the resulting policies require admitting all students with the same probability, regardless of academic qualifications or group membership. Our results highlight formal limitations and potential adverse consequences of common mathematical notions of causal fairness.more » « less
-
Specifications (specs) grading systems use a “checklist” approach to assessing students that asks them to demonstrate a high level of proficiency in course content, often coupled with multiple attempts at revision. Students also must demonstrate mastery in some specs to earn high letter grades. There have been several reports in lower division college chemistry courses that use specs grading systems (e.g., general and organic chemistries), but there remains a dearth of accounts of specs grading systems in upper division courses. In this manuscript, we report on the use of specs grading systems at a primarily undergraduate women’s college in four upper division chemistry courses: biochemistry, inorganic chemistry, thermodynamics, and quantum mechanics. The conceptual framework for designing specs tailored to upper division chemistry courses and their use to assess student understanding of course content are shared along with student outcomes and feedback. The upper division students generally had a positive view of the specs grading system with students viewing themselves as working hard on assessments that were tough but fair. Finally, instructor comments are presented in an effort to highlight the perceived benefits and challenges of specs grading to future adopters.more » « less
-
We present and evaluate a machine learning based system that automatically grades audios of students speaking a foreign language. The use of automated systems to aid the assessment of student performance holds great promise in augmenting the teacher’s ability to provide meaningful feedback and instruction to students. Teachers spend a significant amount of time grading student work and the use of these tools can save teachers a significant amount of time on their grading. This additional time could be used to give personalized attention to each student. Significant prior research has focused on the grading of closed-form problems, open-ended essays and textual content. However, little research has focused on audio content that is much more prevalent in the language-study education. In this paper, we explore the development of automated assessment tools for audio responses in a college-level Chinese language-learning course. We analyze several challenges faced while working with data of this type as well as the generation and extraction of features for the purpose of building machine learning models to aid in the assessment of student language learning.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3287560.3287578
