We study a two-stage model, in which students are 1) admitted to
college on the basis of an entrance exam which is a noisy signal
about their qualifications (type), and then 2) those students who
were admitted to college can be hired by an employer as a function
of their college grades, which are an independently drawn noisy signal of their type. Students are drawn from one of two populations,
which might have different type distributions. We assume that the
employer at the end of the pipeline is rational, in the sense that
it computes a posterior distribution on student type conditional
on all information that it has available (college admissions, grades,
and group membership), and makes a decision based on posterior
expectation. We then study what kinds of fairness goals can be
achieved by the college by setting its admissions rule and grading
policy. For example, the college might have the goal of guaranteeing equal opportunity across populations: that the probability
of passing through the pipeline and being hired by the employer
should be independent of group membership, conditioned on type.
Alternately, the college might have the goal of incentivizing the
employer to have a group blind hiring rule. We show that both
goals can be achieved when the college does not report grades. On
the other hand, we show that under reasonable conditions, these
goals are impossible to achieve even in isolation when the college
uses an (even minimally) informative grading policy
more »
« less
Best vs. All: Equity and Accuracy of Standardized Test Score Reporting
We study a game theoretic model of standardized testing for college admissions. Students are of two types; High and Low. There is a college that would like to admit the High type students. Students take a potentially costly standardized exam which provides a noisy signal of their type.
The students come from two populations, which are identical in talent (i.e. the type distribution is the same), but differ in their access to resources: the higher resourced population can at their option take the exam multiple times, whereas the lower resourced population can only take the exam once. We study two models of score reporting, which capture existing policies used by colleges. The first policy (sometimes known as "super-scoring") allows students to report the max of the scores they achieve. The other policy requires that all scores be reported.
We find in our model that requiring that all scores be reported results in superior outcomes in equilibrium, both from the perspective of the college (the admissions rule is more accurate), and from the perspective of equity across populations: a student's probability of admission is independent of their population, conditional on their type. In particular, the false positive rates and false negative rates are identical in this setting, across the highly and poorly resourced student populations. This is the case despite the fact that the more highly resourced students can -- at their option -- either report a more accurate signal of their type, or pool with the lower resourced population under this policy.
more »
« less
- Award ID(s):
- 1763307
- NSF-PAR ID:
- 10333171
- Date Published:
- Journal Name:
- ACM Conference on Fairness, Accountability, and Transparancy (ACM FAccT)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Spatial visualization training has been shown to increase GPAs and graduation rates in science, technology and math. Furthermore, prior research has correlated sketching on paper to improvement on the standardized spatial visualization test PSVT:R. To take advantage of touchscreen technology, an App, in which students draw orthographic and isometric assignments, was developed for spatial visualization training. Students draw on the touchscreen and then submit their sketch to be graded automatically. If the sketch is incorrect, the students are provided with the option to try again or get customized guidance from the app. This allows students to work independently and get immediate feedback. In 2014, a trial using the App with college engineering students showed that it increased students’ performance on the PSVT:R. The 2014 trial also showed that student persistence, as measured by the number of times they tried a sketch again without asking for help, correlated to increases in the PSVT:R. Since 2014, the App was modified significantly. The assignments were rewritten to take advantage of the touchscreen interface, and persistence was encouraged using gamification and by providing varying levels of guidance. In 2017, two trials were conducted with college engineering students; an elective class (n=32) and a required class (n=137). Overall the persistence metric increased from 40% in 2014 to 77% in 2017. The overall gains on the PSVT:R increased from 7% to 9%. However, much larger gains occurred among students who entered the class with low PSVT:R scores (70% and below). These students are considered “at-risk” in terms of low graduation rate due to low spatial visualization ability. In 2014, 23% of these at-risk students improved to the point of moving out of the at-risk category. In 2017 this percentage increased to 82% and 67%. This paper describes the modifications to the App that led to the successful trials in 2017. In O=one of the 2017 trials , the app was implemented as homework, thereby not taking up classroom lecture time, which further eases the incorporation of spatial visualization training into a crowded curriculum.more » « less
-
Mechanics instructors frequently employ hands-on learning with goals such as demonstrating physical phenomena, aiding visualization, addressing misconceptions, exposing students to “real-world” problems, and promoting an engaging classroom environment. This paper presents results from a study exploring the importance of the “hands-on” aspect of a hands-on modeling curriculum we have been developing that spans several topics in statics. The curriculum integrates deep conceptual exploration with analysis procedure tutorials and aims to scaffold students’ development of representational competence, the ability to use multiple representations of a concept as appropriate for learning, problem solving, and communication. We conducted this study over two subsequent terms in an online statics course taught in the context of remote learning amidst the COVID-19 pandemic. The intervention section used a take-home adaptation of the original classroom curriculum. This adaptation consisted of eight activity worksheets with a supplied kit of manipulatives and model-building supplies students could use to construct and explore concrete representations of figures and diagrams used in the worksheets. In contrast, the control section used activity worksheets nearly identical to those used in the hands-on curriculum, but without the associated modeling parts kit. We only made minor revisions to the worksheets to remove reference to the models. The control and intervention sections were otherwise identical in how they were taught by the same instructor. We compare learning outcomes between the two sections as measured via pre-post administration of a test of 3D vector concepts and representations called the Test of Representational Competence with Vectors (TRCV). We also compare end of course scores on the Concept Assessment Test in Statics (CATS) and final exam scores. In addition, we analyze student responses on two “multiple choice plus explain” concept questions paired with each of five activities covering the topics of 3D moments, 3D particle equilibrium, rigid body equilibrium (2D and 3D), and frame analysis (2D). The mean pre/post gain across all ten questions was higher for the intervention section, with the largest differences observed on questions relating to 3D rigid body equilibrium. Students in the intervention section also made larger gains on the TRCV and scored better on the final exam compared to the control section, but these results are not statistically significant perhaps due to the small study population. There were no appreciable differences in end-of-course CATS scores. We also present student feedback on the activity worksheets that was slightly more positive for the versions with the models.more » « less
-
Mechanics instructors frequently employ hands-on learning with goals such as demonstrating physical phenomena, aiding visualization, addressing misconceptions, exposing students to “real-world” problems, and promoting an engaging classroom environment. This paper presents results from a study exploring the importance of the “hands-on” aspect of a hands-on modeling curriculum we have been developing that spans several topics in statics. The curriculum integrates deep conceptual exploration with analysis procedure tutorials and aims to scaffold students’ development of representational competence, the ability to use multiple representations of a concept as appropriate for learning, problem solving, and communication. We conducted this study over two subsequent terms in an online statics course taught in the context of remote learning amidst the COVID-19 pandemic. The intervention section used a take-home adaptation of the original classroom curriculum. This adaptation consisted of eight activity worksheets with a supplied kit of manipulatives and model-building supplies students could use to construct and explore concrete representations of figures and diagrams used in the worksheets. In contrast, the control section used activity worksheets nearly identical to those used in the hands-on curriculum, but without the associated modeling parts kit. We only made minor revisions to the worksheets to remove reference to the models. The control and intervention sections were otherwise identical in how they were taught by the same instructor. We compare learning outcomes between the two sections as measured via pre-post administration of a test of 3D vector concepts and representations called the Test of Representational Competence with Vectors (TRCV). We also compare end of course scores on the Concept Assessment Test in Statics (CATS) and final exam scores. In addition, we analyze student responses on two “multiple choice plus explain” concept questions paired with each of five activities covering the topics of 3D moments, 3D particle equilibrium, rigid body equilibrium (2D and 3D), and frame analysis (2D). The mean pre/post gain across all ten questions was higher for the intervention section, with the largest differences observed on questions relating to 3D rigid body equilibrium. Students in the intervention section also made larger gains on the TRCV and scored better on the final exam compared to the control section, but these results are not statistically significant perhaps due to the small study population. There were no appreciable differences in end-of-course CATS scores. We also present student feedback on the activity worksheets that was slightly more positive for the versions with the models.more » « less
-
This paper is a work-in-progress, focused on the utilization of the Rising Scholars Program to introduce minority students to experiential engineering projects within Agricultural and Biological Engineering. Traditional admissions processes at top institutions predominately utilize standardized test scores when comparing student applications. The equity of these high-stakes tests most severely affects students of low socioeconomic status (SES). The NSF-sponsored program, Rising Scholars: Web of Support used as an Indicator of Success in Engineering, was created to investigate whether alternative admission criteria could be used to identify low-SES applicants who would excel within STEM fields in higher education, even if they did not have the superior standardized testing metrics preferred by current admissions processes. The students underwent a pre-selection process to determine their eligibility. The overall experience was designed to enhance student connectivity within the collegiate environment. The Gallup-Purdue Index (2014) found that feeling supported and having learning experiences that illustrated learned principles produced a graduate who would be engaged in their work. The Rising Scholar (RS) program utilized a prescribed path through college designed to enhance these features. These positive experiences are exemplified by the Purdue Agricultural and Biological Engineering (ABE) department and how they approach the overall educational process. Faculty are motivated in their teaching, research, and extension efforts by a focus on meeting the world’s grand challenges, in which most college students are also highly interested. The Rising Scholars Program utilized the Vertically Integrated Projects model to introduce their students to real-life projects at the freshman and sophomore level, which could potentially be continued on into graduate school. Several of the RS students have worked with the Purdue ABE Hog Cooling Pad Project and these students have conducted research, prototyping, and design modifications on the pad. They have participated in five experimental bench tests of the design and four consecutive live animal studies related to the pad performance. Within these experiments, Rising Scholars students were able to work on real-life projects, with real-world impact. The preliminary hypothesis question is: Are future graduates of the Rising Scholars Program more likely to thrive in all areas of well-being due to their collegiate experiences?more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
