Title: Causal Conceptions of Fairness and their Consequences
Recent work highlights the role of causality in designing equitable decision-making algorithms. It is not immediately clear, however, how existing causal conceptions of fairness relate to one another, or what the consequences are of using these definitions as design principles. Here, we first assemble and categorize popular causal definitions of algorithmic fairness into two broad families: (1) those that constrain the effects of decisions on counterfactual disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions almost always—in a measure theoretic sense—result in strongly Pareto dominated decision policies, meaning there is an alternative, unconstrained policy favored by every stakeholder with preferences drawn from a large, natural class. For example, in the case of college admissions decisions, policies constrained to satisfy causal fairness definitions would be disfavored by every stakeholder with neutral or positive preferences for both academic preparedness and diversity. Indeed, under a prominent definition of causal fairness, we prove the resulting policies require admitting all students with the same probability, regardless of academic qualifications or group membership. Our results highlight formal limitations and potential adverse consequences of common mathematical notions of causal fairness. more »« less
Corbett-Davies, Sam; Gaebler, Johann D.; Nilforoshan, Hamed; Shroff, Ravi; Goel, Sharad(
, Journal of machine learning research)
Weinberger, Kilian
(Ed.)
The field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last decade, several formal, mathematical definitions of fairness have gained prominence. Here we first assemble and categorize these definitions into two broad families: (1) those that constrain the effects of decisions on disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions typically result in strongly Pareto dominated decision policies. For example, in the case of college admissions, adhering to popular formal conceptions of fairness would simultaneously result in lower student-body diversity and a less academically prepared class, relative to what one could achieve by explicitly tailoring admissions policies to achieve desired outcomes. In this sense, requiring that these fairness definitions hold can, perversely, harm the very
groups they were designed to protect. In contrast to axiomatic notions of fairness, we argue that the equitable design of algorithms requires grappling with their context-specific consequences, akin to the equitable design of policy. We conclude by listing several open challenges in fair machine learning and offering strategies to ensure algorithms are better aligned with policy goals.
Studies have shown that the graduation rate for underrepresented minorities (URM)
students enrolled in engineering doctorates is significantly lower than their peers. In response,
we created the “Rising Doctoral Institute (RDI)”. This project aims to address issues that URM
students encounter when transitioning into a Ph.D. in engineering and their decision to persist in
the program. To suggest institutional policies that increase the likelihood of URM students to
persist in their doctorate, we identify and analyze some factors in the academic system that
reinforce or hinder the retention of URM students in doctoral education.
Although the factors that influence persistence in URM students have been largely studied as
direct causes of attrition or retention, there is a need for a system perspective that takes into
account the complexity and dynamic interaction that exists between those factors. The academic
system is a complex system that, by nature, is policy resistant. This means that a positive
variation of a factor can incur unintended consequences that could lead to a negative variation in
other factors and ultimately hinder the positive outcomes of that policy.
In this work-in-progress article, we analyze the dynamics of the factors in the academic system
that reinforce or hinder the retention of URM graduate students in engineering. The purpose is to
build some of the causal loops that involve those factors, to improve the understanding of how
the complex system works, and prevent unintended consequences of institutional policies. We
used Causal Loop Diagrams (CLD) to model the feedback loops of the system based on initial
hypotheses of causal relationships between the factors.
We followed a process that started with establishing hypotheses from a previous literature
review, then using a different set of articles we identified the factors related to the hypotheses
and the causal links between them. Next, we did axial coding to group the concepts into smaller
categories and established the causal relations between categories. With these categories and
relations, we created the CLDs for each hypothesis. For the CLDs that have connections missing
to close the loop, we went to find additional literature to close them. Finally, we analyzed the
implications of each CLD.
In this article, we analyze and describe three major CLDs found in literature. The first one was
built around the factor of having a positive relationship with the supervisor. The second centered
on the student’s experience. The third focused on factors that relate to university initiatives
Wu, Yongkai; Zhang, Lu; Wu, Xintao(
, Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, {KDD} 2018)
Predictive models learned from historical data are widely used to help companies and organizations make decisions. However, they may digitally unfairly treat unwanted groups, raising concerns about fairness and discrimination. In this paper, we study the fairness-aware ranking problem which aims to discover discrimination in ranked datasets and reconstruct the fair ranking. Existing methods in fairness-aware ranking are mainly based on statistical parity that cannot measure the true discriminatory effect since discrimination is causal. On the other hand, existing methods in causal-based anti-discrimination learning focus on classification problems and cannot be directly applied to handle the ranked data. To address these limitations, we propose to map the rank position to a continuous score variable that represents the qualification of the candidates. Then, we build a causal graph that consists of both the discrete profile attributes and the continuous score. The path-specific effect technique is extended to the mixed-variable causal graph to identify both direct and indirect discrimination. The relationship between the path-specific effects for the ranked data and those for the binary decision is theoretically analyzed. Finally, algorithms for discovering and removing discrimination from a ranked dataset are developed. Experiments using the real-world dataset show the effectiveness of our approaches.
Zhang, Junzhe; Bareinboim, Elias(
, Proceedings of the ... AAAI Conference on Artificial Intelligence)
AI plays an increasingly prominent role in society since decisions that were once made by humans are now delegated to automated systems. These systems are currently in charge of deciding bank loans, criminals’ incarceration, and the hiring of new employees, and it’s not difficult to envision that they will in the future underpin most of the decisions in society. Despite the high complexity entailed by this task, there is still not much understanding of basic properties of such systems. For instance, we currently cannot detect (neither explain nor correct) whether an AI system is operating fairly (i.e., is abiding by the decision-constraints agreed by society) or it is reinforcing biases and perpetuating a preceding prejudicial practice. Issues of discrimination have been discussed extensively in legal circles, but there exists still not much understanding of the formal conditions that an automated system must adhere to be deemed fair. In this paper, we use the language of structural causality (Pearl, 2000) to fill in this gap. We start by introducing three new fine-grained measures of transmission of change from stimulus to effect called counterfactual direct (Ctf-DE), indirect (Ctf-IE), and spurious (Ctf-SE) effects. Building on these measures, we derive the causal explanation formula, which allows the AI designer to quantitatively evaluate fairness and explain the total observed disparity of decisions through different discriminatory mechanisms. We apply these results to various discrimination analysis tasks and run extensive simulations, including detection, evaluation, and optimization of decision-making under fairness constraints. We conclude studying the trade-off between different types of fairness criteria (outcome and procedural), and provide a quantitative approach to policy implementation and the design of fair decision-making systems.
BACKGROUND Charles Darwin’s Descent of Man, and Selection in Relation to Sex tackled the two main controversies arising from the Origin of Species: the evolution of humans from animal ancestors and the evolution of sexual ornaments. Most of the book focuses on the latter, Darwin’s theory of sexual selection. Research since supports his conjecture that songs, perfumes, and intricate dances evolve because they help secure mating partners. Evidence is overwhelming for a primary role of both male and female mate choice in sexual selection—not only through premating courtship but also through intimate interactions during and long after mating. But what makes one prospective mate more enticing than another? Darwin, shaped by misogyny and sexual prudery, invoked a “taste for the beautiful” without speculating on the origin of the “taste.” How to explain when the “final marriage ceremony” is between two rams? What of oral sex in bats, cloacal rubbing in bonobos, or the sexual spectrum in humans, all observable in Darwin’s time? By explaining desire through the lens of those male traits that caught his eyes and those of his gender and culture, Darwin elided these data in his theory of sexual evolution. Work since Darwin has focused on how traits and preferences coevolve. Preferences can evolve even if attractive signals only predict offspring attractiveness, but most attention has gone to the intuitive but tenuous premise that mating with gorgeous partners yields vigorous offspring. By focusing on those aspects of mating preferences that coevolve with male traits, many of Darwin’s influential followers have followed the same narrow path. The sexual selection debate in the 1980s was framed as “good genes versus runaway”: Do preferences coevolve with traits because traits predict genetic benefits, or simply because they are beautiful? To the broader world this is still the conversation. ADVANCES Even as they evolve toward ever-more-beautiful signals and healthier offspring, mate-choice mechanisms and courter traits are locked in an arms race of coercion and resistance, persuasion and skepticism. Traits favored by sexual selection often do so at the expense of chooser fitness, creating sexual conflict. Choosers then evolve preferences in response to the costs imposed by courters. Often, though, the current traits of courters tell us little about how preferences arise. Sensory systems are often tuned to nonsexual cues like food, favoring mating signals resembling those cues. And preferences can emerge simply from selection on choosing conspecifics. Sexual selection can therefore arise from chooser biases that have nothing to do with ornaments. Choice may occur before mating, as Darwin emphasized, but individuals mate multiple times and bias fertilization and offspring care toward favored partners. Mate choice can thus occur in myriad ways after mating, through behavioral, morphological, and physiological mechanisms. Like other biological traits, mating preferences vary among individuals and species along multiple dimensions. Some of this is likely adaptive, as different individuals will have different optimal mates. Indeed, mate choice may be more about choosing compatible partners than picking the “best” mate in the absolute sense. Compatibility-based choice can drive or reinforce genetic divergence and lead to speciation. The mechanisms underlying the “taste for the beautiful” determine whether mate choice accelerates or inhibits reproductive isolation. If preferences are learned from parents, or covary with ecological differences like the sensory environment, then choice can promote genetic divergence. If everyone shares preferences for attractive ornaments, then choice promotes gene flow between lineages. OUTLOOK Two major trends continue to shift the emphasis away from male “beauty” and toward how and why individuals make sexual choices. The first integrates neuroscience, genomics, and physiology. We need not limit ourselves to the feathers and dances that dazzled Darwin, which gives us a vastly richer picture of mate choice. The second is that despite persistent structural inequities in academia, a broader range of people study a broader range of questions. This new focus confirms Darwin’s insight that mate choice makes a primary contribution to sexual selection, but suggests that sexual selection is often tangential to mate choice. This conclusion challenges a persistent belief with sinister roots, whereby mate choice is all about male ornaments. Under this view, females evolve to prefer handsome males who provide healthy offspring, or alternatively, to express flighty whims for arbitrary traits. But mate-choice mechanisms also evolve for a host of other reasons Understanding mate choice mechanisms is key to understanding how sexual decisions underlie speciation and adaptation to environmental change. New theory and technology allow us to explicitly connect decision-making mechanisms with their evolutionary consequences. A century and a half after Darwin, we can shift our focus to females and males as choosers, rather than the gaudy by-products of mate choice. Mate choice mechanisms across domains of life. Sensory periphery for stimulus detection (yellow), brain for perceptual integration and evaluation (orange), and reproductive structures for postmating choice among pollen or sperm (teal). ILLUSTRATION: KELLIE HOLOSKI/ SCIENCE
Nilforoshan, Hamed, Gaebler, Johann D., Shroff, Ravi, and Goel, Sharad. Causal Conceptions of Fairness and their Consequences. Retrieved from https://par.nsf.gov/biblio/10392252. Proceedings of the 39th International Conference on Machine Learning 162.
Nilforoshan, Hamed, Gaebler, Johann D., Shroff, Ravi, & Goel, Sharad. Causal Conceptions of Fairness and their Consequences. Proceedings of the 39th International Conference on Machine Learning, 162 (). Retrieved from https://par.nsf.gov/biblio/10392252.
Nilforoshan, Hamed, Gaebler, Johann D., Shroff, Ravi, and Goel, Sharad.
"Causal Conceptions of Fairness and their Consequences". Proceedings of the 39th International Conference on Machine Learning 162 (). Country unknown/Code not available. https://par.nsf.gov/biblio/10392252.
@article{osti_10392252,
place = {Country unknown/Code not available},
title = {Causal Conceptions of Fairness and their Consequences},
url = {https://par.nsf.gov/biblio/10392252},
abstractNote = {Recent work highlights the role of causality in designing equitable decision-making algorithms. It is not immediately clear, however, how existing causal conceptions of fairness relate to one another, or what the consequences are of using these definitions as design principles. Here, we first assemble and categorize popular causal definitions of algorithmic fairness into two broad families: (1) those that constrain the effects of decisions on counterfactual disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions almost always—in a measure theoretic sense—result in strongly Pareto dominated decision policies, meaning there is an alternative, unconstrained policy favored by every stakeholder with preferences drawn from a large, natural class. For example, in the case of college admissions decisions, policies constrained to satisfy causal fairness definitions would be disfavored by every stakeholder with neutral or positive preferences for both academic preparedness and diversity. Indeed, under a prominent definition of causal fairness, we prove the resulting policies require admitting all students with the same probability, regardless of academic qualifications or group membership. Our results highlight formal limitations and potential adverse consequences of common mathematical notions of causal fairness.},
journal = {Proceedings of the 39th International Conference on Machine Learning},
volume = {162},
author = {Nilforoshan, Hamed and Gaebler, Johann D. and Shroff, Ravi and Goel, Sharad},
editor = {Chaudhuri, Kamalika and Jegelka, Stefanie and Song, Le and Szepesvari, Csaba and Niu, Gang and Sabato, Sivan}
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
