More Like this

1. Fairness Testing: Testing Software for Discrimination

   Galhotra, Sainyam; Brun, Yuriy; Meliou, Alexandra (September 2017, Proceedings of 2017 11th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering)

   This paper defines the notions of software fairness and discrimination and develops a testing-based method for measuring if and how much software discriminates. Specifically, the paper focuses on measuring causality in discriminatory behavior. Modern software contributes to important societal decisions and evidence of software discrimination has been found in systems that recommend criminal sentences, grant access to financial loans and products, and determine who is allowed to participate in promotions and receive services. Our approach, Themis, measures discrimination in software by generating efficient, discrimination-testing test suites. Given a schema describing valid system inputs, Themis generates discrimination tests automatically and, notably, does not require an oracle. We evaluate Themis on 20 software systems, 12 of which come from prior work with explicit focus on avoiding discrimination. We find that (1) Themis is effective at discovering software discrimination, (2) state-of-the-art techniques for removing discrimination from algorithms fail in many situations, at times discriminating against as much as 98% of an input subdomain, (3) Themis optimizations are effective at producing efficient test suites for measuring discrimination, and (4) Themis is more efficient on systems that exhibit more discrimination. We thus demonstrate that fairness testing is a critical aspect of the software development cycle in domains with possible discrimination and provide initial tools for measuring software discrimination. 

   more » « less
2. Distribution-Specific Auditing for Subgroup Fairness

   https://doi.org/10.4230/LIPIcs.FORC.2024.5  

   Hsu, Daniel; Huang, Jizhou; Juba, Brendan (January 2024, Schloss Dagstuhl – Leibniz-Zentrum für Informatik)

   Rothblum, Guy N (Ed.)

   We study the problem of auditing classifiers for statistical subgroup fairness. Kearns et al. [Kearns et al., 2018] showed that the problem of auditing combinatorial subgroups fairness is as hard as agnostic learning. Essentially all work on remedying statistical measures of discrimination against subgroups assumes access to an oracle for this problem, despite the fact that no efficient algorithms are known for it. If we assume the data distribution is Gaussian, or even merely log-concave, then a recent line of work has discovered efficient agnostic learning algorithms for halfspaces. Unfortunately, the reduction of Kearns et al. was formulated in terms of weak, "distribution-free" learning, and thus did not establish a connection for families such as log-concave distributions. In this work, we give positive and negative results on auditing for Gaussian distributions: On the positive side, we present an alternative approach to leverage these advances in agnostic learning and thereby obtain the first polynomial-time approximation scheme (PTAS) for auditing nontrivial combinatorial subgroup fairness: we show how to audit statistical notions of fairness over homogeneous halfspace subgroups when the features are Gaussian. On the negative side, we find that under cryptographic assumptions, no polynomial-time algorithm can guarantee any nontrivial auditing, even under Gaussian feature distributions, for general halfspace subgroups. 

   more » « less
3. Fairness and Abstraction in Sociotechnical Systems

   https://doi.org/10.1145/3287560.3287598  

   Selbst, Andrew D.; boyd, danah; Friedler, Sorelle A.; Venkatasubramanian, Suresh; Vertesi, Janet (January 2019, Proceedings of the Conference on Fairness, Accountability, and Transparency)

   A key goal of the fair-ML community is to develop machine-learning based systems that, once introduced into a social context, can achieve social and legal outcomes such as fairness, justice, and due process. Bedrock concepts in computer science---such as abstraction and modular design---are used to define notions of fairness and discrimination, to produce fairness-aware learning algorithms, and to intervene at different stages of a decision-making pipeline to produce "fair" outcomes. In this paper, however, we contend that these concepts render technical interventions ineffective, inaccurate, and sometimes dangerously misguided when they enter the societal context that surrounds decision-making systems. We outline this mismatch with five "traps" that fair-ML work can fall into even as it attempts to be more context-aware in comparison to traditional data science. We draw on studies of sociotechnical systems in Science and Technology Studies to explain why such traps occur and how to avoid them. Finally, we suggest ways in which technical designers can mitigate the traps through a refocusing of design in terms of process rather than solutions, and by drawing abstraction boundaries to include social actors rather than purely technical ones. 

   more » « less
4. Hunting for Discriminatory Proxies in Linear Regression Models

   Yeom, Samuel; Datta, Anupam; Fredrikson, Matt (January 2018, Advances in neural information processing systems)

   A machine learning model may exhibit discrimination when used to make decisions involving people. One potential cause for such outcomes is that the model uses a statistical proxy for a protected demographic attribute. In this paper we formulate a definition of proxy use for the setting of linear regression and present algorithms for detecting proxies. Our definition follows recent work on proxies in classification models, and characterizes a model's constituent behavior that: 1) correlates closely with a protected random variable, and 2) is causally influential in the overall behavior of the model. We show that proxies in linear regression models can be efficiently identified by solving a second-order cone program, and further extend this result to account for situations where the use of a certain input variable is justified as a business necessity''. Finally, we present empirical results on two law enforcement datasets that exhibit varying degrees of racial disparity in prediction outcomes, demonstrating that proxies shed useful light on the causes of discriminatory behavior in models. 

   more » « less
5. Does Shared Gender Group Membership Mitigate the Effect of Implicit Bias Attributions on Accountability for Gender-Based Discrimination?

   https://doi.org/10.1177/0146167220965306  

   Daumeyer, Natalie_M; Onyeador, Ivuoma_N; Richeson, Jennifer_A (November 2020, Personality and Social Psychology Bulletin)

   Attributing gender discrimination to implicit bias has become increasingly common. However, research suggests that when discrimination is attributed to implicit rather than explicit bias, the perpetrators are held less accountable and deemed less worthy of punishment. The present work examines (a) whether this effect replicates in the domain of gender discrimination, and (b) whether sharing a group membership with the victim moderates the effect. Four studies revealed that both men and women hold perpetrators of gender discrimination less accountable if their behavior is attributed to implicit rather than explicit bias. Moreover, women held male (Studies 1–3), but not female (Study 4), perpetrators of gender discrimination more accountable than did men. Together, these findings suggest that while shared gender group membership may inform judgments of accountability for gender discrimination, it does not weaken the tendency to hold perpetrators less accountable for discrimination attributed to implicit, compared with explicit, bias. 

   more » « less