skip to main content

Explore Research Products in the NSF-PAR

Title: Individual Fairness in Pipelines
It is well understood that a system built from individually fair components may not itself be individually fair. In this work, we investigate individual fairness under pipeline composition. Pipelines differ from ordinary sequential or repeated composition in that individuals may drop out at any stage, and classification in subsequent stages may depend on the remaining “cohort” of individuals. As an example, a company might hire a team for a new project and at a later point promote the highest performer on the team. Unlike other repeated classification settings, where the degree of unfairness degrades gracefully over multiple fair steps, the degree of unfairness in pipelines can be arbitrary, even in a pipeline with just two stages. Guided by a panoply of real-world examples, we provide a rigorous framework for evaluating different types of fairness guarantees for pipelines. We show that naïve auditing is unable to uncover systematic unfairness and that, in order to ensure fairness, some form of dependence must exist between the design of algorithms at different stages in the pipeline. Finally, we provide constructions that permit flexibility at later stages, meaning that there is no need to lock in the entire pipeline at the time that the early stage is constructed.  more » « less
Award ID(s):
1763665
NSF-PAR ID:
10217368
Author(s) / Creator(s):
; ;
Editor(s):
Roth, A
Date Published:
Journal Name:
Leibniz international proceedings in informatics
ISSN:
1868-8969
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ( , ESEC/FSE’2021: The 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (Aug. 2021).)
    null (Ed.)
    In recent years, many incidents have been reported where machine learning models exhibited discrimination among people based on race, sex, age, etc. Research has been conducted to measure and mitigate unfairness in machine learning models. For a machine learning task, it is a common practice to build a pipeline that includes an ordered set of data preprocessing stages followed by a classifier. However, most of the research on fairness has considered a single classifier based prediction task. What are the fairness impacts of the preprocessing stages in machine learning pipeline? Furthermore, studies showed that often the root cause of unfairness is ingrained in the data itself, rather than the model. But no research has been conducted to measure the unfairness caused by a specific transformation made in the data preprocessing stage. In this paper, we introduced the causal method of fairness to reason about the fairness impact of data preprocessing stages in ML pipeline. We leveraged existing metrics to define the fairness measures of the stages. Then we conducted a detailed fairness evaluation of the preprocessing stages in 37 pipelines collected from three different sources. Our results show that certain data transformers are causing the model to exhibit unfairness. We identified a number of fairness patterns in several categories of data transformers. Finally, we showed how the local fairness of a preprocessing stage composes in the global fairness of the pipeline. We used the fairness composition to choose appropriate downstream transformer that mitigates unfairness in the machine learning pipeline. 
    more » « less
  2. ( , International Conference on Machine Learning (ICML))
    Although machine learning (ML) algorithms are widely used to make decisions about individuals in various domains, concerns have arisen that (1) these algorithms are vulnerable to strategic manipulation and "gaming the algorithm"; and (2) ML decisions may exhibit bias against certain social groups. Existing works have largely examined these as two separate issues, e.g., by focusing on building ML algorithms robust to strategic manipulation, or on training a fair ML algorithm. In this study, we set out to understand the impact they each have on the other, and examine how to characterize fair policies in the presence of strategic behavior. The strategic interaction between a decision maker and individuals (as decision takers) is modeled as a two-stage (Stackelberg) game; when designing an algorithm, the former anticipates the latter may manipulate their features in order to receive more favorable decisions. We analytically characterize the equilibrium strategies of both, and examine how the algorithms and their resulting fairness properties are affected when the decision maker is strategic (anticipates manipulation), as well as the impact of fairness interventions on equilibrium strategies. In particular, we identify conditions under which anticipation of strategic behavior may mitigate/exacerbate unfairness, and conditions under which fairness interventions can serve as (dis)incentives for strategic manipulation. 
    more » « less
  3. ; ; ; ( , ACM/IEEE International Conference on Software Engineering (ICSE), SEIS Track)
    The typical hiring pipeline for software engineering occurs over several stages—from phone screening and technical on-site interviews, to offer and negotiation. When these hiring pipelines are “leaky,” otherwise qualified candidates are lost at some stage of the pipeline. These leaky pipelines impact companies in several ways, including hindering a company’s ability to recruit competitive candidates and build diverse software teams. To understand where candidates become disengaged in the hiring pipeline—and what companies can do to prevent it—we conducted a qualitative study on over 10,000 reviews on 19 companies from Glassdoor, a website where candidates can leave reviews about their hiring process experiences. We identified several poor practices which prematurely sabotage the hiring process—for example, not adequately communicating hiring criteria, conducting interviews with inexperienced interviewers, and ghosting candidates. Our findings provide a set of guidelines to help companies improve their hiring pipeline practices—such as being deliberate about phrasing and language during initial contact with the candidate, providing candidates with constructive feedback after their interviews, and bringing salary transparency and long-term career discussions into offers and negotiations. Operationalizing these guidelines helps make the hiring pipeline more transparent, fair, and inclusive. 
    more » « less
  4. ; ; ; ; ; ( , 2019 ASEE Virtual Annual Conference Content Access)
    null (Ed.)
    his panel paper presents research on connecting theory to practice and the lessons learned in a change project, with a focus on team formation during the early stages of change making. An important yet often overlooked step in any change project is pulling together individuals to form a competent and efficient team. A functional change-making team requires a variety of complementary skill sets, which may come from different disciplinary backgrounds and/or different prior experiences. Kotter (1996) uses the term “guiding coalition” to refer to an effective change-making team. He identifies four key characteristics of guiding coalitions: position power, expertise, credibility, leadership. Kotter also goes on to examine the importance of trust and a common goal. In a review of the literature on guiding coalitions, Have, Have, Huijsmans, and Otto (2017) found that though the concept of a guiding coalition is widely advocated in the literature, only one study showed a moderate correlation between the existence of a guiding coalition and the success of a change process (Abraham, Griffin, & Crawford, 1999). Have et al. (2017) conclude that while the literature provides little evidence to the value of a guiding coalition, it does provide evidence that Kotter’s characteristics of a guiding coalition (position power, expertise, credibility, leadership skills, trust in leadership, and setting common goals) individually have positive effects on the outcomes of a change project. However, we don’t know how these characteristics interact. This analysis of team building and complementary skill sets emerges from our participatory action research with the NSF REvolutionizing engineering and computer science Departments (RED) teams to investigate the change process within STEM higher education. The research-to-practice cycle is integral to our project; data gathered through working with the RED teams provides insights that are then translated into applied, hands-on practices. We utilize an abductive analysis approach, a qualitative methodology that moves recursively between the data and theory-building to remain open to new or contradictory findings, keeping existing theory in mind while not developing formal hypotheses (Timmermans & Tavory, 2012). We find that many of the teams have learned lessons in the early stages of the change process around the guiding coalition characteristics, and our analysis builds on the literature by examining how these characteristics interact. For example, the expertise of the social scientists and education researchers help discern which change strategies have supporting evidence and fit the context, in addition to what is reasonable for planning, implementation, and evaluation. The results presented in this paper connect theory to practice, clarifying practices for building effective change-making teams within higher education. 
    more » « less
  5. ; ; ; ( , Neural information Processing Systems (NeurIPS))
    We consider the problem of online learning in the linear contextual bandits setting, but in which there are also strong individual fairness constraints governed by an unknown similarity metric. These constraints demand that we select similar actions or individuals with approximately equal probability, which may be at odds with optimizing reward, thus modeling settings where profit and social policy are in tension. We assume we learn about an unknown Mahalanobis similarity metric from only weak feedback that identifies fairness violations, but does not quantify their extent. This is intended to represent the interventions of a regulator who “knows unfairness when he sees it” but nevertheless cannot enunciate a quantitative fairness metric over individuals. Our main result is an algorithm in the adversarial context setting that has a number of fairness violations that depends only logarithmically on T, while obtaining an optimal O(√T) regret bound to the best fair policy. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email