More Like this

1. Designing Fair Ranking Schemes

   https://doi.org/10.1145/3299869.3300079  

   Asudeh, Abolfazl; Jagadish, H. V.; Stoyanovich, Julia; Das, Gautam (July 2019, Proceedings of the ACM SIGMOD Intl Conference on Management of Data)

   Items from a database are often ranked based on a combination of criteria. The weight given to each criterion in the combination can greatly affect the fairness of the produced ranking, for example, preferring men over women. A user may have the flexibility to choose combinations that weigh these criteria differently, within limits. In this paper, we develop a system that helps users choose criterion weights that lead to greater fairness. We consider ranking functions that compute the score of each item as a weighted sum of (numeric) attribute values, and then sort items on their score. Each ranking function can be expressed as a point in a multidimensional space. For a broad range of fairness criteria, including proportionality, we show how to efficiently identify regions in this space that satisfy these criteria. Using this identification method, our system is able to tell users whether their proposed ranking function satisfies the desired fairness criteria and, if it does not, to suggest the smallest modification that does. Our extensive experiments on real datasets demonstrate that our methods are able to find solutions that satisfy fairness criteria effectively (usually with only small changes to proposed weight vectors) and efficiently (in interactive time, after some initial pre-processing). 

   more » « less
2. Causal Intersectionality and Fair Ranking

   Yang, Ke; Loftus, Joshua R.; Stoyanovich, Julia (January 2021, 2nd Symposium on Foundations of Responsible Computing (FORC))

   null (Ed.)

   In this paper we propose a causal modeling approach to intersectional fairness, and a flexible, task-specific method for computing intersectionally fair rankings. Rankings are used in many contexts, ranging from Web search to college admissions, but causal inference for fair rankings has received limited attention. Additionally, the growing literature on causal fairness has directed little attention to intersectionality. By bringing these issues together in a formal causal framework we make the application of intersectionality in algorithmic fairness explicit, connected to important real world effects and domain knowledge, and transparent about technical limitations. We experimentally evaluate our approach on real and synthetic datasets, exploring its behavior under different structural assumptions. 

   more » « less
3. Explainable Disparity Compensation for Efficient Fair Ranking

   Gale, Abraham; Marian, Amelie (May 2024, 40th International Conference on Data Engineering, ICDE 2024)
4. FairFuse: Interactive Visual Support for Fair Consensus Ranking

   https://doi.org/10.1109/VIS54862.2022.00022  

   Shrestha, Hilson; Cachel, Kathleen; Alkhathlan, Mallak; Rundensteiner, Elke; Harrison, Lane (October 2022, 2022 IEEE Visualization and Visual Analytics (VIS))

   Fair consensus building combines the preferences of multiple rankers into a single consensus ranking, while ensuring any group defined by a protected attribute (such as race or gender) is not disadvantaged compared to other groups. Manually generating a fair consensus ranking is time-consuming and impractical- even for a fairly small number of candidates. While algorithmic approaches for auditing and generating fair consensus rankings have been developed, these have not been operationalized in interactive systems. To bridge this gap, we introduce FairFuse, a visualization system for generating, analyzing, and auditing fair consensus rankings. We construct a data model which includes base rankings entered by rankers, augmented with measures of group fairness, and algorithms for generating consensus rankings with varying degrees of fairness. We design novel visualizations that encode these measures in a parallel-coordinates style rank visualization, with interactions for generating and exploring fair consensus rankings. We describe use cases in which FairFuse supports a decision-maker in ranking scenarios in which fairness is important, and discuss emerging challenges for future efforts supporting fairness-oriented rank analysis. Code and demo videos available at https://osf.io/hd639/. 

   more » « less
5. Estimation of Fair Ranking Metrics with Incomplete Judgments

   https://doi.org/10.1145/3442381.3450080  

   Kırnap, Ömer; Diaz, Fernando; Biega, Asia; Ekstrand, Michael; Carterette, Ben; Yilmaz, Emine (April 2021, Proceedings of the Web Conference 2021)

   null (Ed.)

   There is increasing attention to evaluating the fairness of search system ranking decisions. These metrics often consider the membership of items to particular groups, often identified using protected attributes such as gender or ethnicity. To date, these metrics typically assume the availability and completeness of protected attribute labels of items. However, the protected attributes of individuals are rarely present, limiting the application of fair ranking metrics in large scale systems. In order to address this problem, we propose a sampling strategy and estimation technique for four fair ranking metrics. We formulate a robust and unbiased estimator which can operate even with very limited number of labeled items. We evaluate our approach using both simulated and real world data. Our experimental results demonstrate that our method can estimate this family of fair ranking metrics and provides a robust, reliable alternative to exhaustive or random data annotation. 

   more » « less