The increasing amount of data and computational power have empowered machine learning algorithms to play crucial roles in automated decision-making for a variety of real-world applications, including credit scoring [1], criminal justice [2] and healthcare analysis [3]. As the application landscape of machine learning continues to broaden and deepen, so does the concern regarding the potential, often unintentional, bias it could introduce or amplify. For example, recent media coverage has revealed that a well-trained image generator could turn a low-resolution picture of a black man into a high-resolution image of a white man due to the skewed data distribution that causes the model to disfavor the minority group, 1 and another article highlighted an automated credit card application system assigning a dramatically higher credit 1 https://www.theverge.com/21298762/face-depixelizer-ai-machine-learning-tool-pulse-stylegan-obama-bias limit to a man than to his female partner, even though his partner has a better credit history. 2As such, algorithmic fairness, which aims to mitigate unintentional bias caused by automated learning algorithms, has become increasingly important. To date, researchers have proposed a variety of fairness notions [4], [5]. Among them, one of the most fundamental notions is group fairness. 3 Generally speaking, to ensure group fairness, the first step is to partition the entire population into a few demographic groups based on a pre-defined sensitive attribute (e.g., gender). Then the fair learning algorithm will enforce parity of a certain statistical measure among those demographic groups. Group fairness can be instantiated with many statistical notions of fairness. Statistical parity [6] enforces the learned classifier to accept equal proportion of population from the pre-defined majority group and minority group. Likewise, disparate impact [5] ensures the acceptance rate for the minority group should be no less than four-fifth of that for the group with the highest acceptance rate, which is analogous to the famous 'four-fifth' rule in the legal support area [7]. In addition, equalized odds and equal opportunity [8] are used to enforce the classification accuracies to be equal across all demographic groups conditioned on ground-truth outcomes or positively labeled populations, respectively. The vast majority of the existing works in group fairness primarily focus on debiasing with respect to a single sensitive attribute. However, it is quite common for multiple sensitive attributes (e.g., gender, race, marital status, etc.) to co-exist in a real-world application. We ask: would a debiasing algorithm designed to ensure the group fairness for a particular sensitive attribute (e.g., marital status) unintentionally amplify the group bias with respect to another sensitive attribute (e.g., gender)? If so, how can we ensure a fair learning outcome with respect to all sensitive attributes of concern simultaneously?
Existing works for answering these questions [5], [6], [9], [10] have two major limitations. The first limitation is that some existing works could only debias multiple distinct Rows indicate gender (e.g., male vs. female) and columns indicate race (e.g., orange vs. green). 4 Boxed individuals receive job offers. If we consider gender or race alone, statistical parity is enforced due to the equal acceptance rate. However, when considering gender and race (i.e., forming finer-grained gender-race groups), the classification result is biased in the fine-grained gender-race groups. This is because, the acceptance rates in two fine-grained groups (i.e., male-green group and female-orange) are lower than that of the two other finegrained groups (i.e., male-orange and female-green).
sensitive attributes [10], which fails to mitigate bias on the fine-grained groups formed by multiple sensitive attributes.
Figure 1 provides an illustrative example of the difference between fairness with respect to multiple distinct sensitive attributes and fairness among fine-grained groups of multiple sensitive attributes. The second limitation is that the optimization problems behind some existing works are often subject to surrogate constraints of statistical parity [5], [6], [9] instead of directly optimizing statistical parity itself, resulting in unstable performance on bias mitigation unless the learned models could perfectly model the relationship between the training data and the ground-truth outcome.
In this paper, we tackle these two limitations by studying the problem of information-theoretic intersectional fairness (INFOFAIR), which aims to directly enforce statistical parity on multiple sensitive attributes simultaneously. Though our focused fairness notion is statistical parity, the proposed method can be generalized to other statistical fairness notions (e.g., equalized odds and equal opportunity) with minor modifications. The key idea in solving the INFOFAIR problem is to consider all sensitive attributes of interest as a vectorized sensitive attribute in order to partition the demographic groups and then minimize the dependence between learning outcomes and this vectorized attribute. More specifically, we measure the dependence using mutual information originated in information theory [11]. Building upon it, we formulate the INFOFAIR problem as an optimization problem regularized on mutual information minimization.
The main contributions of this paper are as follows.
• Problem Definition. We formally define the problem of information-theoretic intersectional fairness and formulate it as an optimization problem, where the key idea is to minimize both the task-specific loss function (e.g., cross-entropy loss in classification) and mutual information between learning outcomes and the vectorized sensitive attribute.
• End-to-End Algorithmic Framework. We propose a novel end-to-end bias mitigation framework, named IN-FOFAIR, by optimizing a variational representation of mutual information. The proposed framework is extensible and capable of solving any learning task with a gradientbased optimizer. • Empirical Evaluations. We perform empirical evaluations in the fair classification task on three real-world datasets. The evaluation results demonstrate that our proposed framework can effectively mitigate bias with little sacrifice in the classification accuracy.
In this section, we present a table of the main symbols used in this paper. Then, we briefly review the concepts of statistical parity and mutual information, as well as their relationships. Finally, we formally define the problem of information-theoretic intersectional fairness. In this paper, matrices are denoted by bold uppercase letters (e.g., X), vectors are denoted by bold lowercase letters (e.g., y), scalars are denoted by italic lowercase letters (e.g., c) and sets are denoted by calligraphic letters (e.g., D). We use superscript T to denote transpose (e.g., h T is the transpose of h) and superscript C to denote the complement of a set (e.g., set D C is the complement of set D). We use a convention similar to NumPy for vector indexing (e.g., h[i] is the i-th element in vector h).
Statistical Parity is one of the most intuitive and widelyused group fairness notions. Given a set of data points X , their corresponding labels y and a sensitive attribute s, classification with statistical parity aims to learn a classifier to predict outcomes that (1) are as accurate as possible with respect to y and (2) do not favor one group over another with respect to s. Mathematically, statistical parity is defined as follows.
Definition 1: (Statistical Parity [6]). Suppose we have (1) a population X , (2) a hypothesis h : X → {0, 1} which assigns a binary label to individual x drawn from X and (3) a sensitive attribute which splits the population X into majority group M and minority group M C (i.e., X = M ∪ M C ). An individual x is accepted if h(x) = 1 and rejected if h(x) = 0. The hypothesis h : X → {0, 1} is said to have statistical parity on the population X as long as
where Pr[•] denotes the probability of an event happening.
Many methods have been proposed to achieve statistical parity. For example, Zemel et al. [12] learn fair representation by regularizing the difference in expected positive rate for majority and minority groups. Zhang et al. [13] propose an adversarial learning-based framework for fair classification, in which the output of the predictor is used to predict the sensitive attribute by the adversary. Kearns et al. [9] propose a learnerauditor framework to enforce subgroup fairness through fictitious play strategy. Mutual Information was first introduced in the 1940s [11]. Given two random variables, mutual information measures the dependence between them by quantifying the amount of information in bits obtained on one random variable through observing the other one. Let (x, y) be a pair of random variables x and y. Suppose their joint distribution is p x,y and the marginal distributions are p x and p y . The mutual information between x and y is defined as
x y p x,y log p x,y p x p y dxdy
(2) where H(x) =x p x log p x dx is the entropy of x and H(x|y) =x y p x,y log p x|y dxdy is the conditional entropy of x given y. Unlike correlation coefficients (e.g., Pearson's correlation coefficient) which could only capture the linear dependence between two random variables, mutual information is more general in capturing both the linear and nonlinear dependence between two random variables. We have I(x; y) = 0 if and only if two random variables x and y are independent to each other.
According to Lemma 1, there is an equivalence between statistical parity and zero mutual information.
Lemma 1: (Equivalence between statistical parity and zero mutual information [12], [14]). Statistical parity requires a sensitive attribute to be statistically independent to the learning results, which is equivalent to zero mutual information. Mathematically, given a learning outcome ỹ and the sensitive attribute s, we have
Proof: Omitted for brevity.
In order to generalize Lemma 1 from a single sensitive attribute to a set of sensitive attributes S = {s (1) , . . . , s (k) }, we first introduce the concept of vectorized sensitive attribute s given S. We define the vectorized sensitive attribute s = [s (1) , . . . , s (k) ] as a multi-dimensional random variable where each element of s represents the corresponding sensitive attribute in S (e.g., s[i] = s (i) is the i-th sensitive attribute). Based on that, we have the following equivalence. For notational simplicity, we denote I(ỹ; s (1) , . . . , s (k) ), p ỹ,s (1) ,...,s (k) and p s (1) ,...,s (k) with I(ỹ; s), p ỹ,s and p s , respectively.
p ỹ|s = p ỹ ⇔ p ỹ,s = p ỹp s ⇔ I(ỹ; s) = 0 (4) Based on Eq. ( 4), we formally define the problem of information-theoretic intersectional fairness as a mutual information minimization problem.
Input: (1) a set of k sensitive attributes S = {s (1) , . . . , s (k) }; (2) a set of n data points D = {(x i , s i , y i )|i = 1, . . . , n} where x i is the feature vector of the i-th data point, y i is its label and
describes the vectorized sensitive attributes on S of the i-th data point (with s (j) i being the corresponding attribute value of the j-th sensitive attribute s (j) ); and (3) a learning algorithm represented by l(x; s; y; ỹ; θ), where l is the loss function, ỹ * = argmin ỹl(x; s; y; ỹ; θ) is the optimal learning outcome on the input data with θ being model parameters.
Output: a set of revised learning outcomes {ỹ * } which minimizes (1) the empirical risk E (x,s,y)∼D [l(x; s; y; ỹ; θ)] and ( 2) the expectation of mutual information between the learning outcomes and the sensitive attributes E (x,s,y)∼D I(ỹ; s) .
Remark: a byproduct of INFOFAIR is that the statistical parity can also be achieved on any subset of sensitive attributes included in S, which is summarized in Lemma 2. This could be particularly useful in that the algorithm administrator does not need to re-train the model in order to obtain fair learning results if s/he is only interested in a subset of available sensitive attributes.
Lemma 2: Consider statistical parity as the fairness notion. Given a learning outcome ỹ, a set of k sensitive attributes S = {s (1) , . . . , s (k) } and the vectorized sensitive attribute s = [s (1) , . . . , s (k) ]. If ỹ is fair with respect to s, then ỹ is fair with respect to any vectorized sensitive attribute s sub induced from the subset of sensitive attributes S sub ⊆ S = {s (1) , . . . , s (k) }.
Proof: Omitted for brevity. III. PROPOSED METHOD In this section, we present a generic end-to-end algorithmic framework, named INFOFAIR, for information-theoretic intersectional fairness. We first formulate the problem as a mutual information minimization problem, and then present a variational representation of mutual information. Based on that, we present the INFOFAIR framework to solve the optimization problem, followed by discussions on generalizations and variants of our proposed framework.
Given a dataset D = {(x i , s i , y i )|i = 1, . . . , n}, the INFOFAIR problem (Problem 1) can be naturally formulated as minimizing the following objective function, J = E (x,s,y)∼D l(x; s; y; ỹ; θ) + αI(ỹ; s) (5) where l is a task-specific loss function for a learning task, θ is the model parameter, ỹ is the learning outcome and α > 0 is the regularization hyperparameter. An example of loss function l is the negative log likelihood l(x; s; y; ỹ; θ) = -log ỹ[y] (6) where y is the class label and ỹ denotes the probabilities of being classfied into the corresponding class.
To optimize the above objective function, a key challenge lies in optimizing the mutual information between the learning outcome and the vectorized sensitive feature I(ỹ; s). Inspired by the seminal work of Belghazi et al. [15], a natural choice would be to apply off-the-shelf mutual information estimation methods for high-dimensional data. Examples include MINE [15], Deep Infomax [16] and CCMI [17], which estimate mutual information by parameterizing neural networks to maximize tight lower bounds of mutual information. However, in a mutual information minimization problem like Eq. ( 5), it is often counter-intuitive to maximize a lower bound of mutual information. Though one could still maximize the objective function of these estimators to estimate the mutual information and use such estimation to guide the optimization of Eq. ( 5) as a minimax game, it is hindered by two hurdles. First, it requires learning a well-trained estimator to estimate the mutual information during each epoch of optimizing Eq. ( 5). Second, if the estimator is not initialized with proper parameter settings, mutual information may be poorly estimated, which could further result in failing to find a good saddle point in such a minimax game.
In this paper, we take a different strategy from MINE and other similar methods by deriving a variational representation of mutual information I(ỹ; s). Our variational representation leverages a variational distribution of the vectorized sensitive feature s given the learning outcome ỹ (Lemma 3).
Lemma 3: Suppose the joint distribution of the learning outcome ỹ and the vectorized sensitive feature s is p ỹ,s and the marginal distributions of ỹ and s are p ỹ and p s , respectively. Mutual information I(ỹ, s) between ỹ and s is as follows. I(ỹ; s) = H(s)+E (ỹ,s)∼p ỹ,s log q s|ỹ +E (ỹ,s)∼p ỹ,s log p ỹ,s p ỹq s|ỹ (7) where q s|ỹ is the variational distribution of s given ỹ.
Proof: Omitted for brevity. Next, we minimize the variational representation shown in Lemma 3, which contains three terms: (1) the entropy H(s), (2) the expectation of log likelihood E (ỹ,s)∼p ỹ,s log q s|ỹ and (3) the expectation of log density ratio E (ỹ,s)∼p ỹ,s log p ỹ,s p ỹ q s|ỹ . For the first term H(s), we assume it to be a constant term, which can be ignored in the optimization stage. The rationale behind our assumption is that, in most (if not all) use cases, the vectorized sensitive feature s relates to the demographic information of an individual (e.g., gender, race, marital status, etc.), which should remain unchanged during the learning process. Then the remaining key challenges lie in (C1) calculating log q s|ỹ and (C2) estimating log p ỹ,s p ỹ q s|ỹ . The intuition of C1 and C2 is that we strive to find a learning outcome ỹ such that (1) ỹ fails to predict the vectorized sensitive feature s (refer to C1), while (2) making it hard to distinguish if the vectorized sensitive feature s is generated from the variational distribution or sampled from the original distribution (refer to C2). C1 -Calculating log q s|ỹ . It can be naturally formulated as a prediction problem, where the input is the learning outcome ỹ and the output is the probability of s being predicted. To solve it, we parameterize a decoder f (ỹ; s; W) (e.g., a neural network) as a sensitive feature predictor to 'reconstruct' s, where W is the learnable parameters in the decoder. log q s|ỹ = log f (ỹ; s; W) (8) For categorical sensitive attribute, log q s|ỹ refers to the log likelihood of classifying ỹ into label s, which can be interpreted as the negative of cross-entropy loss of the decoder f (ỹ; s; W). Moreover, if s contains multiple categorical sensitive attributes, solving Eq. ( 8) requires solving a multilabel classification problem, which itself is not trivial to solve. In this case, we further reduce it to a single-label problem by applying a mapping function map() to map the multi-hot encoding s into a one-hot encoding ŝ (i.e., ŝ = map(s)). C2 -Estimating log p ỹ,s p ỹ q s|ỹ . In practice, calculating p ỹ,s and p ỹq s|ỹ individually is hard since the underlying distributions p ỹ,s and p ỹ are often unknown. Recall that our goal is to estimate the log of the ratio between these two joint distributions. Therefore, we estimate it through density ratio estimation, where the numerator p ỹ,s denotes the original joint distribution of the learning outcome ỹ and ground-truth vectorized sensitive feature s, and the denominator p ỹq s|ỹ denotes the joint distribution of the learning outcome ỹ and the vectorized sensitive feature s generated from the learning outcome using the aforementioned decoder.
We further reduce this density ratio estimation problem to a class probability estimation problem, which was originally developed in [18] for solving a different problem (i.e., the classification problem with the input distribution and the test distribution differing arbitrarily). The core idea is that, given a pair of learning outcome and vectorized sensitive feature, we want to predict whether it is drawn from the original joint distribution or from the joint distribution inferred by the decoder. We label each pair of learning outcome and ground-truth vectorized sensitive feature (ỹ, s) with a positive label (c = 1) and each pair of learning outcome and generated vectorized sensitive feature (ỹ, s) with a negative label (c = -1). After that, we rewrite the probability densities as p ỹ,s = Pr[c = 1|ỹ, s] and p ỹq s|ỹ = Pr[c = -1|ỹ, s]. Then the density ratio can be further rewritten as
Furthermore, if we model Pr[c = 1|ỹ, s] using logistic regression (i.e., Pr[c = 1|ỹ, s] = logistic(ỹ, s)), Eq. ( 9) is reduced to a simple linear function as log p ỹ,s
where both w 1 and w 2 are learnable parameters. Putting everything together, we rewrite Eq. ( 5) as J = E (x,y)∼D l(x; s; y; ỹ; θ)
where p ỹ,s is the joint distribution of the learning outcome ỹ and ground-truth vectorized sensitive feature s, p ỹq s|ỹ is the joint distribution of the learning outcome ỹ and predicted vectorized sensitive feature s.
Based on the objective function (Eq. ( 11)), we propose a generic end-to-end framework to solve the informationtheoretic intersectional fairness problem. A general overview 1. The non-sensitive features and sensitive features (optional) are passed into a feature extractor to extract the learning outcomes; 2. The learning outcomes will be fed into a target predictor to predict the targets for a certain downstream task (i.e., l(x; s; y; ỹ; θ) in Eq. ( 11)); 3. The learning outcomes will be passed into the sensitive feature predictor to 'reconstruct' the vectorized sensitive features (i.e., log q s|ỹ in Eq. ( 11)); 4. Together with the learning outcomes and the ground-truth vectorized sensitive features, the predicted vectorized sensitive features will be used to estimate the density ratio between the original distribution and the variational distribution (i.e., w T 1 ỹ + w T 2 s in Eq. ( 11)). Given a data point with categorical sensitive attribute(s), the predicted vectorized sensitive feature s is usually denoted as a one-hot vector. However, learning a one-hot vector is a difficult problem due to the discrete nature of vector elements, which makes the computation non-differentiable. To address this issue, we approximate such one-hot encoding by Gumbel-Softmax [19], which can be calculated as
, where o s is the output of the sensitive feature predictor, n s is the dimension of s, g 1 , . . . , g ns are i.i.d. points drawn from Gumbel(0, 1) distribution, and τ is the softmax temperature. As τ → ∞, the Gumbel-Softmax samples are uniformly distributed; while as τ → 0, the Gumbel-Softmax distribution converges to a onehot categorical distribution. In INFOFAIR, we start with a high temperature and then anneal it during epochs of training.
The proposed INFOFAIR is able to be generalized in multiple aspects. Due to the space limitation, we only give some brief descriptions, each of which could be a future direction in applying our proposed framework.
A -INFOFAIR with equal opportunity. Our INFOFAIR framework is generalizable to enforce equal opportunity [8], another widely-used group fairness notions. We leave for future work to explore the potential of INFOFAIR in enforcing equal opportunity.
Equal opportunity ensures equality across demographic groups for a preferred label (i.e., the label that benefits an individual). Mathematically, it is defined as follows.
Definition 2: (Equal Opportunity [8]). Following the settings of Definition 1, if equal opportunity is enforced, the hypothesis h :
where Pr[•] denotes the probability of an event happening.
Analogous to the relationship between mutual information and statistical parity, ensuring equal opportunity is essentially a conditional mutual information minimization problem.
p ỹ|s,y=1 = p ỹ|y=1 equal opportunity ⇔ I(ỹ; s|y = 1) = 0 zero conditional mutual information (13) By the definition of conditional mutual information, we have I(ỹ; s|y = 1) = H(s|y = 1) -H(s|ỹ, y = 1). For H(s|y = 1), we assume it as a constant term by the similar rationale of statistical parity. Similarly, we can rewrite H(s|ỹ, y = 1) as H(s|ỹ, y = 1) =E (ỹ,s)∼p ỹ,s|y=1 -log q s|ỹ,y=1
-E (ỹ,s)∼p ỹ,s|y=1 log p ỹ,s|y=1 p ỹ|y=1 q s|ỹ,y=1 (14) Then, to compute log q s|ỹ,y=1 , we could adopt similar strategy as computing log q s|ỹ in Section III-B by constructing a decoder f (ỹ, s, W) to 'reconstruct' s for positive training samples. Similarly, estimating the density ratio can be achieved by applying Eq. ( 10) on positive training samples. Thus, INFOFAIR is able to enforce equal opportunity by minimizing J = E (x,y)∼D l(x; s; y; ỹ; θ) + α log q s|ỹ + αE {(ỹ,s)∼p ỹ,s|y=1 }∪{(ỹ,s)∼p ỹ|y=1 q s|ỹ,y=1 } w T 1 ỹ + w T 2 s (15) B -Relationship to adversarial debiasing. Adversarial debiasing framework [13] consists of (1) a predictor that predicts the class membership probabilities using given data and (2) an adversary that takes the output of the predictor to predict the sensitive attribute of given data. The framework is optimized to minimize the loss function of the predictor while maximizing the loss function of the adversary. If we merge feature extractor and target predictor to one single module and remove the density ratio estimator, INFOFAIR will degenerate to the adversarial debiasing method. C -Relationship to Information Bottleneck. If we set the loss function l in Eq. ( 5) as the negative mutual information -I(ỹ; y), Eq. ( 5) becomes the information bottleneck method [20]. Then the goal becomes to learn ỹ that depends on the vectorized sensitive attribute s minimally and ground truth y maximally. D -Fairness for continuous-valued sensitive features. Most existing works in fair machine learning only consider categorical sensitive attribute (e.g., gender, race). Our proposed INFOFAIR framework could be generalized to continuousvalued features as mutual information supports continuousvalued random variables. This advantage could empower our framework to work in even more application scenarios. For example, in image classification, we can classify images without the impact of certain image patches (e.g., patches that relate to individual's skin color). However, a major difficulty lies in modeling the variational distribution of sensitive attribute given the learning outcomes extracted from feature extractor. A potential solution could be utilizing a generative model (e.g., VAEs [21]) as the sensitive feature predictor. E -Fairness for non-i.i.d. graph data. For fair graph mining, given a graph G = (A, X) where A is the adjacency matrix and X is the node feature matrix, we can use graph convolutional layer(s) as a feature extractor with the weight of the last layer to be identity matrix I and no nonlinear activation in the last graph convolution layer, in order to extract node representations. The reason for such a specific architecture in the last graph convolution layer is as follows. In general, a graph convolutional layer consists of two operations: feature aggregation Z = f aggregate (A; X) = AX and feature transformation H = f transform (Z; W) = σ(ZW) where W is learnable parameters and σ is usually a nonlinear activation. The last layer in GCN [22] is simply softmax(AXW), which can be viewed as a general multi-class logistic regression on the aggregated feature Z = AX (i.e., softmax(ZW)). F -Fairness beyond classification. Note that INFOFAIR does not have specific restrictions on the architecture of the feature extractor, target predictor or sensitive target predictor, which empowers it to handle many different types of tasks by selecting the proper architecture for each module. For example, if an analyst aims to learn fair representations with respect to gender for recommendation, s/he can set the feature extractor to be a multi-layer perceptron (MLP) for learning outcome extraction, the target predictor layer to be an MLP that predicts a rating and minimizes the mean squared error (MSE) between the predicted rating and ground-truth rating, and the sensitive target predictor to be another MLP with softmax to predict the gender based on extracted embedding.
IV. EXPERIMENTAL EVALUATION In this section, we conduct experimental evaluations. All experiments are designed to answer the following questions:
RQ1. How does the fairness impact the learning performance? RQ2. How effective is INFOFAIR in mitigating bias? A. Experimental Settings A -Datasets. We test the proposed method on three commonly-used datasets in fair machine learning research. The statistics of these datasets are summarized in Table II. B -Baseline Methods. We compare INFOFAIR with several baseline methods, including Learning Fair Representations (LFR) [12], MinDiff [23], Generalized Demographic Parity (GDP) [24], Adversarial Debiasing (Adversarial) [13], Fair Classification with Fairness Constraints (FCFC) [6], Gerry-Fair [9] and Disparate Impact (DI) [5]. C -Metrics. To answer RQ1, we measure the performance of classification using micro F1 and macro F1 (Micro/Macro F1).
To . Note that relative bias reduction defined above can be negative if the debiased learning outcome contains more biases than the vanilla learning outcome.
More experimental settings regarding reproducibility are provided in Appendix.
We test our proposed framework, as well as baseline methods, in three different settings: debiasing binary sensitive attribute (i.e., gender for all three datasets), debiasing nonbinary sensitive attribute (i.e., race for COMPAS and Adult Income, marital status for Dutch Census) and debiasing multiple sensitive attributes (i.e., gender & race for COMPAS and Adult Income, gender & marital status for Dutch Census). For each dataset and each setting, we train each model on training set, then select the trained model with best bias mitigation performance on validation set and report its performance on the test set. For the vanilla model (without any fairness consideration), we report the model with the highest micro and macro F1 scores. This is because the algorithm administrators are often more concerned with maximizing the utility of classification algorithms. The results of LFR and MinDiff in debiasing non-binary sensitive attribute and multiple sensitive attributes are absent since they only handle binary sensitive attribute by design. When debiasing race on COMPAS dataset and debiasing marital status on Dutch Census dataset, we observe that the cardinalities of the demographic groups are more imbalanced. Recall the goal of sensitive feature predictor is to reduce the accuracy of predicting sensitive feature using the extracted embedding. When the demographic groups are more imbalanced, it tends to learn an embedding that contains information about a wrong demographic group to reduce its accuracy. Thus, though T + S may achieve lower imparity, the statistical dependence between the extracted embeddings and the sensitive feature may not be reduced, meaning that the extracted embeddings are merely shifted to correlate with wrong demographic groups. However, with the addition of density ratio estimator, we ensure that (1) not only the sensitive feature predictor makes wrong prediction (2) but also the distribution of sensitive attribute and the extracted embeddings modeled by S (p ỹq s|ỹ ) are similar to its corresponding original distribution (i.e., p ỹ,s ).
V. RELATED WORK A -Group fairness aims to ensure statistical-based fairness notions across the entire populations. It has been extensively studied in many application domains, including credit scoring [5], recidivism [25], healthcare [12], recommender systems [26] and natural language processing [27]. Kamishima et al. [28] estimate mutual information between the learning outcome and sensitive attribute by marginalizing the output of a probabilistic discriminative model. Zemel et al. [12] use a regularized approach to learn fair embeddings for group fairness and individual fairness. Feldman et al. [5] debias input data distribution by linear interpolation of original data distribution and fair data distribution. Zhang et al. [13] propose an adversarial debiasing framework. Bose et al. [10] fair representations by However, their proposed framework could only multiple sensitive attributes instead of multiple sensitive attributes simultaneously. Kearns [9] further consider fairness from Different from [9], INFOFAIR directly statistical parity through mutual information minimization instead of optimizing the selfdefined surrogate 'fairness violation' functions using gametheoretic method. Zafar et al. [6] ensure statistical parity by minimizing the covariance between the sensitive attribute of each data sample and its distance to the decision boundary of a convex margin-based classifier. Adeli et al. [29] remove statistical dependence by minimizing Pearson's correlation for a convolutional neural network. Nevertheless, [6], [29] only remove the linear dependence whereas our proposed INFOFAIR removes both linear nonlinear dependence directly. In addition to statistical parity and disparate impact, Hardt et al. [8] propose another widely-used fairness notion named equal opportunity. Prost et al. [23] achieve equal false positive rate through maximum mean discrepancy (MMD) minimization. However, it can only debias with respect to binary sensitive attribute by design. Jiang et al. [24] propose generalized demographic parity for tractable calculation demographic parity with respect to continuous-valued sensitive attributes. In terms of intersectional fairness, Kim et al. [30] propose Multiaccuracy Boost to ensure low classification error for each intersectional demographic group. Foulds et al. [31] propose ϵ-differential fairness, which ensures pairwise equal acceptance rate. They further estimate ϵ-differential fairness and its corresponding uncertainty [32]. Morina et al. [33] develop the equivalence between minimizing ϵ-differential fairness and minimizing a linear combination of false positive rate and false negative rate in a binary classification problem. Ramos et al. [34] ensure intersectional fairness in reputationbased ranking systems by minimizing the difference among the average reputations of a user from different demographic groups. Different from [34], our proposed INFOFAIR ensures intersectional fairness from information-theoretic perspective, and is applicable to various learning tasks as shown in Section III-D. B -Mutual information estimation for high-dimensional data has been made possible in recent decades by analyzing variational bounds of mutual information with machine learning techniques. Regarding variational upper bound of mutual information, Kingma et al. [21] and Rezende et al. [35] almost concurrently propose Variational Auto-Encoders (VAEs) which optimizes a variational upper bound of mutual informa-tion conceptually. Variational lower bounds of mutual information have been extensively studied recently. Barber et al. [36] propose a variational lower bound of mutual information and maximize the mutual information through moment matching. Belghazi et al. [15] propose Mutual Information Neural Estimation (MINE) which maximizes Donsker-Varadhan representation of Kullback-Leibler (KL) divergence [37]. In [15], MINE-f , a variant of MINE, is proposed to maximize the variational estimation of f -divergence introduced by Nguyen et al. [38]. The same variational representation of fdivergence has been applied to other generative models like f -GAN [39]. Mukherjee et al. [17] propose a classifier-based neural estimator for conditional mutual information named CCMI. In addition, van den Oord et al. [40] propose infoNCE based on noise contrastive estimation (NCE) [41]. Hjelm et al. [16] propose Deep Infomax (DIM) to maximize the mutual information between global representation and local regions of the input, which is further generalized to graphs [42].
In this paper, we study information-theoretic intersectional fairness, where we aim to simultaneously debias the learning results with respect to multiple sensitive attributes. We formally define the information-theoretic intersectional fairness problem by measuring the dependence between the learning results and multiple sensitive attributes as the mutual information between learning results and a joint attribute formed by these sensitive attributes. Based on that, we formulate it as an optimization problem and further propose a generic end-to-end framework, which effectively minimizes mutual information between the learning results and the joint attribute through its variational representation. We perform fair classification on three real-world datasets with the consideration of categorical sensitive attributes. The empirical evaluation results demonstrate that our proposed framework can effectively debias the classification results with respect to one or more sensitive attribute(s) with little sacrifice to the classification accuracy. Our framework is generalizable to different settings beyond the scope of fair classification with categorical sensitive attributes in our experimental evaluation. In the future, we will investigate our framework in other learning tasks (e.g., recommendation) and its effectiveness in mitigating bias for continuous-valued sensitive attributes (e.g., age, income).
https://www.nytimes.com/2019/11/10/business/Apple-credit-cardinvestigation.html
An orthogonal work in algorithmic fairness is individual fairness. Although it promises fairness by 'treating similar individuals similarly' in principle, it is often hard to be operationalized in practice due to its strong assumption on distance metrics and data distributions.
We use imaginary race groups to avoid potential offenses.