More Like this

1. Stochastic Differentially Private and Fair Learning

   Lowy, Andrew ; Gupta, Devansh ( February 2023 , The Eleventh International Conference on Learning Representations)

   Machine learning models are increasingly used in high-stakes decision-making systems. In such applications, a major concern is that these models sometimes discriminate against certain demographic groups such as individuals with certain race, gender, or age. Another major concern in these applications is the violation of the privacy of users. While fair learning algorithms have been developed to mitigate discrimination issues, these algorithms can still leak sensitive information, such as individuals’ health or financial records. Utilizing the notion of differential privacy (DP), prior works aimed at developing learning algorithms that are both private and fair. However, existing algorithms for DP fair learning are either not guaranteed to converge or require full batch of data in each iteration of the algorithm to converge. In this paper, we provide the first stochastic differentially private algorithm for fair learning that is guaranteed to converge. Here, the term “stochastic" refers to the fact that our proposed algorithm converges even when minibatches of data are used at each iteration (i.e. stochastic optimization). Our framework is flexible enough to permit different fairness notions, including demographic parity and equalized odds. In addition, our algorithm can be applied to non-binary classification tasks with multiple (non-binary) sensitive attributes. As a byproduct of our convergence analysis, we provide the first utility guarantee for a DP algorithm for solving nonconvex-strongly concave min-max problems. Our numerical experiments show that the proposed algorithm consistently offers significant performance gains over the state-of-the-art baselines, and can be applied to larger scale problems with non-binary target/sensitive attributes. 

   more » « less
2. A Stochastic Optimization Framework for Fair Risk Minimization

   Lowy, Andrew ; Baharlouei, Sina ; Pavan, Rakesh ; Razaviyayn, Meisam ; Beirami, Ahmad ( December 2022 , Transactions on machine learning research)

   Despite the success of large-scale empirical risk minimization (ERM) at achieving high accuracy across a variety of machine learning tasks, fair ERM is hindered by the incompatibility of fairness constraints with stochastic optimization. We consider the problem of fair classification with discrete sensitive attributes and potentially large models and data sets, requiring stochastic solvers. Existing in-processing fairness algorithms are either impractical in the large-scale setting because they require large batches of data at each iteration or they are not guaranteed to converge. In this paper, we develop the first stochastic in-processing fairness algorithm with guaranteed convergence. For demographic parity, equalized odds, and equal opportunity notions of fairness, we provide slight variations of our algorithm–called FERMI–and prove that each of these variations converges in stochastic optimization with any batch size. Empirically, we show that FERMI is amenable to stochastic solvers with multiple (non-binary) sensitive attributes and non-binary targets, performing well even with minibatch size as small as one. Extensive experiments show that FERMI achieves the most favorable tradeoffs between fairness violation and test accuracy across all tested setups compared with state-of-the-art baselines for demographic parity, equalized odds, equal opportunity. These benefits are especially significant with small batch sizes and for non-binary classification with large number of sensitive attributes, making FERMI a practical, scalable fairness algorithm. The code for all of the experiments in this paper is available at: https://github.com/optimization-for-data-driven-science/FERMI. 

   more » « less
3. Privacy-Preserving Motor Intent Classification via Feature Disentanglement

   Jiahao Fan, Xiaogang Hu ( April 2023 , 11th International IEEE EMBS Conference on Neural Engineering)

   Recent studies have revealed that sensitive and private attributes could be decoded from sEMG signals, which incurs significant privacy threats to the users of sEMG applications. Most researches so far focus on improving the accuracy and reliability of sEMG models, but much less attention has been paid to their privacy. To fill this gap, this paper implemented and evaluated a framework to optimize the sEMG-based data-sharing mechanism. Our primary goal is to remove sensitive attributes in the sEMG features before sharing them with primary tasks while maintaining the data utility. We disentangled the identity-invariant task-relevant representations from original sEMG features. We shared it with the downstream pattern recognition tasks to reduce the chance of sensitive attributes being inferred by potential attackers. The proposed method was evaluated on data from twenty subjects, with training and testing data acquired 3-25 days apart. Experimental results show that the disentangled representations significantly lower the success rate of identity inference attacks compared to the original feature and its sparse representations generated by the state-of-the-art feature projection methods. Furthermore, the utility of the disentangled representation is also evaluated in hand gesture recognition tasks, showing superior performance over other methods. This work shows that disentangled representations of sEMG signals are a promising solution for privacy-reserving applications. 

   more » « less
4. Fairness Through Robustness: Investigating Robustness Disparity in Deep Learning

   https://doi.org/10.1145/3442188.3445910  

   Nanda, Vedant ; Dooley, Samuel ; Singla, Sahil ; Feizi, Soheil ; Dickerson, John P. ( March 2021 , Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency)

   Deep neural networks (DNNs) are increasingly used in real-world applications (e.g. facial recognition). This has resulted in concerns about the fairness of decisions made by these models. Various notions and measures of fairness have been proposed to ensure that a decision-making system does not disproportionately harm (or benefit) particular subgroups of the population. In this paper, we argue that traditional notions of fairness that are only based on models' outputs are not sufficient when the model is vulnerable to adversarial attacks. We argue that in some cases, it may be easier for an attacker to target a particular subgroup, resulting in a form of robustness bias. We show that measuring robustness bias is a challenging task for DNNs and propose two methods to measure this form of bias. We then conduct an empirical study on state-of-the-art neural networks on commonly used real-world datasets such as CIFAR-10, CIFAR-100, Adience, and UTKFace and show that in almost all cases there are subgroups (in some cases based on sensitive attributes like race, gender, etc) which are less robust and are thus at a disadvantage. We argue that this kind of bias arises due to both the data distribution and the highly complex nature of the learned decision boundary in the case of DNNs, thus making mitigation of such biases a non-trivial task. Our results show that robustness bias is an important criterion to consider while auditing real-world systems that rely on DNNs for decision making. Code to reproduce all our results can be found here: https://github.com/nvedant07/Fairness-Through-Robustness 

   more » « less
5. Social media and volunteer rescue requests prediction with random forest and algorithm bias detection: a case of Hurricane Harvey

   https://doi.org/10.1088/2515-7620/acde35  

   Mihunov, Volodymyr V. ; Wang, Kejin ; Wang, Zheye ; Lam, Nina S. N. ; Sun, Mingxuan ( June 2023 , Environmental Research Communications)

   AI fairness is tasked with evaluating and mitigating bias in algorithms that may discriminate towards protected groups. This paper examines if bias exists in AI algorithms used in disaster management and in what manner. We consider the 2017 Hurricane Harvey when flood victims in Houston resorted to social media to request for rescue. We evaluate a Random Forest regression model trained to predict Twitter rescue request rates from social-environmental data using three fairness criteria (independence, separation, and sufficiency). The Social Vulnerability Index (SVI), its four sub-indices, and four variables representing digital divide were considered sensitive attributes. The Random Forest regression model extracted seven significant predictors of rescue request rates, and from high to low importance they were percent of renter occupied housing units, percent of roads in flood zone, percent of flood zone area, percent of wetland cover, percent of herbaceous, forested and shrub cover, mean elevation, and percent of households with no computer or device. Partial Dependence plots of rescue request rates against each of the seven predictors show the non-linear nature of their relationships. Results of the fairness evaluation of the Random Forest model using the three criteria show no obvious biases for the nine sensitive attributes, except that a minor imperfect sufficiency was found with the SVI Housing and Transportation sub-index. Future AI modeling in disaster research could apply the same methodology used in this paper to evaluate fairness and help reduce unfair resource allocation and other social and geographical disparities.

    

   more » « less