More Like this

1. Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach

   https://doi.org/10.18653/v1/2021.acl-long.168  

   Cheng, L. ; Mosallanezhad, A. ; Silva, Y. N. ; Hall, D. L. ; Liu, H. ( August 2021 , The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP))

   null (Ed.)

   The element of repetition in cyberbullying behavior has directed recent computational studies toward detecting cyberbullying based on a social media session. In contrast to a single text, a session may consist of an initial post and an associated sequence of comments. Yet, emerging efforts to enhance the performance of session-based cyberbullying detection have largely overlooked unintended social biases in existing cyberbullying datasets. For example, a session containing certain demographic-identity terms (e.g., “gay” or “black”) is more likely to be classified as an instance of cyberbullying. In this paper, we first show evidence of such bias in models trained on sessions collected from different social media platforms (e.g., Instagram). We then propose a context-aware and model-agnostic debiasing strategy that leverages a reinforcement learning technique, without requiring any extra resources or annotations apart from a pre-defined set of sensitive triggers commonly used for identifying cyberbullying instances. Empirical evaluations show that the proposed strategy can simultaneously alleviate the impacts of the unintended biases and improve the detection performance. 

   more » « less
2. Assessing the Quality of Southern Ocean Circulation in CMIP5 AOGCM and Earth System Model Simulations

   https://doi.org/10.1175/JCLI-D-19-0263.1  

   Beadling, R. L. ; Russell, J. L. ; Stouffer, R. J. ; Goodman, P. J. ; Mazloff, M. ( August 2019 , Journal of Climate)

   The Southern Ocean (SO) is vital to Earth’s climate system due to its dominant role in exchanging carbon and heat between the ocean and atmosphere and transforming water masses. Evaluating the ability of fully coupled climate models to accurately simulate SO circulation and properties is crucial for building confidence in model projections and advancing model fidelity. By analyzing multiple biases collectively across large model ensembles, physical mechanisms governing the diverse mean-state SO circulation found across models can be identified. This analysis 1) assesses the ability of a large ensemble of models contributed to phase 5 of the Coupled Model Intercomparison Project (CMIP5) to simulate observationally based metrics associated with an accurate representation of the Antarctic Circumpolar Current (ACC), and 2) presents a framework by which the quality of the simulation can be categorized and mechanisms governing the resulting circulation can be deduced. Different combinations of biases in critical metrics including the magnitude and position of the zonally averaged westerly wind stress maximum, wind-driven surface divergence, surface buoyancy fluxes, and properties and transport of North Atlantic Deep Water entering the SO produce distinct mean-state ACC transports. Relative to CMIP3, the quality of the CMIP5 SO simulations has improved. Eight of the thirty-one models simulate an ACC within observational uncertainty (2σ) for approximately the right reasons; that is, the models achieve accuracy in the surface wind stress forcing and the representation of the difference in the meridional density across the current. Improved observations allow for a better assessment of the SO circulation and its properties.

    

   more » « less
3. Learning for Counterfactual Fairness from Observational Data

   https://doi.org/10.1145/3580305.3599408  

   Ma, Jing ; Guo, Ruocheng ; Zhang, Aidong ; Li, Jundong ( August 2023 , ACM)

   Fairness-aware machine learning has attracted a surge of attention in many domains, such as online advertising, personalized recommendation, and social media analysis in web applications. Fairness-aware machine learning aims to eliminate biases of learning models against certain subgroups described by certain protected (sensitive) attributes such as race, gender, and age. Among many existing fairness notions, counterfactual fairness is a popular notion defined from a causal perspective. It measures the fairness of a predictor by comparing the prediction of each individual in the original world and that in the counterfactual worlds in which the value of the sensitive attribute is modified. A prerequisite for existing methods to achieve counterfactual fairness is the prior human knowledge of the causal model for the data. However, in real-world scenarios, the underlying causal model is often unknown, and acquiring such human knowledge could be very difficult. In these scenarios, it is risky to directly trust the causal models obtained from information sources with unknown reliability and even causal discovery methods, as incorrect causal models can consequently bring biases to the predictor and lead to unfair predictions. In this work, we address the problem of counterfactually fair prediction from observational data without given causal models by proposing a novel framework CLAIRE. Specifically, under certain general assumptions, CLAIRE effectively mitigates the biases from the sensitive attribute with a representation learning framework based on counterfactual data augmentation and an invariant penalty. Experiments conducted on both synthetic and real-world datasets validate the superiority of CLAIRE in both counterfactual fairness and prediction performance. 

   more » « less
4. Bias Mitigation for Toxicity Detection via Sequential Decisions

   Cheng, L. ; Mosallanezhad, A. ; Silva, Y. N. ; Hall, D. ( January 2022 , The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR))

   Increased social media use has contributed to the greater prevalence of abusive, rude, and offensive textual comments. Machine learning models have been developed to detect toxic comments online, yet these models tend to show biases against users with marginalized or minority identities (e.g., females and African Americans). Established research in debiasing toxicity classifiers often (1) takes a static or batch approach, assuming that all information is available and then making a one-time decision; and (2) uses a generic strategy to mitigate different biases (e.g., gender and racial biases) that assumes the biases are independent of one another. However, in real scenarios, the input typically arrives as a sequence of comments/words over time instead of all at once. Thus, decisions based on partial information must be made while additional input is arriving. Moreover, social bias is complex by nature. Each type of bias is defined within its unique context, which, consistent with intersectionality theory within the social sciences, might be correlated with the contexts of other forms of bias. In this work, we consider debiasing toxicity detection as a sequential decision-making process where different biases can be interdependent. In particular, we study debiasing toxicity detection with two aims: (1) to examine whether different biases tend to correlate with each other; and (2) to investigate how to jointly mitigate these correlated biases in an interactive manner to minimize the total amount of bias. At the core of our approach is a framework built upon theories of sequential Markov Decision Processes that seeks to maximize the prediction accuracy and minimize the bias measures tailored to individual biases. Evaluations on two benchmark datasets empirically validate the hypothesis that biases tend to be correlated and corroborate the effectiveness of the proposed sequential debiasing strategy. 

   more » « less
5. Unfairness Discovery and Prevention For Few-Shot Regression

   https://doi.org/10.1109/ICBK50248.2020.00029  

   Zhao, Chen ; Chen, Feng ( April 2021 , in Proceedings of the IEEE International Conference on Knowledge Graph (ICKG))

   null (Ed.)

   We study fairness in supervised few-shot meta-learning models that are sensitive to discrimination (or bias) in historical data. A machine learning model trained based on biased data tends to make unfair predictions for users from minority groups. Although this problem has been studied before, existing methods mainly aim to detect and control the dependency effect of the protected variables (e.g. race, gender) on target prediction based on a large amount of training data. These approaches carry two major drawbacks that (1) lacking showing a global cause-effect visualization for all variables; (2) lacking generalization of both accuracy and fairness to unseen tasks. In this work, we first discover discrimination from data using a causal Bayesian knowledge graph which not only demonstrates the dependency of the protected variable on target but also indicates causal effects between all variables. Next, we develop a novel algorithm based on risk difference in order to quantify the discriminatory influence for each protected variable in the graph. Furthermore, to protect prediction from unfairness, a the fast-adapted bias-control approach in meta-learning is proposed, which efficiently mitigates statistical disparity for each task and it thus ensures independence of protected attributes on predictions based on biased and few-shot data samples. Distinct from existing meta-learning models, group unfairness of tasks are efficiently reduced by leveraging the mean difference between (un)protected groups for regression problems. Through extensive experiments on both synthetic and real-world data sets, we demonstrate that our proposed unfairness discovery and prevention approaches efficiently detect discrimination and mitigate biases on model output as well as generalize both accuracy and fairness to unseen tasks with a limited amount of training samples. 

   more » « less