skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Fairness in representation: quantifying stereotyping as a representational harm
While harms of allocation have been increasingly studied as part of the subfield of algorithmic fairness, harms of representation have received considerably less attention. In this paper, we formalize two notions of stereotyping and show how they manifest in later allocative harms within the machine learning pipeline. We also propose mitigation strategies and demonstrate their effectiveness on synthetic datasets.  more » « less
Award ID(s):
1633387
PAR ID:
10121133
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Proceedings of the 2019 SIAM International Conference on Data Mining
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Nah, Fiona (Ed.)
    Misinformation about the coronavirus disease of 2019 (COVID-19) health crisis has been widespread on social media and caused various types of harms in society. While some researchers have investigated the way in which people perceive misinformation harm in crises, little research has systematically examined harms from health-related misinformation. In order to address this gap, we focus on non-comparative and comparative harm perceptions of the affected community in the COVID-19 pandemic context. We examine non-comparative harms (which component harms and contextual harms reflect) and comparative harms (which counter-contextual harms reflect) in order to understand harm perceptions. We also investigate how harm perception varies based on COVID-19 victimization experience. We used a professional survey company named Cint to collect data using a scenario-based survey with 343 participants. We extract various findings such as how contextual features shape perceived harms and reveal the scenarios in which COVID-19 victims perceive higher contextual harms but lower counter-contextual harms. We also examine how corrective actions of social media shape harm perceptions. 
    more » « less
  2. While harms of allocation have been increasingly studied as part of the subfield of algorithmic fairness, harms of representation have received considerably less attention. In this paper, we formalize two notions of stereotyping and show how they manifest in later allocative harms within the machine learning pipeline. We also propose mitigation strategies and demonstrate their effectiveness on synthetic datasets. 
    more » « less
  3. Recent advances in the capacity of large language models to generate human-like text have resulted in their increased adoption in user-facing settings. In parallel, these improvements have prompted a heated discourse around the risks of societal harms they introduce, whether inadvertent or malicious. Several studies have explored these harms and called for their mitigation via development of safer, fairer models. Going beyond enumerating the risks of harms, this work provides a survey of practical methods for addressing potential threats and societal harms from language generation models. We draw on several prior works’ taxonomies of language model risks to present a structured overview of strategies for detecting and ameliorating different kinds of risks/harms of language generators. Bridging diverse strands of research, this survey aims to serve as a practical guide for both LM researchers and practitioners, with explanations of different strategies’ motivations, their limitations, and open problems for future research. 
    more » « less
  4. During humanitarian crises, people face dangers and need a large amount of information in a short period of time. Such need creates the base for misinformation such as rumors, fake news or hoaxes to spread within and outside the affected community. It could be unintended misinformation with unconfirmed details, or intentional disinformation created to trick people for benefits. It results in information harms that can generate serious short term or long-term consequences. Although some researchers have created misinformation detection systems and algorithms, examined the roles of involved parties, examined the way misinformation spreads and convinces people, very little attention has been paid to the types of misinformation harms. In the context of humanitarian crises, we propose a taxonomy of information harms and assess people’s perception of risk regarding the harms. Such a taxonomy can act as the base for future research to quantitatively measure the harms in specific contexts. Furthermore, perceptions of related people were also investigated in four specifically chosen scenarios through two dimensions: Likelihood of occurrence and Level of impacts of the harms. 
    more » « less
  5. Abstract Research partnerships between Tribal Nations and rural colleges and universities can support rural development and strengthen Tribal Nation building through reclamation of economic, political, cultural, and social affairs. However, Tribal Nation–University relationships have received little attention in rural sociology. While scholars identify best practices for research engagement in light of colonial harms, the ideal visions that Tribally and university‐affiliated people have for research partnerships and the barriers to achieving those ideals are poorly understood. Without identifying these visions and barriers, we risk making wrong assumptions about each party's needs and cannot implement appropriate policies. Semi‐structured interviews with Tribally‐affiliated (n = 20) and university‐affiliated (n = 20) people in rural southeastern Idaho suggest, contrary to literature on best practices for collaborative research, that participants in both groups viewed what we term “Tribally‐responsive research engagement” as ideal, though few projects met this goal. Tribally‐responsive research directly addressed Tribal priorities but did not necessarily involve close collaboration. The University's failure to acknowledge past or colonial harms, university‐affiliated researchers' historicization of those harms, and negative Native student experiences reinforced distrust, limiting desired research engagement. In sum, Tribally‐responsive research engagement could strengthen Native Nation building, but requires universities to acknowledge harms, create more welcoming campus environments, and prioritize Tribal benefits in research. 
    more » « less