skip to main content


Title: SoK: Hate, Harassment, and the Changing Landscape of Online Abuse
We argue that existing security, privacy, and anti-abuse protections fail to address the growing threat of online hate and harassment. In order for our community to understand and address this gap, we propose a taxonomy for reasoning about online hate and harassment. Our taxonomy draws on over 150 interdisciplinary research papers that cover disparate threats ranging from intimate partner violence to coordinated mobs. In the process, we identify seven classes of attacks—such as toxic content and surveillance—that each stem from different attacker capabilities and intents. We also provide longitudinal evidence from a three-year survey that hate and harassment is a pervasive, growing experience for online users, particularly for at-risk communities like young adults and people who identify as LGBTQ+. Responding to each class of hate and harassment requires a unique strategy and we highlight five such potential research directions that ultimately empower individuals, communities, and platforms to do so.  more » « less
Award ID(s):
1916096
NSF-PAR ID:
10295029
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ; ; ; ; ;
Date Published:
Journal Name:
42nd IEEE Symposium on Security and Privacy
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. While most moderation actions on major social platforms are performed by either the platforms themselves or volunteer moderators, it is rare for platforms to collaborate directly with moderators to address problems. This paper examines how the group-chatting platform Discord coordinated with experienced volunteer moderators to respond to hate and harassment toward LGBTQ+ communities during Pride Month, June 2021, in what came to be known as the "Pride Mod" initiative. Representatives from Discord and volunteer moderators collaboratively identified and communicated with targeted communities, and volunteers temporarily joined servers that requested support to supplement those servers' existing volunteer moderation teams. Though LGBTQ+ communities were subject to a wave of targeted hate during Pride Month, the communities that received the requested volunteer support reported having a better capacity to handle the issues that arose. This paper reports the results of interviews with 11 moderators who participated in the initiative as well as the Discord employee who coordinated it. We show how this initiative was made possible by the way Discord has cultivated trust and built formal connections with its most active volunteers, and discuss the ethical implications of formal collaborations between for-profit platforms and volunteer users. 
    more » « less
  2. Online harassment against women - particularly in gaming and virtual worlds contexts - remains a salient and pervasive issue, and arguably reflects the systems of offline structural oppression to control women’s bodies and rights in today’s world. Harassment in social Virtual Reality (VR) is also a growing new frontier of research in HCI and CSCW, particularly focusing on marginalized users such as women. Based on interviews with 31 women users of social VR, our findings present women’s experiences of harassment risks in social VR as compared to harassment targeting women in pre-existing, on-screen online gaming and virtual worlds, along with strategies women employ to manage harassment in social VR with varying degrees of success. This study contributes to the growing body of literature on harassment in social VR by highlighting how women’s marginalization online and offline impact their perceptions of and strategies to mitigate harassment in this unique space. It also provides a critical reflection on women’s mitigation strategies and proposes important implications to rethink social VR design to better prevent harassment against women and other marginalized communities in the future metaverse. 
    more » « less
  3. Journalists, fact-checkers, academics, and community media are overwhelmed in their attempts to support communities suffering from gender-, race- and ethnicity-targeted information ecosystem threats, including but not limited to misinformation, hate speech, weaponized controversy and online-to-offline harassment. Yet, for a plethora of reasons, minoritized groups are underserved by current approaches to combat such threats. In this panel, we will present and discuss the challenges and open problems facing such communities and the researchers hoping to serve them. We will also discuss the current state-of-the-art as well as the most promising future directions, both within IR specifically, across Computer Science more broadly, as well as that requiring transdisciplinary and cross-sectoral collaborations. The panel will attract both IR practitioners and researchers and include at least one panelist outside of IR, with unique expertise in this space. 
    more » « less
  4. Harassment has long been considered a severe social issue and a culturally contextualized construct. More recently, understanding and mitigating emerging harassment in social Virtual Reality (VR) has become a growing research area in HCI and CSCW. Based on the perspective of harassment in the U.S. culture, in this paper we identify new characteristics of online harassment in social VR using 30 in-depth interviews. We especially attend to how people who are already considered marginalized in the gaming and virtual worlds contexts (e.g., women, LGBTQ, and ethnic minorities) experience such harassment. As social VR is still a novel technology, our proactive approach highlights embodied harassment as an emerging but understudied form of harassment in novel online social spaces. Our critical review of social VR users' experiences of harassment and recommendations to mitigate such harassment also extends the current conceptualization of online harassment in CSCW. We therefore contribute to the active prevention of future harassment in nuanced online environments, platforms, and experiences. 
    more » « less
  5. The 100,000 ha Sevilleta National Wildlife Refuge (SNWR) in central New Mexico lies in a transition zone that straddles several major biomes of the Southwest, including Great Basin Shrub-Steppe, Mogollon Pinon-Juniper Woodland, Great Plains Grassland and Chihuahuan Desert. During 9 years, (1990-1998), collaborating with the University of New Mexico's Long Term Ecological Research (LTER) program, 3,235 rodents (28 species in 4 families) were collected and identified from permanent collecting sites on the 3 major habitat types (grassland, desert/creosote, woodland) on the SNWR. Hosts were necropsied for endoparasites (protozoa [coccidia], helminths) and some ectoparasites. We identified and analyzed all the parasites found in these hosts. By 1998, we had in place the means to easily identify and moniter the parasites from all mammalian hosts caught on the LTER Phase II grant.This is not just another parasite survey; the data we collected was unique for several reasons: 1) This was the first complete inventory of a natural assemblage of parasites from all mammalian (rodent) hosts in 3 different communities, each from a distinctly defined geographic locality (habitat type) over the period of a decade, and beyond; 2) This study was part of a multidisciplinary approach to address conceptual issues of climate change on ecosystem structure and function at multiple scales (individuals, communities, etc) and correlative data from these related studies will strengthen and contribute to the robustness of this data set; 3) As the only parasite study on any of the LTER projects nationwide, it provided an ideal model, and perhaps incentive for parallel longterm studies of parasite communities to be examined in a variety of other habitat types, and from a variety of different perspectives, and other LTER sites in the network.Upon completing the work, we were able to use these long-term data to try to understand the dynamics of natural host-parasite assemblages. Hypotheses were then erected to test/address at least these questions: How do the different parasite communities colonize, mature, climax and senesce over time (or do they?), Do they vary in response to abiotic (climate change) and/or biotic (dispersal, colonization) factors? What temporal and spatial scales, and among what kinds of organisms, do coevolutionary processes influence the community organization of these parasites? Studies of the dynamics of multiple, coexisting species are confined primarily to microtine rodents and have hinted that multiannual cycles tend to be synchronous (Brown and Heske 1990). Are similar patterns seen for the parasites of our desert rodents? Answers to these questions relating to community structure, as well as to questions concerning parasite biodiversity on the SNWR, can be answered paritially or completely by the information we gathered on the parasite species infecting rodents collected on the SNWR. Initial emphasis of our work was on identifying all the parasites collected, by processing 8 consecutive years of parasite data, and on training the undergraduate and graduate students involved in the art of taxonomy and nomenclature of parasitic protozoans and helminths,  to supply some of these answers. 
    more » « less