An official website of the United States government
Researchers in areas as diverse as computer science and political science must increasingly navigate the possible risks of their research to society. However, the history of medical experiments on vulnerable individuals influenced many research ethics reviews to focus exclusively on risks to human subjects rather than risks to human society. We describe an Ethics and Society Review board (ESR), which fills this moral gap by facilitating ethical and societal reflection as a requirement to access grant funding: Researchers cannot receive grant funding from participating pro-grams until the researchers complete the ESR process for their proposal. Researchers author an initial statement describing their proposed research’s risks to society, subgroups within society, and globally and commit to mitigation strategies for these risks. An interdisciplinary faculty panel iterates with the researchers to refine these risks and mitigation strategies. We describe a mixed-method evaluation of the ESR over 1 y, in partnership with an artificial intelligence grant program run by Stanford HAI. Surveys and interviews of researchers who interacted with the ESR found100% (95% CI: 87 to 100%) were willing to continue submitting future projects to the ESR, and 58% (95% CI: 37 to 77%) felt that it had influenced the design of their research project. The ESR panel most commonly identified issues of harms to minority groups, inclusion of diverse stakeholders in the research plan, dual use, and representation in datasets. These principles, paired with possible mitigation strategies, offer scaffolding for future research designs.
more »
« less
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Recent advances in the capacity of large language models to generate human-like text have resulted in their increased adoption in user-facing settings. In parallel, these improvements have prompted a heated discourse around the risks of societal harms they introduce, whether inadvertent or malicious. Several studies have explored these harms and called for their mitigation via development of safer, fairer models. Going beyond enumerating the risks of harms, this work provides a survey of practical methods for addressing potential threats and societal harms from language generation models. We draw on several prior works’ taxonomies of language model risks to present a structured overview of strategies for detecting and ameliorating different kinds of risks/harms of language generators. Bridging diverse strands of research, this survey aims to serve as a practical guide for both LM researchers and practitioners, with explanations of different strategies’ motivations, their limitations, and open problems for future research.
more »
« less
- PAR ID:
- 10433138
- Date Published:
- Journal Name:
- European Chapter of the Association for Computational Linguistics
- Page Range / eLocation ID:
- 3299–3321
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Recent studies show that Natural Language Processing (NLP) technologies propagate societal biases about demographic groups associated with attributes such as gender, race, and nationality. To create interventions and mitigate these biases and associated harms, it is vital to be able to detect and measure such biases. While existing works propose bias evaluation and mitigation methods for various tasks, there remains a need to cohesively understand the biases and the specific harms they measure, and how different measures compare with each other. To address this gap, this work presents a practical framework of harms and a series of questions that practitioners can answer to guide the development of bias measures. As a validation of our framework and documentation questions, we also present several case studies of how existing bias measures in NLP—both intrinsic measures of bias in representations and extrinsic measures of bias of downstream applications—can be aligned with different harms and how our proposed documentation questions facilitates more holistic understanding of what bias measures are measuring.more » « less
-
Mounting evidence indicates that the artificial intelligence (AI) systems that rank our social media feeds bear nontrivial responsibility for amplifying partisan animosity: negative thoughts, feelings, and behaviors toward political out-groups. Can we design these AIs to consider democratic values such as mitigating partisan animosity as part of their objective functions? We introduce a method for translating established, vetted social scientific constructs into AI objective functions, which we term societal objective functions, and demonstrate the method with application to the political science construct of anti-democratic attitudes. Traditionally, we have lacked observable outcomes to use to train such models-however, the social sciences have developed survey instruments and qualitative codebooks for these constructs, and their precision facilitates translation into detailed prompts for large language models. We apply this method to create a democratic attitude model that estimates the extent to which a social media post promotes anti-democratic attitudes, and test this democratic attitude model across three studies. In Study 1, we first test the attitudinal and behavioral effectiveness of the intervention among US partisans (N=1,380) by manually annotating (alpha=.895) social media posts with anti-democratic attitude scores and testing several feed ranking conditions based on these scores. Removal (d=.20) and downranking feeds (d=.25) reduced participants' partisan animosity without compromising their experience and engagement. In Study 2, we scale up the manual labels by creating the democratic attitude model, finding strong agreement with manual labels (rho=.75). Finally, in Study 3, we replicate Study 1 using the democratic attitude model instead of manual labels to test its attitudinal and behavioral impact (N=558), and again find that the feed downranking using the societal objective function reduced partisan animosity (d=.25). This method presents a novel strategy to draw on social science theory and methods to mitigate societal harms in social media AIs.more » « less
-
Abstract Stratospheric aerosol injection (SAI) and gene drive organisms (GDOs) have been proposed as technological responses to complex entrenched environmental challenges. They also share several characteristics of emerging risks, including extensive uncertainties, systemic interdependencies, and risk profiles intertwined with societal contexts. This Perspective conducts a comparative analysis of the two technologies, and identifies ways in which their research and policy communities may learn from each other to inform future risk governance strategies. We find that SAI and GDOs share common features of aiming to improve or restore a public good, are characterized by numerous potential ecological, societal, and ethical risks associated with deep uncertainty, and are challenged by how best to coordinate behavior of different actors. Meanwhile, SAI and GDOs differ in their temporal and spatial mode of deployment, spread, degree and type of reversibility, and potential for environmental monitoring. Based on this analysis, we find the field of SAI may learn from GDOs by enhancing its international collaborations for governance and oversight, while the field of GDOs may learn from SAI by investing in research focused on economics and decision-modeling. Additionally, given the relatively early development stages of SAI and GDOs, there may be ample opportunities to learn from risk governance efforts of other emerging technologies, including the need for improved monitoring and incorporating aspects of responsible innovation in research and any deployment.more » « less
-
Abstract The overuse and expansion of artificial light at night (ALAN) has emerged from complex social, economic, and political factors, making it a societal problem that negatively impacts wildlife and people. We propose that a convergence research approach combining ecological forecasting with community engagement and public policy is needed to address this diverse societal problem. To begin this convergence research approach, we hosted a workshop to strengthen connections among key biodiversity‐oriented ALAN stakeholders and to better understand how stakeholder groups function across the United States through facilitated discussions. We have prioritized the input of stakeholders early in our research design by including them in the formulation of a national survey on public perceptions surrounding ALAN and received their input on existing ecological forecasting tools to improve those research products for their future use.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
