skip to main content

Explore Research Products in the NSF-PAR

Title: ALLOCATING RESPONSIBILITY IN CONTENT MODERATION: A FUNCTIONAL FRAMEWORK
This Article develops a framework for both assessing and designing content moderation systems consistent with public values. It argues that moderation should not be understood as a single function, but as a set of subfunctions common to all content governance regimes. By identifying the particular values implicated by each of these subfunctions, it explores the appropriate ways the constituent tasks might best be allocated-specifically to which actors (public or private, human or technological) they might be assigned, and what constraints or processes might be required in their performance. This analysis can facilitate the evaluation and design of content moderation systems to ensure the capacity and competencies necessary for legitimate, distributed systems of content governance. Through a combination of methods, legal schemes delegate at least a portion of the responsibility for governing online expression to private actors. Sometimes, statutory schemes assign regulatory tasks explicitly. In others, this delegation often occurs implicitly, with little guidance as to how the treatment of content should be structured. In the law's shadow, online platforms are largely given free rein to configure the governance of expression. Legal scholarship has surfaced important concerns about the private sector's role in content governance. In response, private platforms engaged in content moderation have adopted structures that mimic public governance forms. Yet, we largely lack the means to measure whether these forms are substantive, effectively infusing public values into the content moderation process, or merely symbolic artifice designed to deflect much needed public scrutiny. This Article's proposed framework addresses that gap in two ways. First, the framework considers together all manner of legal regimes that induce platforms to engage in the function of content moderation. Second, it focuses on the shared set of specific tasks, or subfunctions, involved in the content moderation function across these regimes. Examining a broad range of content moderation regimes together highlights the existence of distinct common tasks and decision points that together constitute the content moderation function. Focusing on this shared set of subfunctions highlights the different values implicated by each and the way they can be "handed off' to human and technical actors to perform in different ways with varying normative and political implications. This Article identifies four key content moderation subfunctions: (1) definition of policies, (2) identification of potentially covered content, (3) application of policies to specific cases, and (4) resolution of those cases. Using these four subfunctions supports a rigorous analysis of how to leverage the capacities and competencies of government and private parties throughout the content moderation process. Such attention also highlights how the exercise of that power can be constrained-either by requiring the use of particular decision-making processes or through limits on the use of automation-in ways that further address normative concerns. Dissecting the allocation of subfunctions in various content moderation regimes reveals the distinct ethical and political questions that arise in alternate configurations. Specifically, it offers a way to think about four key questions: (1) what values are most at issue regarding each subfunction; (2) which activities might be more appropriate to delegate to particular public or private actors; (3) which constraints must be attached to the delegation of each subfunction; and (4) where can investments in shared content moderation infrastructures support relevant values? The functional framework thus provides a means for both evaluating the symbolic legal forms that firms have constructed in service of content moderation and for designing processes that better reflect public values.  more » « less
Award ID(s):
1650589
NSF-PAR ID:
10393008
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Berkeley technology law journal
Volume:
36
Issue:
3
ISSN:
1086-3818
Page Range / eLocation ID:
1091-1172
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ; ; ; et al ( , Proceedings of the National Academy of Sciences)

    The speed and uncertainty of environmental change in the Anthropocene challenge the capacity of coevolving social–ecological–technological systems (SETs) to adapt or transform to these changes. Formal government and legal structures further constrain the adaptive capacity of our SETs. However, new, self-organized forms of adaptive governance are emerging at multiple scales in natural resource-based SETs. Adaptive governance involves the private and public sectors as well as formal and informal institutions, self-organized to fill governance gaps in the traditional roles of states. While new governance forms are emerging, they are not yet doing so rapidly enough to match the pace of environmental change. Furthermore, they do not yet possess the legitimacy or capacity needed to address disparities between the winners and losers from change. These emergent forms of adaptive governance appear to be particularly effective in managing complexity. We explore governance and SETs as coevolving complex systems, focusing on legal systems to understand the potential pathways and obstacles to equitable adaptation. We explore how governments may facilitate the emergence of adaptive governance and promote legitimacy in both the process of governance despite the involvement of nonstate actors, and its adherence to democratic values of equity and justice. To manage the contextual nature of the results of change in complex systems, we propose the establishment of long-term study initiatives for the coproduction of knowledge, to accelerate learning and synergize interactions between science and governance and to foster public science and epistemic communities dedicated to navigating transitions to more just, sustainable, and resilient futures.

     
    more » « less
  2. ( , Berkeley technology law journal)
    null (Ed.)
    At every level of government, officials contract for technical systems that employ machine learning-systems that perform tasks without using explicit instructions, relying on patterns and inference instead. These systems frequently displace discretion previously exercised by policymakers or individual front-end government employees with an opaque logic that bears no resemblance to the reasoning processes of agency personnel. However, because agencies acquire these systems through government procurement processes, they and the public have little input into-or even knowledge about-their design or how well that design aligns with public goals and values. This Article explains the ways that the decisions about goals, values, risk, and certainty, along with the elimination of case-by-case discretion, inherent in machine-learning system design create policies-not just once when they are designed, but over time as they adapt and change. When the adoption of these systems is governed by procurement, the policies they embed receive little or no agency or outside expertise beyond that provided by the vendor. Design decisions are left to private third-party developers. There is no public participation, no reasoned deliberation, and no factual record, which abdicates Government responsibility for policymaking. This Article then argues for a move from a procurement mindset to policymaking mindset. When policy decisions are made through system design, processes suitable for substantive administrative determinations should be used: processes that foster deliberation reflecting both technocratic demands for reason and rationality informed by expertise, and democratic demands for public participation and political accountability. Specifically, the Article proposes administrative law as the framework to guide the adoption of machine learning governance, describing specific ways that the policy choices embedded in machine learning system design fail the prohibition against arbitrary and capricious agency actions absent a reasoned decision-making process that both enlists the expertise necessary for reasoned deliberation about, and justification for, such choices, and makes visible the political choices being made. Finally, this Article sketches models for machine-learning adoption processes that satisfy the prohibition against arbitrary and capricious agency actions. It explores processes by which agencies might garner technical expertise and overcome problems of system opacity, satisfying administrative law's technocratic demand for reasoned expert deliberation. It further proposes both institutional and engineering design solutions to the challenge of policymaking opacity, offering process paradigms to ensure the "political visibility" required for public input and political oversight. In doing so, it also proposes the importance of using "contestable design"-design that exposes value-laden features and parameters and provides for iterative human involvement in system evolution and deployment. Together, these institutional and design approaches further both administrative law's technocratic and democratic mandates. 
    more » « less
  3. ; ; ( , International Journal of Urban and Regional Research)
    Abstract

    As municipalities across the global North highlight urban agriculture as a marker of their ‘greenness’, how can we best understand how the spaces and practices of urban food production are governed? This article develops an analysis of urban agriculture as a complex site of governance in which numerous interests engage. We underscore the politics of governance, through which some actors resist the imposition of a narrowly normative and exclusive notion of urban agriculture and against which they envision and enact alternatives. The article contributes to efforts to transcend the often dichotomous framing of urban agriculture as radical or neoliberal, formal or informal, political or post‐political by employing ‘everyday governance’ and ‘everyday resistance’ as lenses through which to focus on the prosaic practices of engaging with, pushing back against, and stepping beyond the imposition of hegemonic models of urban agriculture. We argue that the co‐constitutive, ‘braided’ nature of urban agricultural governance is revealed through attention to the manifold forms of negotiation and resistance to formal urban agricultural governance. Moreover, our perspective highlights the ways that some practitioners are excluded by, challenge, or re‐vision formal definitions of urban agriculture. We draw on the cases of Portland, OR and Vancouver, BC to illustrate our argument.

     
    more » « less
  4. ; ; ; ; ; ( , 2021 IEEE Visualization Conference (VIS))
    Modern social media platforms like Twitch, YouTube, etc., embody an open space for content creation and consumption. However, an unintended consequence of such content democratization is the proliferation of toxicity and abuse that content creators get subjected to. Commercial and volunteer content moderators play an indispensable role in identifying bad actors and minimizing the scale and degree of harmful content. Moderation tasks are often laborious, complex, and even if semi-automated, they involve high-consequence human decisions that affect the safety and popular perception of the platforms. In this paper, through an interdisciplinary collaboration among researchers from social science, human-computer interaction, and visualization, we present a systematic understanding of how visual analytics can help in human-in-the-loop content moderation. We contribute a characterization of the data-driven problems and needs for proactive moderation and present a mapping between the needs and visual analytic tasks through a task abstraction framework. We discuss how the task abstraction framework can be used for transparent moderation, design interventions for moderators’ well-being, and ultimately, for creating futuristic human-machine interfaces for data-driven content moderation. 
    more » « less
  5. ; ; ; ; ; ( , Environmental Research Letters)
    Abstract

    Collaborative, or participatory governance is an increasingly common means of addressing natural resource issues, especially in the American West where patchworks of public, private, and tribal interests characterize the region’s resources. In this context, unlikely alliances, or partnerships among diverse actors who have historically been at odds, have a growing potential to shape social and ecological outcomes, for better or worse. While these unlikely alliances have received greater attention in recent years, relatively little research has worked to synthesize the concept across diverse contexts and disciplines. Based on a review of the literature on unlikely alliances in natural resource governance, we develop a framework that synthesizes the individual motivations and contextual factors that influence their formation, as well as the social and ecological outcomes that they create. We use this framework to analyze six illustrative cases of unlikely alliances. Our analysis of these cases suggests that unlikely alliances in the American West are likely to arise in the presence of a crisis, when appropriate leadership is present, when some of the actors have interacted effectively in the past, and when actors need to pool resources. The cases also illustrate some common outcomes, including environmental improvement, transformation of social networks, policy change, and shifts in power relationships. We discuss the implications of unlikely alliances for the social-ecological future of the American West. Our paper highlights the role of unlikely alliances in shaping patterns of natural resource governance, and provides a focus for further research in this realm.

     
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email