In widely used sociological descriptions of how accountability is structured through institutions, an “actor” (e.g., the developer) is accountable to a “forum” (e.g., regulatory agencies) empowered to pass judgements on and demand changes from the actor or enforce sanctions. However, questions about structuring accountability persist: why and how is a forum compelled to keep making demands of the actor when such demands are called for? To whom is a forum accountable in the performance of its responsibilities, and how can its practices and decisions be contested? In the context of algorithmic accountability, we contend that a robust accountability regime requires a triadic relationship, wherein the forum is also accountable to another entity: the public(s). Typically, as is the case with environmental impact assessments, public(s) make demands upon the forum's judgements and procedures through the courts, thereby establishing a minimum standard of due diligence. However, core challenges relating to: (1) lack of documentation, (2) difficulties in claiming standing, and (3) struggles around admissibility of expert evidence on and achieving consensus over the workings of algorithmic systems in adversarial proceedings prevent the public from approaching the courts when faced with algorithmic harms. In this paper, we demonstrate that the courts are the primary route—and the primary roadblock—in the pursuit of redress for algorithmic harms. Courts often find algorithmic harms non-cognizable and rarely require developers to address material claims of harm. To address the core challenges of taking algorithms to court, we develop a relational approach to algorithmic accountability that emphasizes not what the actors do nor the results of their actions, but rather how interlocking relationships of accountability are constituted in a triadic relationship between actors, forums, and public(s). As is the case in other regulatory domains, we believe that impact assessments (and similar accountability documentation) can provide the grounds for contestation between these parties, but only when that triad is structured such that the public(s) are able to cohere around shared experiences and interests, contest the outcomes of algorithmic systems that affect their lives, and make demands upon the other parties. Where courts now find algorithmic harms non-cognizable, an impact assessment regime can potentially create procedural rights to protect substantive rights of the public(s). This would require algorithmic accountability policies currently under consideration to provide the public(s) with adequate standing in courts, and opportunities to access and contest the actor's documentation and the forum's judgments.
more »
« less
Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Algorithmic impact assessments (AIAs) are an emergent form of accountability for entities that build and deploy automated decision-support systems. These are modeled after impact assessments in other domains. Our study of the history of impact assessments shows that "impacts" are an evaluative construct that enable institutions to identify and ameliorate harms experienced because of a policy decision or system. Every domain has different expectations and norms about what constitutes impacts and harms, how potential harms are rendered as the impacts of a particular undertaking, who is responsible for conducting that assessment, and who has the authority to act on the impact assessment to demand changes to that undertaking. By examining proposals for AIAs in relation to other domains, we find that there is a distinct risk of constructing algorithmic impacts as organizationally understandable metrics that are nonetheless inappropriately distant from the harms experienced by people, and which fall short of building the relationships required for effective accountability. To address this challenge of algorithmic accountability, and as impact assessments become a commonplace process for evaluating harms, the FAccT community should A) understand impacts as objects constructed for evaluative purposes, B) attempt to construct impacts as close as possible to actual harms, and C) recognize that accountability governance requires the input of various types of expertise and affected communities. We conclude with lessons for assembling cross-expertise consensus for the co-construction of impacts and to build robust accountability relationships.
more »
« less
- Award ID(s):
- 1704369
- NSF-PAR ID:
- 10283954
- Date Published:
- Journal Name:
- ACM Conference on Fairness, Accountability,and Transparency (FAccT ’21)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Algorithmic impact assessments (AIA) are increasingly being proposed as a mechanism for algorithmic accountability. These assessments are seen as potentially useful for anticipating, avoiding, and mitigating the negative consequences of algorithmic decision-making systems (ADS). At the same time, what an AIA would entail remains under-specified. While promising, AIAs raise as many questions as they answer. Choices about the methods, scope, and purpose of impact assessments structure the possible governance outcomes. Decisions about what type of effects count as an impact, when impacts are assessed, whose interests are considered, who is invited to participate, who conducts the assessment, the public availability of the assessment, and what the outputs of the assessment might be all shape the forms of accountability that AIA proponents seek to encourage. These considerations remain open, and will determine whether and how AIAs can function as a viable governance mechanism in the broader algorithmic accountability toolkit, especially with regard to furthering the public interest. Because AlAs are still an incipient governance strategy, approaching them as social constructions that do not require a single or universal approach offers a chance to produce interventions that emerge from careful deliberation.more » « less
-
null (Ed.)A quiet revolution is afoot in the field of law. Technical systems employing algorithms are shaping and displacing professional decision making, and they are disrupting and restructuring relationships between law firms, lawyers, and clients. Decision-support systems marketed to legal professionals to support e-discovery—generally referred to as “technology assisted review” (TAR)—increasingly rely on “predictive coding”: machine-learning techniques to classify and predict which of the voluminous electronic documents subject to litigation should be withheld or produced to the opposing side. These systems and the companies offering them are reshaping relationships between lawyers and clients, introducing new kinds of professionals into legal practice, altering the discovery process, and shaping how lawyers construct knowledge about their cases and professional obligations. In the midst of these shifting relationships—and the ways in which these systems are shaping the construction and presentation of knowledge—lawyers are grappling with their professional obligations, ethical duties, and what it means for the future of legal practice. Through in-depth, semi-structured interviews of experts in the e-discovery technology space—the technology company representatives who develop and sell such systems to law firms and the legal professionals who decide whether and how to use them in practice—we shed light on the organizational structures, professional rules and norms, and technical system properties that are shaping and being reshaped by predictive coding systems. Our findings show that AI-supported decision systems such as these are reconfiguring professional work practices. In particular, they highlight concerns about potential loss of professional agency and skill, limited understanding and thereby both over- and under reliance on decision-support systems, and confusion about responsibility and accountability as new kinds of technical professionals and technologies are brought into legal practice. The introduction of predictive coding systems and the new professional and organizational arrangements they are ushering into legal practice compound general concerns over the opacity of technical systems with specific concerns about encroachments on the construction of expert knowledge, liability frameworks, and the potential (mis)alignment of machine reasoning with professional logic and ethics. Based on our findings, we conclude that predictive coding tools—and likely other algorithmic systems lawyers use to construct knowledge and reason about legal practice— challenge the current model for evaluating whether and how tools are appropriate for legal practice. As tools become both more complex and more consequential, it is unreasonable to rely solely on legal professionals—judges, law firms, and lawyers—to determine which technologies are appropriate for use. The legal professionals we interviewed report relying on the evaluation and judgment of a range of new technical experts within law firms and, increasingly, third-party vendors and their technical experts. This system for choosing technical systems upon which lawyers rely to make professional decisions—e.g., whether documents are responsive, or whether the standard of proportionality has been met—is no longer sufficient. As the tools of medicine are reviewed by appropriate experts before they are put out for consideration and adoption by medical professionals, we argue that the legal profession must develop new processes for determining which algorithmic tools are fit to support lawyers’ decision making. Relatedly, because predictive coding systems are used to produce lawyers’ professional judgment, we argue they must be designed for contestability— providing greater transparency, interaction, and configurability around embedded choices to ensure decisions about how to embed core professional judgments, such as relevance and proportionality, remain salient and demand engagement from lawyers, not just their technical experts.more » « less
-
Assessing the ecological and economic impacts of non-native species is crucial to providing managers and policymakers with the information necessary to respond effectively. Most non-native species have minimal impacts on the environment in which they are introduced, but a small fraction are highly deleterious. The definition of ‘damaging’ or ‘high-impact’ varies based on the factors determined to be valuable by an individual or group, but interpretations of whether non-native species meet particular definitions can be influenced by the interpreter’s bias or level of expertise, or lack of group consensus. Uncertainty or disagreement about an impact classification may delay or otherwise adversely affect policymaking on management strategies. One way to prevent these issues would be to have a detailed, nine-point impact scale that would leave little room for interpretation and then divide the scale into agreed upon categories, such as low, medium, and high impact. Following a previously conducted, exhaustive search regarding non-native, conifer-specialist insects, the authors independently read the same sources and scored the impact of 41 conifer-specialist insects to determine if any variation among assessors existed when using a detailed impact scale. Each of the authors, who were selected to participate in the working group associated with this study because of their diverse backgrounds, also provided their level of expertise and uncertainty for each insect evaluated. We observed 85% congruence in impact rating among assessors, with 27% of the insects having perfect inter-rater agreement. Variance in assessment peaked in insects with a moderate impact level, perhaps due to ambiguous information or prior assessor perceptions of these specific insect species. The authors also participated in a joint fact-finding discussion of two insects with the most divergent impact scores to isolate potential sources of variation in assessor impact scores. We identified four themes that could be experienced by impact assessors: ambiguous information, discounted details, observed versus potential impact, and prior knowledge. To improve consistency in impact decision-making, we encourage groups to establish a detailed scale that would allow all observed and published impacts to fall under a particular score, provide clear, reproducible guidelines and training, and use consensus-building techniques when necessary.more » « less
-
more » « less
Objective Symptomatic head trauma associated with American‐style football (ASF) has been linked to brain pathology, along with physical and mental distress in later life. However, the longer‐term effects of such trauma on objective metrics of cognitive–motor function remain poorly understood. We hypothesized that ASF‐related symptomatic head trauma would predict worse gait performance, particularly during dual task conditions (ie, walking while performing an additional cognitive task), in later life.
Methods Sixty‐six retired professional ASF players aged 29 to 75 years completed a health and wellness questionnaire. They also completed a validated smartphone‐based assessment in their own homes, during which gait was monitored while they walked normally and while they performed a verbalized serial‐subtraction cognitive task.
Results Participants who reported more symptomatic head trauma, defined as the total number of impacts to the head or neck followed by concussion‐related symptoms, exhibited greater dual task cost (ie, percentage increase) to stride time variability (ie, the coefficient of variation of mean stride time). Those who reported ≥1 hit followed by loss of consciousness, compared to those who did not, also exhibited greater dual task costs to this metric. Relationships between reported trauma and dual task costs were independent of age, body mass index, National Football League career duration, and history of musculoskeletal surgery. Symptomatic head trauma was not correlated with average stride times in either walking condition.
Interpretation Remote, smartphone‐based assessments of dual task walking may be utilized to capture meaningful data sensitive to the long‐term impact of symptomatic head trauma in former professional ASF players and other contact sport athletes. ANN NEUROL 2020;87:75–83
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
