skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Auditing Practitioner Judgment for Algorithmic Fairness Implications
The development of Artificial Intelligence (AI) systems involves a significant level of judgment and decision making on the part of engineers and designers to ensure the safety, robustness, and ethical design of such systems. However, the kinds of judgments that practitioners employ while developing AI platforms are rarely foregrounded or examined to explore areas practitioners might need ethical support. In this short paper, we employ the concept of design judgment to foreground and examine the kinds of sensemaking software engineers use to inform their decisionmaking while developing AI systems. Relying on data generated from two exploratory observation studies of student software engineers, we connect the concept of fairness to the foregrounded judgments to implicate their potential algorithmic fairness impacts. Our findings surface some ways in which the design judgment of software engineers could adversely impact the downstream goal of ensuring fairness in AI systems. We discuss the implications of these findings in fostering positive innovation and enhancing fairness in AI systems, drawing attention to the need to provide ethical guidance, support, or intervention to practitioners as they engage in situated and contextual judgments while developing AI systems.  more » « less
Award ID(s):
1909714
PAR ID:
10429711
Author(s) / Creator(s):
;
Date Published:
Journal Name:
2023 IEEE International Symposium on Ethics in Engineering, Science, and Technology (ETHICS)
Page Range / eLocation ID:
01 to 05
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    The engineering disciplines are rigorous in their application of scientific principles, and these principles are the ones most directly addressed in undergraduate engineering classrooms. However, engineers are also called to make decisions that implicitly account for complex criteria, including the welfare of those who use or are impacted by the systems engineers design and the economic needs of their employers. As a result, engineering is an art that requires practitioners to routinely navigate difficult tradeoffs that require professional judgments. These judgments include economic, ethical, social, and value-based dimensions. These dimensions can be conflicting, increasing the complexity of practice and foregrounding the prominence of judgment. And often, these judgements need to be explained to colleagues, managers, and clients through a range of written documents. Yet little work to date has investigated the relationship between the writing engineering students do and the development of engineering judgement, particularly in terms of how these facets intersect in students developing engineering identities . Therefore, the overall goal of this project is to elucidate the interactions between how students’ identification with the engineering profession impacts the way they convey engineering judgments to different audiences. 
    more » « less
  2. How do software engineers identify and act on their ethical concerns? Past work examines how software practitioners navigate specific ethical principles such as “fairness”, but this narrows the scope of concerns to implementing pre-specified principles. In contrast, we report self-identified ethical concerns of 115 survey respondents and 21 interviewees across five continents and in non-profit, contractor, and non-tech firms.We enumerate their concerns – military, privacy, advertising, surveillance, and the scope of their concerns – from simple bugs to questioning their industry’s entire existence. We illustrate howattempts to resolve concerns are limited by factors such as personal precarity and organizational incentives. We discuss how even relatively powerful software engineers often lacked the power to resolve their ethical concerns. Our results suggest that ethics interventions must expand from helping practitioners merely identify issues to instead helping them build their (collective) power to resolve them, and that tech ethics discussions may consider broadening beyond foci on AI or Big Tech. 
    more » « less
  3. Engineering judgment is critical to both engineering education and engineering practice, and the ability to practice or participate in engineering judgment is often considered central to the formation of professional engineering identities. In practice, engineers must make difficult judgments that evaluate potentially competing objectives, ambiguity, uncertainty, incomplete information, and evolving technical knowledge. Nonetheless, while engineering judgment is implicit in engineering work and so central to identification with the profession, educators and practitioners have few actionable frameworks to employ when considering how to develop and assess this capacity in students. In this paper, we propose a theoretical framework designed to inform both educators and researchers that positions engineering judgment at the intersection of the cognitive dimensions of naturalistic decision-making, and discursive dimensions of identity. Our proposed theory positions engineering judgment not only as an individual capacity practiced by individual engineers alone but also as the capacity to position oneself within the discursive community so as to participate in the construction of engineering judgments among a group of professionals working together. Our theory draws on several strands of existing research to theorize a working framework for engineering judgment that considers the cognitive processes associated with making judgments and the inextricable discursive practices associated with negotiating those judgments in context. In constructing this theory, we seek to provide engineering education practitioners and researchers with a framework that can inform the design of assignments, curricula, or experiences that are intended to foster students’ participation in the development and practice of engineering judgment. 
    more » « less
  4. How do practitioners who develop consumer AI products scope, motivate, and conduct privacy work? Respecting pri- vacy is a key principle for developing ethical, human-centered AI systems, but we cannot hope to better support practitioners without answers to that question. We interviewed 35 industry AI practitioners to bridge that gap. We found that practitioners viewed privacy as actions taken against pre-defined intrusions that can be exacerbated by the capabilities and requirements of AI, but few were aware of AI-specific privacy intrusions documented in prior literature. We found that their privacy work was rigidly defined and situated, guided by compliance with privacy regulations and policies, and generally demoti- vated beyond meeting minimum requirements. Finally, we found that the methods, tools, and resources they used in their privacy work generally did not help address the unique pri- vacy risks introduced or exacerbated by their use of AI in their products. Collectively, these findings reveal the need and opportunity to create tools, resources, and support structures to improve practitioners’ awareness of AI-specific privacy risks, motivations to do AI privacy work, and ability to ad- dress privacy harms introduced or exacerbated by their use of AI in consumer products. 
    more » « less
  5. Traditional engineering courses typically approach teaching and problem solving by focusing on the physical dimensions of those problems without consideration of dynamic social and ethical dimensions. As such, projects can fail to consider community questions and concerns, broader impacts upon society, or otherwise result in inequitable outcomes. And, despite the fact that students in engineering receive training on the Professional Code of Ethics for Engineers, to which they are expected to adhere in practice, many students are unable to recognize and analyze real-life ethical challenges as they arise. Indeed, research has found that students are typically less engaged with ethics—defined as the awareness and judgment of microethics and macroethics, sensitivity to diversity, and interest in promoting organizational ethical culture—at the end of their engineering studies than they were at the beginning. As such, many studies have focused on developing and improving the curriculum surrounding ethics through, for instance, exposing students to ethics case studies. However, such ethics courses often present a narrow and simplified view of ethics that students may struggle to integrate with their broader experience as engineers. Thus, there is a critical need to unpack the complexity of ethical behavior amongst engineering students in order to determine how to better foster ethical judgment and behavior. Promoting ethical behavior among engineering students and developing a culture of ethical behavior within institutions have become goals of many engineering programs. Towards this goal, we present an overview of the current scholarship of engineering ethics and propose a theoretical framework of ethical behavior using a review of articles related to engineering ethics from 1990-2020. These articles were selected based upon their diversity of scope and methods until saturation was reached. A thematic analysis of articles was then performed using Nvivo. The review engages in theories across disciplines including philosophy, education and psychology. Preliminary results identify two major kinds of drivers of ethical behavior, namely individual level ethical behavior drivers (awareness of microethics, awareness of macroethics, implicit understanding, and explicit understanding) and institutional drivers (diversity and institutional ethical culture). In this paper, we present an overview and discussion of two drivers of ethical behavior at the individual level, namely awareness of microethics and awareness of macroethics, based on a review of 50 articles. Our results indicate that an awareness of both microethics and macroethics is essential in promoting ethical behavior amongst students. The review also points to a need to focus on increasing students’ awareness of macroethics. This research thus addresses the need, driven by existing scholarship, to identify a conceptual framework for explaining how ethical judgment and behavior in engineering can be further promoted. 
    more » « less