How do practitioners who develop consumer AI products
scope, motivate, and conduct privacy work? Respecting pri-
vacy is a key principle for developing ethical, human-centered
AI systems, but we cannot hope to better support practitioners
without answers to that question. We interviewed 35 industry
AI practitioners to bridge that gap. We found that practitioners
viewed privacy as actions taken against pre-defined intrusions
that can be exacerbated by the capabilities and requirements
of AI, but few were aware of AI-specific privacy intrusions
documented in prior literature. We found that their privacy
work was rigidly defined and situated, guided by compliance
with privacy regulations and policies, and generally demoti-
vated beyond meeting minimum requirements. Finally, we
found that the methods, tools, and resources they used in their
privacy work generally did not help address the unique pri-
vacy risks introduced or exacerbated by their use of AI in
their products. Collectively, these findings reveal the need and
opportunity to create tools, resources, and support structures
to improve practitioners’ awareness of AI-specific privacy
risks, motivations to do AI privacy work, and ability to ad-
dress privacy harms introduced or exacerbated by their use of
AI in consumer products.
more »
« less
This content will become publicly available on May 11, 2025
Deepfakes, Phrenology, Surveillance, and More! A Taxonomy of AI Privacy Risks
Privacy is a key principle for developing ethical AI technologies, but
how does including AI technologies in products and services change
privacy risks? We constructed a taxonomy of AI privacy risks by an-
alyzing 321 documented AI privacy incidents. We codifed how the
unique capabilities and requirements of AI technologies described
in those incidents generated new privacy risks, exacerbated known
ones, or otherwise did not meaningfully alter the risk. We present
12 high-level privacy risks that AI technologies either newly created
(e.g., exposure risks from deepfake pornography) or exacerbated
(e.g., surveillance risks from collecting training data). One upshot
of our work is that incorporating AI technologies into a product
can alter the privacy risks it entails. Yet, current approaches to
privacy-preserving AI/ML (e.g., federated learning, diferential pri-
vacy, checklists) only address a subset of the privacy risks arising
from the capabilities and data requirements of AI.
more »
« less
- Award ID(s):
- 2316768
- PAR ID:
- 10543401
- Publisher / Repository:
- ACM
- Date Published:
- ISBN:
- 9798400703300
- Page Range / eLocation ID:
- 1 to 19
- Format(s):
- Medium: X
- Location:
- Honolulu HI USA
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
There is a substantial and ever-growing corpus of evidence and literature exploring the impacts of Artificial intelligence (AI) technologies on society, politics, and humanity as a whole. A separate, parallel body of work has explored existential risks to humanity, including but not limited to that stemming from unaligned Artificial General Intelligence (AGI). In this paper, we problematise the notion that current and near-term artificial intelligence technologies have the potential to contribute to existential risk by acting as intermediate risk factors, and that this potential is not limited to the unaligned AGI scenario. We propose the hypothesis that certain already-documented effects of AI can act as existential risk factors, magnifying the likelihood of previously identified sources of existential risk. Moreover, future developments in the coming decade hold the potential to significantly exacerbate these risk factors, even in the absence of artificial general intelligence. Our main contribution is a (non-exhaustive) exposition of potential AI risk factors and the causal relationships between them, focusing on how AI can affect power dynamics and information security. This exposition demonstrates that there exist causal pathways from AI systems to existential risks that do not presuppose hypothetical future AI capabilities.more » « less
-
more » « less
Emerging Distributed AI systems are revolutionizing big data computing and data processing capabilities with growing economic and societal impact. However, recent studies have identified new attack surfaces and risks caused by security, privacy, and fairness issues in AI systems. In this paper, we review representative techniques, algorithms, and theoretical foundations for trustworthy distributed AI through robustness guarantee, privacy protection, and fairness awareness in distributed learning. We first provide a brief overview of alternative architectures for distributed learning, discuss inherent vulnerabilities for security, privacy, and fairness of AI algorithms in distributed learning, and analyze why these problems are present in distributed learning regardless of specific architectures. Then we provide a unique taxonomy of countermeasures for trustworthy distributed AI, covering (1) robustness to evasion attacks and irregular queries at inference, and robustness to poisoning attacks, Byzantine attacks, and irregular data distribution during training; (2) privacy protection during distributed learning and model inference at deployment; and (3) AI fairness and governance with respect to both data and models. We conclude with a discussion on open challenges and future research directions toward trustworthy distributed AI, such as the need for trustworthy AI policy guidelines, the AI responsibility-utility co-design, and incentives and compliance.
-
The recent advances in artificial intelligence (AI) have captivated the attention of many while raising the alarm among activists and policymakers. Although AI has its benefits, it has simultaneously contributed to increased polarization, the proliferation of mis/disinformation, online safety and privacy concerns, exacerbated mental health challenges, and questions about the ethical use of these technologies. The disproportionate influence of AI biases on Black, Latine, and Native communities and the continued exclusion of these communities from computing requires the adoption of new K-12 educational policies to ensure equitable access to AI education and equip students to be responsible and competent creators of technologies.more » « less
-
This survey paper provides an overview of the current state of Artificial Intelligence (AI) attacks and risks for AI security and privacy as artificial intelligence becomes more prevalent in various applications and services. The risks associated with AI attacks and security breaches are becoming increasingly apparent and cause many financial and social losses. This paper will categorize the different types of attacks on AI models, including adversarial attacks, model inversion attacks, poisoning attacks, data poisoning attacks, data extraction attacks, and membership inference attacks. The paper also emphasizes the importance of developing secure and robust AI models to ensure the privacy and security of sensitive data. Through a systematic literature review, this survey paper comprehensively analyzes the current state of AI attacks and risks for AI security and privacy and detection techniques.more » « less
