skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Reflective Design for Informal Participatory Algorithm Auditing: A Case Study with Emotion AI
This paper suggests how reflective design can aid informal participatory algorithm auditing. Drawing from reflective design, we designed a simple web-form probe to invite critical reflection on Emotion AI, ethically controversial techniques predicting individuals’ emotions. Participants engaged the probe throughout their daily lives for about a week. Then, we interviewed participants about their experiences and reflections. Our findings surface themes around participants’ (i) critiques of Emotion AI, (ii) factors contributing to inaccuracy, and (iii) patterns of miscategorization. Our discussion contributes (1) recommendations for Emotion AI and (2) how reflective design may offer considerations to inform algorithm auditing. Overall, our paper suggests ways critically-oriented design research can engage AI ethics through informal, participatory, exploratory algorithm auditing.  more » « less
Award ID(s):
2335974
PAR ID:
10595799
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
ACM
Date Published:
ISBN:
9798400709661
Page Range / eLocation ID:
1 to 17
Subject(s) / Keyword(s):
Emotion AI reflective design algorithm audit
Format(s):
Medium: X
Location:
Uppsala Sweden
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, Proceedings of the ACM on Human-Computer Interaction (CSCW))
    This study investigates how high school-aged youth engage in algorithm auditing to identify and understand biases in artificial intelligence and machine learning (AI/ML) tools they encounter daily. With AI/ML technologies being increasingly integrated into young people’s lives, there is an urgent need to equip teenagers with AI literacies that build both technical knowledge and awareness of social impacts. Algorithm audits (also called AI audits) have traditionally been employed by experts to assess potential harmful biases, but recent research suggests that non-expert users can also participate productively in auditing. We conducted a two-week participatory design workshop with 14 teenagers (ages 14–15), where they audited the generative AI model behind TikTok’s Effect House, a tool for creating interactive TikTok filters. We present a case study describing how teenagers approached the audit, from deciding what to audit to analyzing data using diverse strategies and communicating their results. Our findings show that participants were engaged and creative throughout the activities, independently raising and exploring new considerations, such as age-related biases, that are uncommon in professional audits. We drew on our expertise in algorithm auditing to triangulate their findings as a way to examine if the workshop supported participants to reach coherent conclusions in their audit. Although the resulting number of changes in race, gender, and age representation uncovered by the teens were slightly different from ours, we reached similar conclusions. This study highlights the potential for auditing to inspire learning activities to foster AI literacies, empower teenagers to critically examine AI systems, and contribute fresh perspectives to the study of algorithmic harms. 
    more » « less
  2. ; ; ; ; ; (, Proceedings of the ACM on humancomputer interaction)
    This study investigates how high school-aged youth engage in algorithm auditing to identify and understand biases in artificial intelligence and machine learning (AI/ML) tools they encounter daily. With AI/ML technologies being increasingly integrated into young people’s lives, there is an urgent need to equip teenagers with AI literacies that build both technical knowledge and awareness of social impacts. Algorithm audits (also called AI audits) have traditionally been employed by experts to assess potential harmful biases, but recent research suggests that non-expert users can also participate productively in auditing. We conducted a two-week participatory design workshop with 14 teenagers (ages 14–15), where they audited the generative AI model behind TikTok’s Effect House, a tool for creating interactive TikTok filters. We present a case study describing how teenagers approached the audit, from deciding what to audit to analyzing data using diverse strategies and communicating their results. Our findings show that participants were engaged and creative throughout the activities, independently raising and exploring new considerations, such as age-related biases, that are uncommon in professional audits. We drew on our expertise in algorithm auditing to triangulate their findings as a way to examine if the workshop supported participants to reach coherent conclusions in their audit. Although the resulting number of changes in race, gender, and age representation uncovered by the teens were slightly different from ours, we reached similar conclusions. This study highlights the potential for auditing to inspire learning activities to foster AI literacies, empower teenagers to critically examine AI systems, and contribute fresh perspectives to the study of algorithmic harms. 
    more » « less
  3. ; ; ; ; (, ISLS)
    Hoadley, C; Wang, C (Ed.)
    While there is widespread interest in supporting young people to critically evaluate machine learning-powered systems, there is little research on how we can support them in inquiring about how these systems work and what their limitations and implications may be. Outside of K-12 education, an effective strategy in evaluating black-boxed systems is algorithm auditing—a method for understanding algorithmic systems’ opaque inner workings and external impacts from the outside in. In this paper, we review how expert researchers conduct algorithm audits and how end users engage in auditing practices to propose five steps that, when incorporated into learning activities, can support young people in auditing algorithms. We present a case study of a team of teenagers engaging with each step during an out-of-school workshop in which they audited peer-designed generative AI TikTok filters. We discuss the kind of scaffolds we provided to support youth in algorithm auditing and directions and challenges for integrating algorithm auditing into classroom activities. This paper contributes: (a) a conceptualization of five steps to scaffold algorithm auditing learning activities, and (b) examples of how youth engaged with each step during our pilot study. 
    more » « less
  4. ; ; ; ; ; (, ISLS)
    Seitamaa_Hakkarainen, P; Kangas, K (Ed.)
    Today’s youth have extensive experience interacting with artificial intelligence and machine learning applications on popular social media platforms, putting youth in a unique position to examine, evaluate, and even challenge these applications. Algorithm auditing is a promising candidate for connecting youth’s everyday practices in using AI applications with more formal scientific literacies (i.e., syncretic designs). In this paper, we analyze high school youth participants’ everyday algorithm auditing practices when interacting with generative AI filters on TikTok, revealing thorough and extensive examinations, with youth rapidly testing filters with sophisticated camera variations and facial manipulations to identify filter limitations. In the discussion, we address how these findings can provide a foundation for developing designs that bring together everyday and more formal algorithm auditing. 
    more » « less
  5. ; ; ; (, Proceedings of Interaction Design and Children)
    Participatory design is an essential design strategy for creating artifacts and experiences that reflect the voices of the population being designed for and with. The participatory design process can serve not only to research resulting artifacts but also as an empowering activity for those who participate. This paper explores how participatory design can serve as a context for young participants to enact and voice their emerging identities and reveals how different participatory design activities have unique affordances for supporting this identity enactment. Focusing on a group of 12 and 13-year-old African American girls, this paper presents a case study showing how participatory design activities served as venues for the girls to reflect characteristics of their current identities, project future identities, and apply aspects of their identities to shape materials for others. In doing so, we contribute a case study showing how participatory design allows participants to enact their identities, helping researchers gain insight into characteristics of those they are designing with and for. This paper advances our understanding of participatory design as a design approach for youth, especially as it relates to issues of broadening participation, identity, and equity. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email