More Like this

1. Conventionalization of Iconic Handshape Preferences in FamilyHomesign Systems

   Quam, M.; Brentari, D.; Coppola, M. (June 2022, Languages)

   Sandler, W.; Aronoff, M.; Padden, C. (Ed.)

   Variation in the linguistic use of handshapes exists across sign languages, but it is unclear how these iconic handshape preferences arise and become conventionalized. In order to understand the factors that shape such handshape preferences in the earliest stages of language emergence, we examined communication within family homesign systems. Homesigners are deaf individuals who have not acquired a signed or spoken language and who innovate unique gesture systems to communicate with hearing friends and family (“communication partners”). We analyzed how characteristics of participants and stimulus items influence handshape preferences and conventionalization. Participants included 11 deaf homesigners, 24 hearing communication partners (CPs), and 8 hearing non-signing adults from Nicaragua. Participants were asked to label items using gestures or signs. The handshape type (Handling, Object, or combined Handling+Object) was then coded. The participants and groups showed variability in iconic handshape preferences. Adult homesigners’ families demonstrated more conventionalization than did child homesigners’ families. Adult homesigners also used a combined Handling + Object form more than other participants. Younger CPs and those with fewer years of experience using a homesign system showed greater conventionalization. Items that elicited a reliable handshape preference were more likely to elicit Handling rather than Object handshapes. These findings suggest that similarity in terms of handshape type varies even within families, including hearing gesturers in the same culture. Although adult homesigners’ families were more conventionalized than child homesigners’ families, full conventionalization of these handshape preferences do not seem to appear reliably within two to three decades of use in a family when only one deaf homesigner uses it as a primary system. 

   more » « less
2. Customizing Generated Signs and Voices of AI Avatars: Deaf-Centric Mixed-Reality Design for Deaf-Hearing Communication

   https://doi.org/10.1145/3710953  

   Chen, Si; Cheng, Haocong; Su, Suzy; Patterson, Stephanie; Kushalnagar, Raja; Huang, Yun; Wang, Qi (May 2025, Proceedings of the ACM on Human-Computer Interaction)

   This study investigates innovative interaction designs for communication and collaborative learning between learners of mixed hearing and signing abilities, leveraging advancements in mixed reality technologies like Apple Vision Pro and generative AI for animated avatars. Adopting a participatory design approach, we engaged 15 d/Deaf and hard of hearing (DHH) students to brainstorm ideas for an AI avatar with interpreting ability (sign language to English and English to sign language) that would facilitate their face-to-face communication with hearing peers. Participants envisioned the AI avatars to address some issues with human interpreters, such as lack of availability, and provide affordable options to expensive personalized interpreting services. Our findings indicate a range of preferences for integrating the AI avatars with actual human figures of both DHH and hearing communication partners. The participants highlighted the importance of having control over customizing the AI avatar, such as AI-generated signs, voices, facial expressions, and their synchronization for enhanced emotional display in communication. Based on our findings, we propose a suite of design recommendations that balance respecting sign language norms with adherence to hearing social norms. Our study offers insights into improving the authenticity of generative AI in scenarios involving specific and sometimes unfamiliar social norms. 

   more » « less
3. "We do use it, but not how hearing people think": How the Deaf and Hard of Hearing Community Uses Large Language Model Tools

   https://doi.org/10.1145/3706599.3719785  

   Huffman, Shuxu; Chen, Si; Mack, Kelly Avery; Su, Haotian; Wang, Qi; Kushalnagar, Raja (April 2025, ACM)

   Generative AI tools, particularly those utilizing large language models (LLMs), are increasingly used in everyday contexts. While these tools enhance productivity and accessibility, little is known about how Deaf and Hard of Hearing (DHH) individuals engage with them or the challenges they face when using them. This paper presents a mixed-method study exploring how the DHH community uses Text AI tools like ChatGPT to reduce communication barriers and enhance information access. We surveyed 80 DHH participants and conducted interviews with 9 participants. Our findings reveal important benefits, such as eased communication and bridging Deaf and hearing cultures, alongside challenges like lack of American Sign Language (ASL) support and Deaf cultural understanding. We highlight unique usage patterns, propose inclusive design recommendations, and outline future research directions to improve Text AI accessibility for the DHH community. 

   more » « less
4. Teleconference Accessibility and Guidelines for Deaf and Hard of Hearing Users

   https://doi.org/10.1145/3373625.3417299  

   Kushalnagar, Raja S.; Vogler, Christian (October 2020, ASSETS '20: The 22nd International ACM SIGACCESS Conference on Computers and Accessibility)

   null (Ed.)

   In this experience report, we describe the accessibility challenges that deaf and hard of hearing users face in teleconferences, based on both our first-hand participation in meetings, and as User Interface and Experience experts. Teleconferencing poses new accessibility challenges compared to face-to-face communication because of limited social, emotional, and haptic feedback. Above all, teleconferencing participants and organizers need to be flexible, because deaf or hard of hearing people have diverse communication preferences. We explain what recurring problems users experience, where current teleconferencing software falls short, and how to address these shortcomings. We offer specific recommendations for best practices and the experiential reasons behind them. 

   more » « less
5. Jod: Examining Design and Implementation of a Videoconferencing Platform for Mixed Hearing Groups

   https://doi.org/10.1145/3597638.3608382  

   Mittal, Anant; Gupta, Meghna; Poddar, Roshni; Naik, Tarini; Kuppuraj, Seethalakshmi; Fogarty, James; Kumar, Pratyush; Jain, Mohit (October 2023, Proceedings of the ACM Conference on Computers and Accessibility)

   Videoconferencing usage has surged in recent years, but current platforms present significant accessibility barriers for the 430 million d/Deaf or hard of hearing people worldwide. Informed by prior work examining accessibility barriers in current videoconferencing platforms, we designed and developed Jod, a videoconferencing platform to facilitate communication in mixed hearing groups. Key features include support for customizing visual layouts and a notification system to request attention and influence behavior. Using Jod, we conducted six mixed hearing group sessions with 34 participants, including 18 d/Deaf or hard of hearing participants, 10 hearing participants, and 6 sign language interpreters. We found participants engaged in visual layout rearrangements based on their hearing ability and dynamically adapted to the changing group communication context, and that notifications were useful but raised a need for designs to cause fewer interruptions. We provide insights for future videoconferencing designs and conclude with recommendations for conducting mixed hearing studies. 

   more » « less