An official website of the United States government
Generative AI tools, particularly those utilizing large language models (LLMs), are increasingly used in everyday contexts. While these tools enhance productivity and accessibility, little is known about how Deaf and Hard of Hearing (DHH) individuals engage with them or the challenges they face when using them. This paper presents a mixed-method study exploring how the DHH community uses Text AI tools like ChatGPT to reduce communication barriers and enhance information access. We surveyed 80 DHH participants and conducted interviews with 9 participants. Our findings reveal important benefits, such as eased communication and bridging Deaf and hearing cultures, alongside challenges like lack of American Sign Language (ASL) support and Deaf cultural understanding. We highlight unique usage patterns, propose inclusive design recommendations, and outline future research directions to improve Text AI accessibility for the DHH community.
more »
« less
Towards Inclusive Video Commenting: Introducing Signmaku for the Deaf and Hard-of-Hearing
Previous research underscored the potential of danmaku–a text-based commenting feature on videos–in engaging hearing audiences. Yet, for many Deaf and hard-of-hearing (DHH) individuals, American Sign Language (ASL) takes precedence over English. To improve inclusivity, we introduce “Signmaku,” a new commenting mechanism that uses ASL, serving as a sign language counterpart to danmaku. Through a need-finding study (N=12) and a within-subject experiment (N=20), we evaluated three design styles: real human faces, cartoon-like figures, and robotic representations. The results showed that cartoon-like signmaku not only entertained but also encouraged participants to create and share ASL comments, with fewer privacy concerns compared to the other designs. Conversely, the robotic representations faced challenges in accurately depicting hand movements and facial expressions, resulting in higher cognitive demands on users. Signmaku featuring real human faces elicited the lowest cognitive load and was the most comprehensible among all three types. Our findings offered novel design implications for leveraging generative AI to create signmaku comments, enriching co-learning experiences for DHH individuals.
more »
« less
- PAR ID:
- 10538322
- Publisher / Repository:
- ACM
- Date Published:
- Page Range / eLocation ID:
- 1 to 18
- Subject(s) / Keyword(s):
- DHH, Social Interactions, Danmaku, Signmaku
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Without a commonly accepted writing system for American Sign Language (ASL), Deaf or Hard of Hearing (DHH) ASL signers who wish to express opinions or ask questions online must post a video of their signing, if they prefer not to use written English, a language in which they may feel less proficient. Since the face conveys essential linguistic meaning, the face cannot simply be removed from the video in order to preserve anonymity. Thus, DHH ASL signers cannot easily discuss sensitive, personal, or controversial topics in their primary language, limiting engagement in online debate or inquiries about health or legal issues. We explored several recent attempts to address this problem through development of “face swap” technologies to automatically disguise the face in videos while preserving essential facial expressions and natural human appearance. We presented several prototypes to DHH ASL signers (N=16) and examined their interests in and requirements for such technology. After viewing transformed videos of other signers and of themselves, participants evaluated the understandability, naturalness of appearance, and degree of anonymity protection of these technologies. Our study revealed users’ perception of key trade-offs among these three dimensions, factors that contribute to each, and their views on transformation options enabled by this technology, for use in various contexts. Our findings guide future designers of this technology and inform selection of applications and design features.more » « less
-
Without a commonly accepted writing system for American Sign Language (ASL), Deaf or Hard of Hearing (DHH) ASL signers who wish to express opinions or ask questions online must post a video of their signing, if they prefer not to use written English, a language in which they may feel less proficient. Since the face conveys essential linguistic meaning, the face cannot simply be removed from the video in order to preserve anonymity. Thus, DHH ASL signers cannot easily discuss sensitive, personal, or controversial topics in their primary language, limiting engagement in online debate or inquiries about health or legal issues. We explored several recent attempts to address this problem through development of “face swap” technologies to automatically disguise the face in videos while preserving essential facial expressions and natural human appearance. We presented several prototypes to DHH ASL signers (N=16) and examined their interests in and requirements for such technology. After viewing transformed videos of other signers and of themselves, participants evaluated the understandability, naturalness of appearance, and degree of anonymity protection of these technologies. Our study revealed users’ perception of key trade-offs among these three dimensions, factors that contribute to each, and their views on transformation options enabled by this technology, for use in various contexts. Our findings guide future designers of this technology and inform selection of applications and design features.more » « less
-
Deaf and hard-of-hearing (DHH) children are often reported to show deficits on working memory (WM) tasks. These deficits are often characterized as contributing to their struggles to acquire spoken language. Here we report a longitudinal study of a large (N = 103) sample of DHH children who acquired American Sign Language (ASL) as their first language. Using an n-back working memory task, we show significant growth in WM performance across the 7–13-year-old age range. Furthermore, we show that children with early access to ASL from their DHH parents demonstrate faster WM growth and that this group difference is mediated by ASL receptive skills. The data suggest the important role of early access to perceivable natural language in promoting typical WM growth during the middle school years. We conclude that the acquisition of a natural visual–gestural language is sufficient to support the development of WM in DHH children. Further research is required to determine how the timing and quality of ASL exposure may play a role, or whether the effects are driven by acquisition-related corollaries, such as parent–child interactions and maternal stress.more » « less
-
Abstract The auditory scaffolding hypothesis states that early experience with sound underpins the development of domain-general sequence processing abilities, supported by studies observing impaired sequence processing in deaf or hard-of-hearing (DHH) children. To test this hypothesis, we administered a sequence processing task to 77 DHH children who use American Sign Language (ASL) and 23 hearing monolingual children aged 7–12 years and found no performance difference between them after controlling for age and nonverbal intelligence. Additionally, neither spoken language comprehension scores nor hearing loss levels predicted sequence processing scores in the DHH group, whereas ASL comprehension scores did. Our results do not indicate sequence processing deficits in DHH children and do not support the auditory scaffolding hypothesis; instead, these findings suggest that factors related to experience with and/or proficiency in an accessible language during development may be more important determinants of sequence processing abilities.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3613904.3642287
