skip to main content


Title: AI-Driven Sign Language Interpretation for Nigerian Children at Home

As many as three million school age children between the ages of 5 and 14 years, live with severe to profound hearing loss in Nigeria. Many of these Deaf or Hard of Hearing (DHH) children developed their hearing loss later in life, non-congenitally, hence their parents are hearing. While their teachers in the Deaf schools they attend can often communicate effectively with them in dialects of American Sign Language (ASL), the unofficial sign lingua franca in Nigeria, communication at home with other family members is challenging and sometimes non-existent. This results in adverse social consequences including stigmatization, for the students.With the recent successes of AI in natural language understanding, the goal of automated sign language understanding is becoming more realistic using neural deep learning technologies. To this effect, the proposed project aims at co-designing and developing an ongoing AI-driven two-way sign language interpretation tool that can be deployed in homes, to improve language accessibility and communication between the DHH students and other family members. This ensures inclusive and equitable social interactions and can promote lifelong learning opportunities for them outside of the school environment.

 
more » « less
Award ID(s):
2223507
PAR ID:
10493843
Author(s) / Creator(s):
; ; ; ;
Publisher / Repository:
International Joint Conferences on Artificial Intelligence Organization
Date Published:
Journal Name:
International Joint Conferences on Artificial Intelligence
ISBN:
978-1-956792-03-4
Page Range / eLocation ID:
6395 to 6404
Format(s):
Medium: X
Location:
Macau, SAR China
Sponsoring Org:
National Science Foundation
More Like this
  1. Deaf and Hard of Hearing (DHH) children need to master at least one language (spoken or signed) to reach their full potential. Providing access to a natural sign language supports this goal. Despite evidence that natural sign languages are beneficial to DHH children, many researchers and practitioners advise families to focus exclusively on spoken language. We critique the Pediatrics article ‘Early Sign Language Exposure and Cochlear Implants’ (Geers et al., 2017) as an example of research that makes unsupported claims against the inclusion of natural sign languages. We refute claims that (1) there are harmful effects of sign language and (2) that listening and spoken language are necessary for optimal development of deaf children. While practical challenges remain (and are discussed) for providing a sign language-rich environment, research evidence suggests that such challenges are worth tackling in light of natural sign languages providing a host of benefits for DHH children – especially in the prevention and reduction of language deprivation.

     
    more » « less
  2. Abstract The auditory scaffolding hypothesis states that early experience with sound underpins the development of domain-general sequence processing abilities, supported by studies observing impaired sequence processing in deaf or hard-of-hearing (DHH) children. To test this hypothesis, we administered a sequence processing task to 77 DHH children who use American Sign Language (ASL) and 23 hearing monolingual children aged 7–12 years and found no performance difference between them after controlling for age and nonverbal intelligence. Additionally, neither spoken language comprehension scores nor hearing loss levels predicted sequence processing scores in the DHH group, whereas ASL comprehension scores did. Our results do not indicate sequence processing deficits in DHH children and do not support the auditory scaffolding hypothesis; instead, these findings suggest that factors related to experience with and/or proficiency in an accessible language during development may be more important determinants of sequence processing abilities. 
    more » « less
  3. Abstract

    Research has shown a link between the acquisition of numerical concepts and language, but exactly how linguistic input matters for numerical development remains unclear. Here, we examine both symbolic (number word knowledge) and non-symbolic (numerical discrimination) numerical abilities in a population in which access to language is limited early in development—oral deaf and hard of hearing (DHH) preschoolers born to hearing parents who do not know a sign language. The oral DHH children demonstrated lower numerical discrimination skills, verbal number knowledge, conceptual understanding of the word “more”, and vocabulary relative to their hearing peers. Importantly, however, analyses revealed that group differences in the numerical tasks, but not vocabulary, disappeared when differences in the amount of time children had had auditory access to spoken language input via hearing technology were taken into account. Results offer insights regarding the role language plays in emerging number concepts.

     
    more » « less
  4. In this paper, I explore metalinguistic discourse in Zinacantec Family Homesign (‘Z sign’), an emergent sign language developed by three deaf siblings and their hearing family members. In particular, I examine how metalinguistic discourse unfolds between a hearing Z signer and various members of her family—including her deaf siblings, her elderly hearing father, and her young hearing son. I do so via a close examination of several snippets of conversation in which the Z signers talk about the “right” way to sign, paying close attention to how they mobilize various semiotic devices, including manual signs, eye gaze, facial expressions, and speech. I aim to understand not only the formal components of metalinguistic discourse in Z sign but also how it functions as a form of social action in this small linguistic community. How do members of this family position themselves and others as (in)competent, (non-)authoritative signers in light of existing social divisions among them? How do they reinforce or challenge those social divisions through metalinguistic discourse? How might metalinguistic discourse contribute to the propagation of emergent linguistic norms throughout the family? I find that a recurrent device for enacting metalinguistic critique among the Z signers is the partial re-production and transformation of others’ utterances and other visible actions, manifested in a way that exploits the availability of multiple, semi-independent manual and non-manual articulators in the visual modality.

     
    more » « less
  5. Abstract

    Most deaf and hard‐of‐hearing (DHH) children are born to hearing parents and steered toward spoken rather than signed language, introducing a delay in language access. This study investigated the effects of this delay on number acquisition. DHH children (N = 44, meanage = 58 months, 21F, >50% White) and typically‐hearing (TH) children (N = 79, meanage = 49 months, 51F, >50% White) were assessed on number and language in 2011–13. DHH children showed similar trajectories to TH children but delayed timing; a binary logistic regression showed that the odds of being a cardinal‐principle (CP) knower were 17 times higher for TH children than DHH children, controlling for age (d = .69). Language fully mediated the association between deaf/hearing group and number knowledge, suggesting that language access sets the pace for number acquisition.

     
    more » « less