Speech and language development in children are crucial for ensuring
effective skills in their long-term learning ability. A child’s
vocabulary size at the time of entry into kindergarten is an early
indicator of their learning ability to read and potential long-term
success in school. The preschool classroom is thus a promising
venue for assessing growth in young children by measuring their
interactions with teachers as well as classmates. However, to date
limited studies have explored such naturalistic audio communications.
Automatic Speech Recognition (ASR) technologies provide an
opportunity for ’Early Childhood’ researchers to obtain knowledge
through automatic analysis of naturalistic classroom recordings in
measuring such interactions. For this purpose, 208 hours of audio
recordings across 48 daylong sessions are collected in a childcare
learning center in the United States using Language Environment
Analysis (LENA) devices worn by the preschool children. Approximately
29 hours of adult speech and 26 hours of child speech is
segmented using manual transcriptions provided by CRSS transcription
team. Traditional as well as End-to-End ASR models are trained
on adult/child speech data subset. Factorized Time Delay Neural
Network provides a best Word-Error-Rate (WER) of 35.05% on the
adult subset of the test set. End-to-End transformer models achieve
63.5% WER on the child subset of the test data. Next, bar plots
demonstrating the frequency of WH-question words in Science vs.
Reading activity areas of the preschool are presented for sessions in
the test set. It is suggested that learning spaces could be configured
to encourage greater adult-child conversational engagement given
such speech/audio assessment strategies.
more »
« less
Objectively measured teacher and preschooler vocalizations: Phonemic diversity is associated with language abilities
Over half of US children are enrolled in preschools, where the quantity and quality of language input from teachers are likely to affect children's language development. Leveraging repeated objective measurements, we examined the rate per minute and phonemic diversity of child and teacher speech‐related vocalizations in preschool classrooms and their association with children's end‐of‐year receptive and expressive language abilities measured with the Preschool Language Scales (PLS‐5). Phonemic diversity was computed as the number of unique consonants and vowels in a speech‐related vocalization. We observed three successive cohorts of 2.5–3.5‐year‐old children enrolled in an oral language classroom that included children with and without hearing loss (
- NSF-PAR ID:
- 10362944
- Publisher / Repository:
- Wiley-Blackwell
- Date Published:
- Journal Name:
- Developmental Science
- Volume:
- 25
- Issue:
- 2
- ISSN:
- 1363-755X
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Speech and language development are early indicators of overall analytical and learning ability in children. The preschool classroom is a rich language environment for monitoring and ensuring growth in young children by measuring their vocal interactions with teachers and classmates. Early childhood researchers are naturally interested in analyzing naturalistic vs. controlled lab recordings to measure both quality and quantity of such interactions. Unfortunately, present-day speech technologies are not capable of addressing the wide dynamic scenario of early childhood classroom settings. Due to the diversity of acoustic events/conditions in such daylong audio streams, automated speaker diarization technology would need to be advanced to address this challenging domain for segmenting audio as well as information extraction. This study investigates an alternate Deep Learning-based diarization solution for segmenting classroom interactions of 3-5 year old children with teachers. In this context, the focus on speech-type diarization which classifies speech segments as being either from adults or children partitioned across multiple classrooms. Our proposed ResNet model achieves a best F1-score of ∼78.0% on data from two classrooms, based on dev and test sets of each classroom. It is utilized with Automatic Speech Recognition-based resegmentation modules to perform child-adult diarization. Additionally, F1-scores are obtained for individual segments with corresponding speaker tags (e.g., adult vs. child), which provide knowledge for educators on child engagement through naturalistic communications. The study demonstrates the prospects of addressing educational assessment needs through communication audio stream analysis, while maintaining both security and privacy of all children and adults. The resulting child communication metrics have been used for broad-based feedback for teachers with the help of visualizations.more » « less
-
Speech and language development are early indicators of overall analytical and learning ability in children. The preschool classroom is a rich language environment for monitoring and ensuring growth in young children by measuring their vocal interactions with teachers and classmates. Early childhood researchers are naturally interested in analyzing naturalistic vs. controlled lab recordings to measure both quality and quantity of such interactions. Unfortunately, present-day speech technologies are not capable of addressing the wide dynamic scenario of early childhood classroom settings. Due to the diversity of acoustic events/conditions in such daylong audio streams, automated speaker diarization technology would need to be advanced to address this challenging domain for segmenting audio as well as information extraction. This study investigates an alternate Deep Learning-based diarization solution for segmenting classroom interactions of 3-5 year old children with teachers. In this context, the focus on speech-type diarization which classifies speech segments as being either from adults or children partitioned across multiple classrooms. Our proposed ResNet model achieves a best F1-score of ∼71.0% on data from two classrooms, based on dev and test sets of each classroom. Additionally, F1-scores are obtained for individual segments with corresponding speaker tags (e.g., adult vs. child), which provide knowledge for educators on child engagement through naturalistic communications. The study demonstrates the prospects of addressing educational assessment needs through communication audio stream analysis, while maintaining both security and privacy of all children and adults. The resulting child communication metrics have been used for broad-based feedback for teachers with the help of visualizations.more » « less
-
null (Ed.)Speech and language development in children is crucial for ensuring optimal outcomes in their long term development and life-long educational journey. A child’s vocabulary size at the time of kindergarten entry is an early indicator of learning to read and potential long-term success in school. The preschool classroom is thus a promising venue for monitoring growth in young children by measuring their interactions with teachers and classmates. Automatic Speech Recognition (ASR) technologies provide the ability for ‘Early Childhood’ researchers for automatically analyzing naturalistic recordings in these settings. For this purpose, data are collected in a high-quality childcare center in the United States using Language Environment Analysis (LENA) devices worn by the preschool children. A preliminary task for ASR of daylong audio recordings would involve diarization, i.e., segmenting speech into smaller parts for identifying ‘who spoke when.’ This study investigates a Deep Learning-based diarization system for classroom interactions of 3-5-year-old children. However, the focus is on ’speaker group’ diarization, which includes classifying speech segments as being from adults or children from across multiple classrooms. SincNet based diarization systems achieve utterance level Diarization Error Rate of 19.1%. Utterance level speaker group confusion matrices also show promising, balanced results. These diarization systems have potential applications in developing metrics for adult-to-child or child-to-child rapid conversational turns in a naturalistic noisy early childhood setting. Such technical advancements will also help teachers better and more efficiently quantify and understand their interactions with children, make changes as needed, and monitor the impact of those changes.more » « less
-
Speech and language development are early indicators of overall analytical and learning ability in children. The preschool classroom is a rich language environment for monitoring and ensuring growth in young children by measuring their vocal interactions with both teachers and classmates. Early childhood researchers recognize the importance in analyzing naturalistic vs. controlled lab recordings to measure both quality and quantity of child interactions. Recently, large language model-based speech technologies have performed well on conversational speech recognition. In this regard, we assess performance of such models on the wide dynamic scenario of early childhood classroom settings. This study investigates an alternate Deep Learning-based Teacher-Student learning solution for recognizing adult speech within preschool interactions. Our proposed adapted model achieves the best F1-score for recognizing most frequent 400 words on test sets for both classrooms. Additionally, F1-scores for alternate word groups provides a breakdown of performance across relevant language-based word-categories. The study demonstrates the prospects of addressing educational assessment needs through communication audio stream analysis, while maintaining both security and privacy of all children and adults. The resulting child communication metrics from this study can also be used for broad-based feedback for teachers.more » « less
