More Like this

1. Incidental Nonspeech Auditory Learning Scaffolds Phonetic, Category, and Word Learning in a Foreign Language Classroom

   https://doi.org/10.1111/lang.12700  

   Wiener, Seth; Murphy, Timothy K; Holt, Lori L (January 2025, Language Learning)

   Abstract There is considerable lab‐based evidence for successful incidental learning, in which a learner's attention is directed away from the to‐be‐learned stimulus and towards another stimulus. In this study, we extend incidental learning research into the language learning classroom. Three groups of adult second language (L2) learners (N= 52) engaged in structured classroom Mandarin learning took part in an 8‐week study. One group served as a classroom‐only control group. The second group underwent additional intentional auditory training involving Mandarin speech and explicit feedback. The third group underwent additional incidental learning combined with nonspeech “perceptual building block” categories—categories that share critical perceptual dimensions with target L2 speech categories but that are not perceived as speech. We demonstrate that when supplemented with structured classroom learning, incidental learning involving nonspeech analogs promotes phonetic, category, and word learning equivalent to learning from more traditional intentional auditory training. 

   more » « less
2. Questions Can Answer Questions About Mechanisms of Preschoolers’ Selective Word Learning

   https://doi.org/10.1111/cdev.13395  

   Luchkina, Elena; Morgan, James L.; Williams, Deijah J.; Sobel, David M. (July 2020, Child Development)

   This study examined how inferences about epistemic competence and generalized labeling errors influence children’s selective word learning. Three‐ to 4‐year‐olds (N = 128) learned words from informants who asked questions about objects, mentioning either correct or incorrect labels. Such questions do not convey stark differences in informants’ epistemic competence. Inaccurate labels, however, generate error signals that can lead to weaker encoding of novel information. Preschoolers retained novel labels from both informants but were slower to respond in the Inaccurate Labeler condition. When the test procedure was not sensitive to the strength of information encoding, children performed above chance in both conditions and their response times did not differ. These results suggest that epistemic‐level inferences and error generalizations influence preschoolers’ selective word learning concurrently. 

   more » « less
3. Bridging the Gap: Using Deep Acoustic Representations to Learn Grounded Language from Percepts and Raw Speech

   Kebe, Gaoussou Youssouf; Richards, Luke E.; Raff, Edward; Ferraro, Francis; Matuszek, Cynthia (February 2022, Conference on the Advancement of Artificial Intelligence (AAAI))

   Learning to understand grounded language, which connects natural language to percepts, is a critical research area. Prior work in grounded language acquisition has focused primarily on textual inputs. In this work, we demonstrate the feasibility of performing grounded language acquisition on paired visual percepts and raw speech inputs. This will allow interactions in which language about novel tasks and environments is learned from end-users, reducing dependence on textual inputs and potentially mitigating the effects of demographic bias found in widely available speech recognition systems. We leverage recent work in self-supervised speech representation models and show that learned representations of speech can make language grounding systems more inclusive towards specific groups while maintaining or even increasing general performance. 

   more » « less
4. Bridging the Gap: Using Deep Acoustic Representations to Learn Grounded Language from Percepts and Raw Speech

   https://doi.org/10.1609/aaai.v36i10.21335  

   Kebe, Gaoussou Y.; Richards, Luke E.; Raff, Edward; Ferraro, Francis; Matuszek, Cynthia (February 2022, Proceedings of the AAAI Conference on Artificial Intelligence)

   Learning to understand grounded language, which connects natural language to percepts, is a critical research area. Prior work in grounded language acquisition has focused primarily on textual inputs. In this work, we demonstrate the feasibility of performing grounded language acquisition on paired visual percepts and raw speech inputs. This will allow interactions in which language about novel tasks and environments is learned from end-users, reducing dependence on textual inputs and potentially mitigating the effects of demographic bias found in widely available speech recognition systems. We leverage recent work in self-supervised speech representation models and show that learned representations of speech can make language grounding systems more inclusive towards specific groups while maintaining or even increasing general performance. 

   more » « less
5. Teacher-Student learning based Acoustic Models for Robust Speech Recognition in Naturalistic Childhood Classroom settings

   Kothalkar, P.V. (February 2023, Speech communication)

   Speech and language development are early indicators of overall analytical and learning ability in children. The preschool classroom is a rich language environment for monitoring and ensuring growth in young children by measuring their vocal interactions with both teachers and classmates. Early childhood researchers recognize the importance in analyzing naturalistic vs. controlled lab recordings to measure both quality and quantity of child interactions. Recently, large language model-based speech technologies have performed well on conversational speech recognition. In this regard, we assess performance of such models on the wide dynamic scenario of early childhood classroom settings. This study investigates an alternate Deep Learning-based Teacher-Student learning solution for recognizing adult speech within preschool interactions. Our proposed adapted model achieves the best F1-score for recognizing most frequent 400 words on test sets for both classrooms. Additionally, F1-scores for alternate word groups provides a breakdown of performance across relevant language-based word-categories. The study demonstrates the prospects of addressing educational assessment needs through communication audio stream analysis, while maintaining both security and privacy of all children and adults. The resulting child communication metrics from this study can also be used for broad-based feedback for teachers. 

   more » « less