More Like this

1. Improving Black-box Speech Recognition using Semantic Parsing

   Corona, R.; Thomason, J.; Mooney, R.J. (November 2017, Proceedings of the 8th International Joint Conference on Natural Language Processing (IJCNLP-17))

   Speech is a natural channel for human-computer interaction in robotics and consumer applications. Natural language understanding pipelines that start with speech can have trouble recovering from speech recognition errors. Black-box automatic speech recognition (ASR) systems, built for general purpose use, are unable to take advantage of in-domain language models that could otherwise ameliorate these errors. In this work, we present a method for re-ranking black-box ASR hypotheses using an in-domain language model and semantic parser trained for a particular task. Our re-ranking method significantly improves both transcription accuracy and semantic understanding over a state-of-the-art ASR’s vanilla output. 

   more » « less
2. Modeling Rhythm in Speech as in Music: Towards a Unified Cognitive Representation

   Li, Ruolan; Schatz, Thomas; Feldman, Naomi H. (August 2022, Proceedings of the Conference on Cognitive Computational Neuroscience)

   Rhythm plays an important role in language perception and learning, with infants perceiving rhythmic differences across languages at birth. While the mechanisms underlying rhythm perception in speech remain unclear, one interesting possibility is that these mechanisms are similar to those involved in the perception of musical rhythm. In this work, we adopt a model originally designed for musical rhythm to simulate speech rhythm perception. We show that this model replicates the behavioral results of language discrimination in newborns, and outperforms an existing model of infant language discrimination. We also find that percussives — fast-changing components in the acoustics — are necessary for distinguishing languages of different rhythms, which suggests that percussives are essential for rhythm perception. Our music-inspired model of speech rhythm may be seen as a first step towards a unified theory of how rhythm is represented in speech and music. 

   more » « less
3. Understanding speech and language in tuberous sclerosis complex

   https://doi.org/10.3389/fnhum.2023.1149071  

   Gipson, Tanjala T.; Oller, D. Kimbrough; Messinger, Daniel S.; Perry, Lynn K. (June 2023, Frontiers in Human Neuroscience)

   Tuberous Sclerosis Complex (TSC), is a neurocutaneous disorder, associated with a high prevalence of autism spectrum disorder (ASD; ∼50% of individuals). As TSC is a leading cause of syndromic ASD, understanding language development in this population would not only be important for individuals with TSC but may also have implications for those with other causes of syndromic and idiopathic ASD. In this mini review, we consider what is known about language development in this population and how speech and language in TSC are related to ASD. Although up to 70% of individuals with TSC report language difficulties, much of the limited research to date on language in TSC has been based on summary scores from standardized assessments. Missing is a detailed understanding of the mechanisms driving speech and language in TSC and how they relate to ASD. Here, we review recent work suggesting that canonical babbling and volubility—two precursors of language development that predict the emergence of speech and are delayed in infants with idiopathic ASD—are also delayed in infants with TSC. We then look to the broader literature on language development to identify other early precursors of language development that tend to be delayed in children with autism as a guide for future research on speech and language in TSC. We argue that vocal turn-taking, shared attention, and fast mapping are three such skills that can provide important information about how speech and language develop in TSC and where potential delays come from. The overall goal of this line of research is to not only illuminate the trajectory of language in TSC with and without ASD, but to ultimately find strategies for earlier recognition and treatment of the pervasive language difficulties in this population. 

   more » « less
4. Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition

   https://doi.org/10.21437/Interspeech.2022-816  

   Ni, Junrui; Wang, Liming; Gao, Heting; Qian, Kaizhi; Zhang, Yang; Chang, Shiyu; Hasegawa-Johnson, Mark (September 2022, Proc. Interspeech 2022)

   An unsupervised text-to-speech synthesis (TTS) system learns to generate speech waveforms corresponding to any written sentence in a language by observing: 1) a collection of untranscribed speech waveforms in that language; 2) a collection of texts written in that language without access to any transcribed speech. Developing such a system can significantly improve the availability of speech technology to languages without a large amount of parallel speech and text data. This paper proposes an unsupervised TTS system based on an alignment module that outputs pseudo-text and another synthesis module that uses pseudo-text for training and real text for inference. Our unsupervised system can achieve comparable performance to the supervised system in seven languages with about 10-20 hours of speech each. A careful study on the effect of text units and vocoders has also been conducted to better understand what factors may affect unsupervised TTS performance. The samples generated by our models can be found at https://cactuswiththoughts.github.io/UnsupTTS-Demo, and our code can be found at https://github.com/lwang114/UnsupTTS. 

   more » « less
5. Hate Speech Classifiers Learn Normative Social Stereotypes

   https://doi.org/10.1162/tacl_a_00550  

   Davani, Aida Mostafazadeh; Atari, Mohammad; Kennedy, Brendan; Dehghani, Morteza (March 2023, Transactions of the Association for Computational Linguistics)

   Abstract Social stereotypes negatively impact individuals’ judgments about different groups and may have a critical role in understanding language directed toward marginalized groups. Here, we assess the role of social stereotypes in the automated detection of hate speech in the English language by examining the impact of social stereotypes on annotation behaviors, annotated datasets, and hate speech classifiers. Specifically, we first investigate the impact of novice annotators’ stereotypes on their hate-speech-annotation behavior. Then, we examine the effect of normative stereotypes in language on the aggregated annotators’ judgments in a large annotated corpus. Finally, we demonstrate how normative stereotypes embedded in language resources are associated with systematic prediction errors in a hate-speech classifier. The results demonstrate that hate-speech classifiers reflect social stereotypes against marginalized groups, which can perpetuate social inequalities when propagated at scale. This framework, combining social-psychological and computational-linguistic methods, provides insights into sources of bias in hate-speech moderation, informing ongoing debates regarding machine learning fairness. 

   more » « less