More Like this

1. Speaker matters: Natural inter-speaker variation affects 4-month-olds’ perception of audio-visual speech

   https://doi.org/10.1177/0142723719876382  

   Pejovic, Jovana ; Yee, Eiling ; Molnar, Monika ( April 2020 , First Language)

   In the language development literature, studies often make inferences about infants’ speech perception abilities based on their responses to a single speaker. However, there can be significant natural variability across speakers in how speech is produced (i.e., inter-speaker differences). The current study examined whether inter-speaker differences can affect infants’ ability to detect a mismatch between the auditory and visual components of vowels. Using an eye-tracker, 4.5-month-old infants were tested on auditory-visual (AV) matching for two vowels (/i/ and /u/). Critically, infants were tested with two speakers who naturally differed in how distinctively they articulated the two vowels within and across the categories. Only infants who watched and listened to the speaker whose visual articulations of the two vowels were most distinct from one another were sensitive to AV mismatch. This speaker also produced a visually more distinct /i/ as compared to the other speaker. This finding suggests that infants are sensitive to the distinctiveness of AV information across speakers, and that when making inferences about infants’ perceptual abilities, characteristics of the speaker should be taken into account. 

   more » « less
2. Speaker Conditioning of Acoustic Models Using Affine Transformation for Multi-Speaker Speech Recognition

   https://doi.org/10.1109/ASRU51503.2021.9688231  

   Yousefi, Midia ; Hansen, John H.L. ( December 2021 , IEEE ASRU-2021: Automatic Speech Recognition & Understanding Workshop)

   This study addresses the problem of single-channel Automatic Speech Recognition of a target speaker within an overlap speech scenario. In the proposed method, the hidden representations in the acoustic model are modulated by speaker auxiliary information to recognize only the desired speaker. Affine transformation layers are inserted into the acoustic model network to integrate speaker information with the acoustic features. The speaker conditioning process allows the acoustic model to perform computation in the context of target-speaker auxiliary information. The proposed speaker conditioning method is a general approach and can be applied to any acoustic model architecture. Here, we employ speaker conditioning on a ResNet acoustic model. Experiments on the WSJ corpus show that the proposed speaker conditioning method is an effective solution to fuse speaker auxiliary information with acoustic features for multi-speaker speech recognition, achieving +9% and +20% relative WER reduction for clean and overlap speech scenarios, respectively, compared to the original ResNet acoustic model baseline. 

   more » « less
3. DyViSE: Dynamic Vision-Guided Speaker Embedding for Audio-Visual Speaker Diarization

   https://doi.org/10.1109/MMSP55362.2022.9948860  

   Wuerkaixi, Abudukelimu ; Yan, Kunda ; Zhang, You ; Duan, Zhiyao ; Zhang, Changshui ( September 2022 , 2022 IEEE 24th International Workshop on Multimedia Signal Processing (MMSP))
4. Count and separate: incorporating speaker counting for continuous speaker separation

   https://doi.org/10.1109/ICASSP39728.2021.9414677  

   Wang, Z.-Q. ; Wang, D.L. ( January 2021 , Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing)

   null (Ed.)
5. Unsupervised speaker adaptation for speaker independent acoustic to articulatory speech inversion

   https://doi.org/10.1121/1.5116130  

   Sivaraman, Ganesh ; Mitra, Vikramjit ; Nam, Hosung ; Tiede, Mark ; Espy-Wilson, Carol ( July 2019 , The Journal of the Acoustical Society of America)