More Like this

1. Zero-Shot Personalized Speech Enhancement Through Speaker-Informed Model Selection

   https://doi.org/10.1109/WASPAA52581.2021.9632752  

   Sivaraman, Aswin; Kim, Minje (October 2021, 2021 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA))

   This paper presents a novel zero-shot learning approach towards personalized speech enhancement through the use of a sparsely active ensemble model. Optimizing speech denoising systems towards a particular test-time speaker can improve performance and reduce run-time complexity. However, test-time model adaptation may be challenging if collecting data from the test-time speaker is not possible. To this end, we propose using an ensemble model wherein each specialist module denoises noisy utterances from a distinct partition of training set speakers. The gating module inexpensively estimates test-time speaker characteristics in the form of an embedding vector and selects the most appropriate specialist module for denoising the test signal. Grouping the training set speakers into non-overlapping semantically similar groups is non-trivial and ill-defined. To do this, we first train a Siamese network using noisy speech pairs to maximize or minimize the similarity of its output vectors depending on whether the utterances derive from the same speaker or not. Next, we perform k-means clustering on the latent space formed by the averaged embedding vectors per training set speaker. In this way, we designate speaker groups and train specialist modules optimized around partitions of the complete training set. Our experiments show that ensemble models made up of low-capacity specialists can outperform high-capacity generalist models with greater efficiency and improved adaptation towards unseen test-time speakers. 

   more » « less
2. A Continuous Articulatory Gesture Based Liveness Detection for Voice Authentication on Smart Devices

   https://doi.org/10.1109/JIOT.2022.3199995  

   Zhang, Linghan; Tan, Sheng; Chen, Yingying; Yang, Jie (August 2022, IEEE Internet of Things Journal)

   Voice biometrics is drawing increasing attention to user authentication on smart devices. However, voice biometrics is vulnerable to replay attacks, where adversaries try to spoof voice authentication systems using pre-recorded voice samples collected from genuine users. To this end, we propose VoiceGesture, a liveness detection solution for voice authentication on smart devices such as smartphones and smart speakers. With audio hardware advances on smart devices, VoiceGesture leverages built-in speaker and microphone pairs on smart devices as Doppler Radar to sense articulatory gestures for liveness detection during voice authentication. The experiments with 21 participants and different smart devices show that VoiceGesture achieves over 99% and around 98% detection accuracy for text-dependent and text-independent liveness detection, respectively. Moreover, VoiceGesture is robust to different device placements, low audio sampling frequency, and supports medium range liveness detection on smart speakers in various use scenarios, including smart homes and smart vehicles. 

   more » « less
3. Engaging hearts and minds in assessment research

   Bostic, J (December 2023, School science and mathematics)

   Miller, B; Martin, C (Ed.)

   Assessment continues to be an important conversation point within Science, Technology, Engineering, and Mathematics (STEM) education scholarship and practice (Krupa et al., 2019; National Research Council, 2001). There are guidelines for developing and evaluating assess- ments (e.g., AERA et al., 2014; Carney et al., 2022; Lavery et al., 2019; Wilson & Wilmot, 2019). There are also Standards for Educational & Psychological Testing (Standards; AERA et al., 2014) that discuss important rele- vant frameworks and information about using assessment results and interpretations. Quantitative assessments are used as part of daily STEM instruction, STEM research, and STEM evaluation; therefore, having robust assess- ments is necessary (National Research Council, 2001). An aim of this editorial is to give readers a few relevant ideas about modern assessment research, some guidance for the use of quantitative assessments, and framing validation and assessment research as equity-forward work. 

   more » « less
4. The role of discourse-level expectations in non-native speakers' referential choices

   Grüter, T.; Rohde, H.; Schafer, A. (May 2014, Proceedings of the annual Boston University Conference on Language Development)

   Expectation is a powerful mechanism in native-language processing. Less is known about its role in non-native language processing, especially for expectations at the discourse level. This study presents evidence from a story-continuation task, adapted from previous work with native speakers (Rohde et al., 2006), probing next-mention and coherence expectations among Japanese- and Korean-speaking learners of English. As in previous work, verbal aspect (perfective/imperfective) in a context sentence describing a transfer-of-possession event (e.g., Ron gave/was giving a towel to Patrick) modulated participants’ choices of next referents in their continuations. However, this effect was diminished in the non-native compared to the native-speaker group, despite comparable performance on an independent task assessing knowledge of verbal aspect in English, and previous evidence for significant effects of aspect on referential patterns in native Japanese and Korean processing (Ueno & Kehler, 2010; Kim et al., 2013). The two groups of speakers were equally sensitive to a cue that does not require predictive processing – the referential form of the story-continuation prompt – in that both groups were significantly more likely to establish reference to the discourse topic/Source of the transfer event for pronoun-initial continuations than for name-initial ones. Moreover, recency played a stronger role in non-native speakers’ referential choices than in those of native speakers. These results suggest that while native speakers engage in proactive discourse processing, non-native speakers are less able to do so, being sufficiently burdened by reactive processes required for information integration that they have only Reduced Ability to Generate Expectations (RAGE). 

   more » « less
5. Eccentric C-V timing across speakers of diaspora Tibetan with and without lexical tone contrasts

   Geissler, Christopher; Shaw, Jason; Hu, Fang; Tiede, Mark (January 2021, Proceedings of the 12th International Seminar on Speech Production.)

   Tiede, Mark; Whalen, Doug; Gracco, Vincent (Ed.)

   This paper investigates the relative timing of onset consonant and vowel gestures in Tibetan as spoken in the Tibetan diaspora. According to the coupled oscillator model of articulatory timing (Browman & Goldstein 2000, Nam & Saltzman 2003), the most readily-available coupling modes among gestures are in-phase (synchronous) or anti-phase (sequential) timing, with competition among these modes also giving rise to a stable timing pattern. The model predicts that other timing relations, i.e. ”eccentric timing”, are possible but not as readily available. Data gathered using electromagnetic articulography (EMA) shows relative C-V timing consistent with either competitive coupling or eccentric timing. Competitive coupling is a plausible explanation for CV syllables in a tone language (Gao 2008), but acoustic analysis showed that some speakers do not produce a pitch contrast corresponding to tone. In the apparent absence of a tone gesture, we conclude that these speakers exhibit eccentric C-V timing. 

   more » « less