Title: Neural correlates of listening to nonnative-accented speech in multi-talker background noise
We examined the neural correlates underlying the semantic processing of native- and nonnative-accented sentences, presented in quiet or embedded in multi-talker noise. Implementing a semantic violation paradigm, 36 English monolingual young adults listened to American-accented (native) and Chinese-accented (nonnative) English sentences with or without semantic anomalies, presented in quiet or embedded in multi-talker noise, while EEG was recorded. After hearing each sentence, participants verbally repeated the sentence, which was coded and scored as an offline comprehension accuracy measure. In line with earlier behavioral studies, the negative impact of background noise on sentence repetition accuracy was higher for nonnative-accented than for native-accented sentences. At the neural level, the N400 effect for semantic anomaly was larger for nativeaccented than for nonnative-accented sentences, and was also larger for sentences presented in quiet than in noise, indicating impaired lexical-semantic access when listening to nonnative-accented speech or sentences embedded in noise. No semantic N400 effect was observed for nonnative-accented sentences presented in noise. Furthermore, the frequency of neural oscillations in the alpha frequency band (an index of online cognitive listening effort) was higher when listening to sentences in noise versus in quiet, but no difference was observed across the accent conditions. Semantic anomalies presented in background noise also elicited higher theta activity, whereas processing nonnative-accented anomalies was associated with decreased theta activity. Taken together, we found that listening to nonnative accents or background noise is associated with processing challenges during online semantic access, leading to decreased comprehension accuracy. However, the underlying cognitive mechanism (e.g., associated listening efforts) might manifest differently across accented speech processing and speech in noise processing.  more » « less
Award ID(s):
2041204 2125865
PAR ID:
10574190
Author(s) / Creator(s):
;
Publisher / Repository:
Neuropsychologia
Date Published:
Journal Name:
Neuropsychologia
Volume:
203
Issue:
C
ISSN:
0028-3932
Page Range / eLocation ID:
108968
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Holliday, Jeff; Lee-KIm, Sang-Im; Cho, Taehong (Ed.)
    Listeners use their knowledge about a talker to guide speech perception. In the present study, we manipulated both the familiarity and predictability of this talker-specific knowledge. Twenty-two listeners from western Pennsylvania completed an audio-visual lexical decision task while EEG was recorded. Critically, listeners were introduced to the talkers beforehand, with short videos establishing the talker’s U.S. English accent identity: Mainstream, more similar to the participants themselves; Southern, a relatively less familiar variety; and Unpredictable, which switched between Mainstream and Southern accents. During the task, listeners watched these talkers produce tokens with accents that either aligned with or violated their accent identity. Behaviorally, alignment between talker and token accent influenced performance on Southern-accented tokens, while accuracy on Mainstream-accented tokens was nearly at ceiling regardless of talker accent identity. Neurally, accent familiarity had the largest effect pre-speech (in response to the visual presentation of the talker), while both predictability and familiarity impacted processing of the speech signal itself. Overall, our results suggest that listeners use information about a talker’s accent, even when it is unfamiliar or unpredictable, to alter their listening strategies for more successful speech recognition. 
    more » « less
  2. When listeners encounter a difficult-to-understand talker in a difficult-to-understand situation, their perceptual mechanisms can adapt, making the talker in the situation easier to understand. This study examined talker-specific perceptual adaptation experimentally by embedding speech from second-language (L2) English talkers in varying levels of noise and collecting transcriptions from first-language English listeners (ten talkers, 100 listeners per experiment). Experiments 1 and 2 demonstrated that prior experience with a L2 talker's speech presented first without noise and then with gradually increasing levels of noise facilitated recognition of that talker in loud noise. Experiment 3 tested whether adaptation is driven by tuning-in to the talker's voice and speech patterns, by examining recognition of speech-in-loud-noise following experience with the talker in quiet. Finally, experiment 4 tested whether adaptation is driven by tuning-out the background noise, by measuring speech-in-loud-noise recognition after experience with the talker in consistently loud noise. The results showed that both tuning-in to the talker and tuning-out the noise contribute to talker-specific perceptual adaptation to L2 speech-in-noise. 
    more » « less
  3. null (Ed.)
    Novel metaphorical language use exemplifies human creativity through production and comprehension of meaningful linguistic expressions that may have never been heard before. Available electrophysiological research demonstrates, however, that novel metaphor comprehension is cognitively costly, as it requires integrating information from distantly related concepts. Herein, we investigate if such cognitive cost may be reduced as a factor of prior domain knowledge. To this end, we asked engineering and nonengineering students to read for comprehension literal, novel metaphorical, and anomalous sentences related to engineering or general knowledge, while undergoing EEG recording. Upon reading each sentence, participants were asked to judge whether or not the sentence was original in meaning (novelty judgment) and whether or not it made sense (sensicality judgment). When collapsed across groups, our findings demonstrate a gradual N400 modulation with N400 being maximal in response to anomalous, followed by metaphorical, and literal sentences. Between-group comparisons revealed a mirror effect on the N400 to novel metaphorical sentences, with attenuated N400 in engineers and enhanced N400 in non-engineers. Critically, planned comparisons demonstrated reduced N400 amplitudes to engineering novel metaphors in engineers relative to non-engineers, pointing to an effect of prior knowledge on metaphor processing. This reduction, however, was observed in the absence of a sentence type × knowledge × group interaction. Altogether, our study provides novel evidence suggesting that prior domain knowledge may have a direct impact on creative language comprehension. 
    more » « less
  4. Abstract Alternating between different talkers during listening typically incurs a cognitive processing cost. How these processing costs manifest, and potentially differ, in a multi-accent setting remains to be examined. Across two experiments, we investigate (1) whether talker and accent switching costs are driven by engagement of a recalibration mechanism, and (2) whether global listening context affects the magnitude of talker and accent switching costs. The results of our first experiment indicate that switching between speakers of the same second language (L2) accent (e.g., between two Mandarin-accented speakers of English) was less cognitively challenging than switching between speakers of different L2 accents (e.g., between a Mandarin-accented speaker and a Turkish-accented speaker of English). This outcome suggests that the perceptual distance (i.e., the holistic estimate of spectral and temporal differences in acoustic signals) between two speakers’ productions determines the size of associated switching costs, such that recalibration is less cognitively demanding for speakers with the same L2 accent. In our second experiment, we examine whether a more challenging block-wide listening context results in a global upregulation of cognitive resources, and, subsequently, reduces the cognitive resources required to (a) process L2 accent and (b) resolve local talker and accent changes. Here, the overall cognitive demands of processing L2 accent were reduced, as predicted, but talker and accent switching costs were not. We conclude that talker and accent switching are supported by a recalibration mechanism and that global upregulation of cognitive resources may reduce L2 accent processing costs but not local switching costs. 
    more » « less
  5. Radek Skarnitzl & Jan Volín (Ed.)
    Unfamiliar native and non-native accents can cause word recognition challenges, particularly in noisy environments, but few studies have incorporated quantitative pronunciation distance metrics to explain intelligibility differences across accents. Here, intelligibility was measured for 18 talkers -- two from each of three native, one bilingual, and five non- native accents -- in three listening conditions (quiet and two noise conditions). Two variations of the Levenshtein pronunciation distance metric, which quantifies phonemic differences from a reference accent, were assessed for their ability to predict intelligibility. An unweighted Levenshtein distance metric was the best intelligibility predictor; talker accent further predicted performance. Accuracy did not fall along a native - non-native divide. Thus, phonemic differences from the listener’s home accent primarily determine intelligibility, but other accent- specific pronunciation features, including suprasegmental characteristics, must be quantified to fully explain intelligibility across talkers and listening conditions. These results have implications for pedagogical practices and speech perception theories. 
    more » « less