An official website of the United States government
Human brains grasp the gists of visual scenes from a single glance, but to what extent is this possible for language? While we typically think of language in terms of sequential speech, our everyday experience involves numerous rapidly flashing written notifications, which we understand instantly. What do our brains detect in the first few hundred milliseconds after seeing such a stimulus? We flashed short sentences during magnetoencephalography measurement, revealing sentence-sensitive neural activity in left temporal cortex within 130 milliseconds. These signals emerged for subject-verb-object sentences regardless of grammatical or semantic well-formedness, suggesting that at-a-glance language comprehension begins by detecting basic phrase structure, independent of meaning or other grammatical details. Our findings unveil one aspect of how our brains process information rapidly in today’s visually saturated world.
more »
« less
This content will become publicly available on November 27, 2025
The Spatiotemporal Dynamics of Bottom–Up and Top–Down Processing during At-a-Glance Reading
Like all domains of cognition, language processing is affected by top–down knowledge. Classic evidence for this is missing blatant errors in the signal. In sentence comprehension, one instance is failing to notice word order errors, such as transposed words in the middle of a sentence: “you that read wrong” (Mirault et al., 2018). Our brains seem to fix such errors, since they are incompatible with our grammatical knowledge, but how do our brains do this? Following behavioral work on inner transpositions, we flashed four-word sentences for 300 ms using rapid parallel visual presentation (Snell and Grainger, 2017). We compared magnetoencephalography responses to fully grammatical and reversed sentences (24 human participants: 21 females, 4 males). The left lateral language cortex robustly distinguished grammatical and reversed sentences starting at 213 ms. Thus, the influence of grammatical knowledge begun rapidly after visual word form recognition (Tarkiainen et al., 1999). At the earliest stage of this neural “sentence superiority effect,” inner transpositions patterned between grammatical and reversed sentences, showing evidence that the brain initially “noticed” the error. However, 100 ms later, inner transpositions became indistinguishable from grammatical sentences, suggesting at this point, the brain had “fixed” the error. These results show that after a single glance at a sentence, syntax impacts our neural activity almost as quickly as higher-level object recognition is assumed to take place (Cichy et al., 2014). The earliest stage involves detailed comparisons between the bottom–up input and grammatical knowledge, while shortly afterward, top–down knowledge can override an error in the stimulus.
more »
« less
- Award ID(s):
- 2335767
- PAR ID:
- 10592131
- Publisher / Repository:
- Journal of Neuroscience
- Date Published:
- Journal Name:
- The Journal of Neuroscience
- Volume:
- 44
- Issue:
- 48
- ISSN:
- 0270-6474
- Page Range / eLocation ID:
- e0374242024
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This paper investigates the ability of artificial neural networks to judge the grammatical acceptability of a sentence, with the goal of testing their linguistic competence. We introduce the Corpus of Linguistic Acceptability (CoLA), a set of 10,657 English sentences labeled as grammatical or ungrammatical from published linguistics literature. As baselines, we train several recurrent neural network models on acceptability classification, and find that our models outperform unsupervised models by Lau et al. (2016) on CoLA. Error-analysis on specific grammatical phenomena reveals that both Lau et al.’s models and ours learn systematic generalizations like subject-verb-object order. However, all models we test perform far below human level on a wide range of grammatical constructions.more » « less
-
We conduct a thorough study to diagnose the behaviors of pre-trained language encoders (ELMo, BERT, and RoBERTa) when confronted with natural grammatical errors. Specifically, we collect real grammatical errors from non-native speakers and conduct adversarial attacks to simulate these errors on clean text data. We use this approach to facilitate debugging models on downstream applications. Results confirm that the performance of all tested models is affected but the degree of impact varies. To interpret model behaviors, we further design a linguistic acceptability task to reveal their abilities in identifying ungrammatical sentences and the position of errors. We find that fixed contextual encoders with a simple classifier trained on the prediction of sentence correctness are able to locate error positions. We also design a cloze test for BERT and discover that BERT captures the interaction between errors and specific tokens in context. Our results shed light on understanding the robustness and behaviors of language encoders against grammatical errors.more » « less
-
null (Ed.)Engineering programs, in general, do not explicitly address the need to enhance divergent thinking. To a certain extent this is due to a lack in knowledge on the cognitive and neural mechanisms underlying divergent thinking, and creative ideation more generally. We hypothesize that we can help enhance our students’ divergent thinking and creative processing outcomes by investigating the impacts of carefully selected methods and tools enabled by developments in the robust analysis of engineering ideation performance, and neurocognitive responses to creativity. In this paper, we present an experiment using the Event-Related brain Potentials (ERP) technique and creative language use (funded by Core R&D Programs). More specifically, we collected ERP responses to literal, nonsense, and novel metaphorical sentences that were either referring to engineering knowledge or general knowledge, testing engineering and non-engineering students. Following Rutter et al. [1], sentences differed in verb only and had been classified in prior sentence norming studies as highly unusual and highly appropriate (novel metaphors), low unusual and highly appropriate (literal sentences), and highly unusual and low appropriate (nonsense sentences). Participants read sentences while their EEG was recorded, and after reading the sentence made judgments about its unusualness and appropriateness. The findings indicate that prior knowledge modulates novel metaphor processing at the stage of lexico–semantic access, indexed by the amplitude of N400 component. Specifically, N400 amplitudes to novel metaphorical sentences are significantly reduced and pattern with literal sentences in engineers; in nonengineers, by contrast, we observed increased N400 amplitudes to novel metaphorical sentences that pattern with anomalous sentences. This mirror effect on the N400 corroborates recent findings demonstrating a strong impact of prior experience and expertise on meaning ambiguity resolution, which may in turn have implications for creative cognition.more » « less
-
Abstract Letter position coding in word recognition has been widely investigated in the visual modality (e.g., labotarory is confusable with laboratory), but not as much in the tactile modality using braille, leading to an incomplete understanding of whether this process is modality-dependent. Unlike sighted readers, braille readers do not show a transposed-letter similarity effect with nonadjacent transpositions (e.g., labotarory = labodanory; Perea et al., 2012). While this latter finding was taken to suggest that the flexibility in letter position coding was due to visual factors (e.g., perceptual uncertainty in the location of visual objects (letters)), it is necessary to test whether transposed-letter effects occur with adjacent letters to reach firm conclusions. Indeed, in the auditory modality (i.e., another serial modality), a transposed-phoneme effect occurs for adjacent but not for nonadjacent transpositions. In a lexical decision task, we examined whether pseudowords created by transposing two adjacent letters of a word (e.g., laboartory) are more confusable with their base word (laboratory) than pseudowords created by replacing those letters (laboestory) in braille. Results showed that transposed-letter pseudowords produced more errors and slower responses than the orthographic controls. Thus, these findings suggest that the mechanism of serial order, while universal, can be shaped by the sensory modality at play.more » « less
