Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the language network. To understand what aspects of linguistic stimuli contribute to ANN-to-brain similarity, we used an fMRI data set of responses to n = 627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated the stimuli for which ANN representations were extracted. In particular, we (i) perturbed sentences’ word order, (ii) removed different subsets of words, or (iii) replaced sentences with other sentences of varying semantic similarity. We found that the lexical-semantic content of the sentence (largely carried by content words) rather than the sentence’s syntactic form (conveyed via word order or function words) is primarily responsible for the ANN-to-brain similarity. In follow-up analyses, we found that perturbation manipulations that adversely affect brain predictivity also lead to more divergent representations in the ANN’s embedding space and decrease the ANN’s ability to predict upcoming tokens in those stimuli. Further, results are robust as to whether the mapping model is trained on intact or perturbed stimuli and whether the ANN sentence representations are conditioned on the same linguistic context that humans saw. The critical result—that lexical-semantic content is the main contributor to the similarity between ANN representations and neural ones—aligns with the idea that the goal of the human language system is to extract meaning from linguistic strings. Finally, this work highlights the strength of systematic experimental manipulations for evaluating how close we are to accurate and generalizable models of the human language network.
Successful communication and cooperation among different members of society depends, in part, on a consistent understanding of the physical and social world. What drives this alignment in perspectives? We present evidence from two neuroimaging studies using functional magnetic resonance imaging (fMRI;
- Award ID(s):
- 1835239
- NSF-PAR ID:
- 10370173
- Publisher / Repository:
- Nature Publishing Group
- Date Published:
- Journal Name:
- Scientific Reports
- Volume:
- 12
- Issue:
- 1
- ISSN:
- 2045-2322
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
more » « less
Abstract Introduction How do multiple sources of information interact to form mental representations of object categories? It is commonly held that object categories reflect the integration of perceptual features and semantic/knowledge‐based features. To explore the relative contributions of these two sources of information, we used functional magnetic resonance imaging (fMRI) to identify regions involved in the representation object categories with shared visual and/or semantic features.
Methods Participants (
N = 20) viewed a series of objects that varied in their degree of visual and semantic overlap in the MRI scanner. We used a blocked adaptation design to identify sensitivity to visual and semantic features in a priori visual processing regions and in a distributed network of object processing regions with an exploratory whole‐brain analysis.Results Somewhat surprisingly, within higher‐order visual processing regions—specifically lateral occipital cortex (LOC)—we did not obtain any difference in neural adaptation for shared visual versus semantic category membership. More broadly, both visual and semantic information affected a distributed network of independently identified category‐selective regions. Adaptation was seen a whole‐brain network of processing regions in response to visual similarity and semantic similarity; specifically, the angular gyrus (AnG) adapted to visual similarity and the dorsomedial prefrontal cortex (DMPFC) adapted to both visual and semantic similarity.
Conclusions Our findings suggest that perceptual features help organize mental categories throughout the object processing hierarchy. Most notably, visual similarity also influenced adaptation in nonvisual brain regions (i.e., AnG and DMPFC). We conclude that category‐relevant visual features are maintained in higher‐order conceptual representations and visual information plays an important role in both the acquisition and neural representation of conceptual object categories.
-
more » « less
Abstract Conscientiousness is a personality trait associated with many important life outcomes, but little is known about the mechanisms that underlie it. We investigated its neural correlates using functional connectivity analysis in fMRI, which identifies brain regions that act in synchrony. We tested the hypothesis that a broad network resembling a combination of the salience and ventral attention networks, which we provisionally label the goal priority network (GPN), is a neural correlate of Conscientiousness. Self‐ and peer‐ratings of Conscientiousness were collected in a community sample of adults who underwent a resting‐state fMRI scan (
N = 218). An independent components analysis yielded five components that overlapped substantially with the GPN. We examined synchrony within and between these GPN subcomponents. Synchrony within one of the components—mainly comprising regions of anterior insula, dorsal anterior cingulate cortex, and dorsolateral prefrontal cortex—was significantly associated with Conscientiousness. Connectivity between this component and the four other GPN components was also significantly associated with Conscientiousness. Our results support the hypothesis that variation in a network that enables prioritization of multiple goals may be central to Conscientiousness. -
more » « less
Abstract Objective Research has long sought to identify which individuals are best at accurately perceiving others' personalities or are
good judges , yet consistent predictors of this ability have been difficult to find. In the current studies, we revisit this question by examining a novel physiological correlate of social sensitivity, cardiac vagal flexibility, which reflects dynamic modulation of cardiac vagal control.Method We examined whether greater cardiac vagal flexibility was associated with forming more accurate personality impressions, defined as viewing targets more in line with their distinctive self‐reported profile of traits, in two studies, including a thin‐slice video perceptions study (
N = 109) and a dyadic interaction study (N = 175).Results Across studies, we found that individuals higher in vagal flexibility formed significantly more accurate first impressions of others' more observable personality traits (e.g., extraversion, creativity, warmth). These associations held while including a range of relevant covariates, including cardiac vagal tone, sympathetic activation, and gender.
Conclusion In sum, social sensitivity as indexed by cardiac vagal flexibility is linked to forming more accurate impressions of others' observable traits, shedding light on a characteristic that may help to identify the elusive good judge and providing insight into its neurobiological underpinnings.
-
more » « less
Abstract A cognitive map is an internal representation of the external world that guides flexible behavior in a complex environment. Cognitive map theory assumes that relationships between entities can be organized using Euclidean‐based coordinates. Previous studies revealed that cognitive map theory can also be generalized to inferences about abstract spaces, such as social spaces. However, it is still unclear whether humans can construct a cognitive map by combining relational knowledge between discrete entities with multiple abstract dimensions in nonsocial spaces. Here we asked subjects to learn to navigate a novel object space defined by two feature dimensions, price and abstraction. The subjects first learned the rank relationships between objects in each feature dimension and then completed a transitive inferences task. We recorded brain activity using functional magnetic resonance imaging (fMRI) while they performed the transitive inference task. By analyzing the behavioral data, we found that the Euclidean distance between objects had a significant effect on response time (RT). The longer the one‐dimensional rank distance and two‐dimensional (2D) Euclidean distance between objects the shorter the RT. The task‐fMRI data were analyzed using both univariate analysis and representational similarity analysis. We found that the hippocampus, entorhinal cortex, and medial orbitofrontal cortex were able to represent the Euclidean distance between objects in 2D space. Our findings suggest that relationship inferences between discrete objects can be made in a 2D nonsocial space and that the neural basis of this inference is related to cognitive maps.
