An official website of the United States government
Category selectivity is a fundamental principle of organization of perceptual brain regions. Human occipitotemporal cortex is subdivided into areas that respond preferentially to faces, bodies, artifacts, and scenes. However, observers need to combine information about objects from different categories to form a coherent understanding of the world. How is this multicategory information encoded in the brain? Studying the multivariate interactions between brain regions of male and female human subjects with fMRI and artificial neural networks, we found that the angular gyrus shows joint statistical dependence with multiple category-selective regions. Adjacent regions show effects for the combination of scenes and each other category, suggesting that scenes provide a context to combine information about the world. Additional analyses revealed a cortical map of areas that encode information across different subsets of categories, indicating that multicategory information is not encoded in a single centralized location, but in multiple distinct brain regions. SIGNIFICANCE STATEMENTMany cognitive tasks require combining information about entities from different categories. However, visual information about different categorical objects is processed by separate, specialized brain regions. How is the joint representation from multiple category-selective regions implemented in the brain? Using fMRI movie data and state-of-the-art multivariate statistical dependence based on artificial neural networks, we identified the angular gyrus encoding responses across face-, body-, artifact-, and scene-selective regions. Further, we showed a cortical map of areas that encode information across different subsets of categories. These findings suggest that multicategory information is not encoded in a single centralized location, but at multiple cortical sites which might contribute to distinct cognitive functions, offering insights to understand integration in a variety of domains.
more »
« less
Region-specific fMRI dictionary for decoding face verification in humans
This paper focuses on decoding the process of face verification in the human brain using fMRI responses. 2400 fMRI responses are collected from different participants while they perform face verification on genuine and imposter stimuli face pairs. The first part of the paper analyzes the responses covering both cognitive and fMRI neuro-imaging results. With an average verification accuracy of 64.79% by human participants, the results of the cognitive analysis depict that the performance of female participants is significantly higher than the male participants with respect to imposter pairs. The results of the neuroimaging analysis identifies regions of the brain such as the left fusiform gyrus, caudate nucleus, and superior frontal gyrus that are activated when participants perform face verification tasks. The second part of the paper proposes a novel two-level fMRI dictionary learning approach to predict if the stimuli observed is genuine or imposter using the brain activation data for selected regions. A comparative analysis with existing machine learning techniques illustrates that the proposed approach yields at least 4.5% higher classification accuracy than other algorithms. It is envisioned that the result of this study is the first step in designing brain-inspired automatic face verification algorithms.
more »
« less
- PAR ID:
- 10053781
- Date Published:
- Journal Name:
- International Joint Conference on Biometrics (IJCB)
- Page Range / eLocation ID:
- 3814 to 3821
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract Naturalistic stimuli evoke strong, consistent, and information-rich patterns of brain activity, and engage large extents of the human brain. They allow researchers to compare highly similar brain responses across subjects, and to study how complex representations are encoded in brain activity. Here, we describe and share a dataset where 25 subjects watched part of the feature film “The Grand Budapest Hotel” by Wes Anderson. The movie has a large cast with many famous actors. Throughout the story, the camera shots highlight faces and expressions, which are fundamental to understand the complex narrative of the movie. This movie was chosen to sample brain activity specifically related to social interactions and face processing. This dataset provides researchers with fMRI data that can be used to explore social cognitive processes and face processing, adding to the existing neuroimaging datasets that sample brain activity with naturalistic movies.more » « less
-
Abstract Multimodal neuroimaging research plays a pivotal role in understanding the complexities of the human brain and its disorders. Independent component analysis (ICA) has emerged as a widely used and powerful tool for disentangling mixed independent sources, particularly in the analysis of functional magnetic resonance imaging (fMRI) data. This paper extends the use of ICA as a unifying framework for multimodal fusion, introducing a novel approach termed parallel multilink group joint ICA (pmg-jICA). The method allows for the fusion of gray matter maps from structural MRI (sMRI) data to multiple fMRI intrinsic networks, addressing the limitations of previous models. The effectiveness of pmg-jICA is demonstrated through its application to an Alzheimer’s dataset, yielding linked structure-function outputs for 53 brain networks. Our approach leverages the complementary information from various imaging modalities, providing a unique perspective on brain alterations in Alzheimer’s disease. The pmg-jICA identifies several components with significant differences between HC and AD groups including thalamus, caudate, putamen with in the subcortical (SC) domain, insula, parahippocampal gyrus within the cognitive control (CC) domain, and the lingual gyrus within the visual (VS) domain, providing localized insights into the links between AD and specific brain regions. In addition, because we link across multiple brain networks, we can also compute functional network connectivity (FNC) from spatial maps and subject loadings, providing a detailed exploration of the relationships between different brain regions and allowing us to visualize spatial patterns and loading parameters in sMRI along with intrinsic networks and FNC from the fMRI data. In essence, developed approach combines concepts from joint ICA and group ICA to provide a rich set of output characterizing data-driven links between covarying gray matter networks, and a (potentially large number of) resting fMRI networks allowing further study in the context of structure/function links. We demonstrate the utility of the approach by highlighting key structure/function disruptions in Alzheimer’s individuals.more » « less
-
Background Schizophrenia is a brain disorder characterized by diffuse, diverse, and wide-spread changes in gray matter volume (GM) and white matter structure (fractional anisotropy, FA), as well as cognitive impairments that greatly impact an individual’s quality of life. While the relationship of each of these image modalities and their links to schizophrenia status and cognitive impairment has been investigated separately, a multimodal fusion via parallel independent component analysis (pICA) affords the opportunity to explore the relationships between the changes in GM and FA, and the implications these network changes have on cognitive performance. Methods Images from 73 subjects with schizophrenia (SZ) and 82 healthy controls (HC) were drawn from an existing dataset. We investigated 12 components from each feature (FA and GM). Loading coefficients from the images were used to identify pairs of features that were significantly correlated and showed significant group differences between HC and SZ. MANCOVA analysis uncovered the relationships the identified spatial maps had with age, gender, and a global cognitive performance score. Results Three component pairs showed significant group differences (HC > SZ) in both gray and white matter measurements. Two of the component pairs identified networks of gray matter that drove significant relationships with cognition (HC > SZ) after accounting for age and gender. The gray and white matter structural networks identified in these three component pairs pull broadly from many regions, including the right and left thalamus, lateral occipital cortex, multiple regions of the middle temporal gyrus, precuneus cortex, postcentral gyrus, cingulate gyrus/cingulum, lingual gyrus, and brain stem. Conclusion The results of this multimodal analysis adds to our understanding of how the relationship between GM, FA, and cognition differs between HC and SZ by highlighting the correlated intermodal covariance of these structural networks and their differential relationships with cognitive performance. Previous unimodal research has found similar areas of GM and FA differences between these groups, and the cognitive deficits associated with SZ have been well documented. This study allowed us to evaluate the intercorrelated covariance of these structural networks and how these networks are involved the differences in cognitive performance between HC and SZ.more » « less
-
Multivariate pattern analysis (MVPA) of functional magnetic resonance imaging (fMRI) data has critically advanced the neuroanatomical understanding of affect processing in the human brain. Central to these advancements is the brain state, a temporally-succinct fMRI-derived pattern of neural activation, which serves as a processing unit. Establishing the brain state’s central role in affect processing, however, requires that it predicts multiple independent measures of affect. We employed MVPA-based regression to predict the valence and arousal properties of visual stimuli sampled from the International Affective Picture System (IAPS) along with the corollary skin conductance response (SCR) for demographically diverse healthy human participants (n = 19). We found that brain states significantly predicted the normative valence and arousal scores of the stimuli as well as the attendant individual SCRs. In contrast, SCRs significantly predicted arousal only. The prediction effect size of the brain state was more than three times greater than that of SCR. Moreover, neuroanatomical analysis of the regression parameters found remarkable agreement with regions long-established by fMRI univariate analyses in the emotion processing literature. Finally, geometric analysis of these parameters also found that the neuroanatomical encodings of valence and arousal are orthogonal as originally posited by the circumplex model of dimensional emotion.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/IJCNN.2017.7966337
