There is growing evidence that rather than using a single brain imaging modality to study its association with physiological or symptomatic features, the field is paying more attention to fusion of multimodal information. However, most current multimodal fusion approaches that incorporate functional magnetic resonance imaging (fMRI) are restricted to second‐level 3D features, rather than the original 4D fMRI data. This trade‐off is that the valuable temporal information is not utilized during the fusion step. Here we are motivated to propose a novel approach called “parallel group ICA+ICA” that incorporates temporal fMRI information from group independent component analysis (GICA) into a parallel independent component analysis (ICA) framework, aiming to enable direct fusion of first‐level fMRI features with other modalities (e.g., structural MRI), which thus can detect linked functional network variability and structural covariations. Simulation results show that the proposed method yields accurate intermodality linkage detection regardless of whether it is strong or weak. When applied to real data, we identified one pair of significantly associated fMRI‐sMRI components that show group difference between schizophrenia and controls in both modalities, and this linkage can be replicated in an independent cohort. Finally, multiple cognitive domain scores can be predicted by the features identified in the linked component pair by our proposed method. We also show these multimodal brain features can predict multiple cognitive scores in an independent cohort. Overall, results demonstrate the ability of parallel GICA+ICA to estimate joint information from 4D and 3D data without discarding much of the available information up front, and the potential for using this approach to identify imaging biomarkers to study brain disorders.
In this article, we focus on estimating the joint relationship between structural magnetic resonance imaging (sMRI) gray matter (GM), and multiple functional MRI (fMRI) intrinsic connectivity networks (ICNs). To achieve this, we propose a multilink joint independent component analysis (ml‐jICA) method using the same core algorithm as jICA. To relax the jICA assumption, we propose another extension called parallel multilink jICA (pml‐jICA) that allows for a more balanced weight distribution over ml‐jICA/jICA. We assume a shared mixing matrix for both the sMRI and fMRI modalities, while allowing for different mixing matrices linking the sMRI data to the different ICNs. We introduce the model and then apply this approach to study the differences in resting fMRI and sMRI data from patients with Alzheimer's disease (AD) versus controls. The results of the pml‐jICA yield significant differences with large effect sizes that include regions in overlapping portions of default mode network, and also hippocampus and thalamus. Importantly, we identify two joint components with partially overlapping regions which show opposite effects for AD versus controls, but were able to be separated due to being linked to distinct functional and structural patterns. This highlights the unique strength of our approach and multimodal fusion approaches generally in revealing potentially biomarkers of brain disorders that would likely be missed by a unimodal approach. These results represent the first work linking multiple fMRI ICNs to GM components within a multimodal data fusion model and challenges the typical view that brain structure is more sensitive to AD than fMRI.
more » « less- NSF-PAR ID:
- 10467811
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Human Brain Mapping
- Volume:
- 44
- Issue:
- 15
- ISSN:
- 1065-9471
- Format(s):
- Medium: X Size: p. 5167-5179
- Size(s):
- ["p. 5167-5179"]
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract -
more » « less
Abstract Acquisition of multimodal brain imaging data for the same subject has become more common leading to a growing interest in determining the intermodal relationships between imaging modalities to further elucidate the pathophysiology of schizophrenia. Multimodal data have previously been individually analyzed and subsequently integrated; however, these analysis techniques lack the ability to examine true modality inter‐relationships. The utilization of a multiset canonical correlation and joint independent component analysis (mCCA + jICA) model for data fusion allows shared or distinct abnormalities between modalities to be examined. In this study, first‐episode schizophrenia patients (nSZ=19) and matched controls (nHC=21) completed a resting‐state functional magnetic resonance imaging (fMRI) scan at 7 T. Grey matter (GM), white matter (WM), cerebrospinal fluid (CSF), and amplitude of low frequency fluctuation (ALFF) maps were used as features in a mCCA + jICA model. Results of the mCCA + jICA model indicated three joint group‐discriminating components (GM‐CSF, WM‐ALFF, GM‐ALFF) and two modality‐unique group‐discriminating components (GM, WM). The joint component findings are highlighted by GM basal ganglia, somatosensory, parietal lobe, and thalamus abnormalities associated with ventricular CSF volume; WM occipital and frontal lobe abnormalities associated with temporal lobe function; and GM frontal, temporal, parietal, and occipital lobe abnormalities associated with caudate function. These results support and extend major findings throughout the literature using independent single modality analyses. The multimodal fusion of 7 T data in this study provides a more comprehensive illustration of the relationships between underlying neuronal abnormalities associated with schizophrenia than examination of imaging data independently.
-
Background Schizophrenia is a brain disorder characterized by diffuse, diverse, and wide-spread changes in gray matter volume (GM) and white matter structure (fractional anisotropy, FA), as well as cognitive impairments that greatly impact an individual’s quality of life. While the relationship of each of these image modalities and their links to schizophrenia status and cognitive impairment has been investigated separately, a multimodal fusion via parallel independent component analysis (pICA) affords the opportunity to explore the relationships between the changes in GM and FA, and the implications these network changes have on cognitive performance. Methods Images from 73 subjects with schizophrenia (SZ) and 82 healthy controls (HC) were drawn from an existing dataset. We investigated 12 components from each feature (FA and GM). Loading coefficients from the images were used to identify pairs of features that were significantly correlated and showed significant group differences between HC and SZ. MANCOVA analysis uncovered the relationships the identified spatial maps had with age, gender, and a global cognitive performance score. Results Three component pairs showed significant group differences (HC > SZ) in both gray and white matter measurements. Two of the component pairs identified networks of gray matter that drove significant relationships with cognition (HC > SZ) after accounting for age and gender. The gray and white matter structural networks identified in these three component pairs pull broadly from many regions, including the right and left thalamus, lateral occipital cortex, multiple regions of the middle temporal gyrus, precuneus cortex, postcentral gyrus, cingulate gyrus/cingulum, lingual gyrus, and brain stem. Conclusion The results of this multimodal analysis adds to our understanding of how the relationship between GM, FA, and cognition differs between HC and SZ by highlighting the correlated intermodal covariance of these structural networks and their differential relationships with cognitive performance. Previous unimodal research has found similar areas of GM and FA differences between these groups, and the cognitive deficits associated with SZ have been well documented. This study allowed us to evaluate the intercorrelated covariance of these structural networks and how these networks are involved the differences in cognitive performance between HC and SZ.more » « less
-
Brain signals can be measured using multiple imaging modalities, such as magnetic resonance imaging (MRI)-based techniques. Different modalities convey distinct yet complementary information; thus, their joint analyses can provide valuable insight into how the brain functions in both healthy and diseased conditions. Data-driven approaches have proven most useful for multimodal fusion as they minimize assumptions imposed on the data, and there are a number of methods that have been developed to uncover relationships across modalities. However, none of these methods, to the best of our knowledge, can discover “one-to-many associations”, meaning one component from one modality is linked with more than one component from another modality. However, such “one-to-many associations” are likely to exist, since the same brain region can be involved in multiple neurological processes. Additionally, most existing data fusion methods require the signal subspace order to be identical for all modalities—a severe restriction for real-world data of different modalities. Here, we propose a new fusion technique—the consecutive independence and correlation transform (C-ICT) model—which successively performs independent component analysis and independent vector analysis and is uniquely flexible in terms of the number of datasets, signal subspace order, and the opportunity to find “one-to-many associations”. We apply C-ICT to fuse diffusion MRI, structural MRI, and functional MRI datasets collected from healthy controls (HCs) and patients with schizophrenia (SZs). We identify six interpretable triplets of components, each of which consists of three associated components from the three modalities. Besides, components from these triplets that show significant group differences between the HCs and SZs are identified, which could be seen as putative biomarkers in schizophrenia.more » « less
-
more » « less
Abstract The study of human brain connectivity, including structural connectivity (SC) and functional connectivity (FC), provides insights into the neurophysiological mechanism of brain function and its relationship to human behavior and cognition. Both types of connectivity measurements provide crucial yet complementary information. However, integrating these two modalities into a single framework remains a challenge, because of the differences in their quantitative interdependencies as well as their anatomical representations due to distinctive imaging mechanisms. In this study, we introduced a new method, joint connectivity matrix independent component analysis (cmICA), which provides a data‐driven parcellation and automated‐linking of SC and FC information simultaneously using a joint analysis of functional magnetic resonance imaging (MRI) and diffusion‐weighted MRI data. We showed that these two connectivity modalities produce common cortical segregation, though with various degrees of (dis)similarity. Moreover, we show conjoint FC networks and structural white matter tracts that directly link these cortical parcellations/sources, within one analysis. Overall, data‐driven joint cmICA provides a new approach for integrating or fusing structural connectivity and FC systematically and conveniently, and provides an effective tool for connectivity‐based multimodal data fusion in brain.
