It is becoming increasingly common to collect multiple related neuroimaging datasets either from different modalities or from different tasks and conditions. In addition, we have non-imaging data such as cognitive or behavioral variables, and it is through the association of these two sets of data—neuroimaging and non-neuroimaging—that we can understand and explain the evolution of neural and cognitive processes, and predict outcomes for intervention and treatment. Multiple methods for the joint analysis or fusion of multiple neuroimaging datasets or modalities exist; however, methods for the joint analysis of imaging and non-imaging data are still in their infancy. Current approaches for identifying brain networks related to cognitive assessments are still largely based on simple one-to-one correlation analyses and do not use the cross information available across multiple datasets. This work proposes two approaches based on independent vector analysis (IVA) to jointly analyze the imaging datasets and behavioral variables such that multivariate relationships across imaging data and behavioral features can be identified. The simulation results show that our proposed methods provide better accuracy in identifying associations across imaging and behavioral components than current approaches. With functional magnetic resonance imaging (fMRI) task data collected from 138 healthy controls and 109 patients with schizophrenia, results reveal that the central executive network (CEN) estimated in multiple datasets shows a strong correlation with the behavioral variable that measures working memory, a result that is not identified by traditional approaches. Most of the identified fMRI maps also show significant differences in activations across healthy controls and patients potentially providing a useful signature of mental disorders.
more »
« less
Consecutive Independence and Correlation Transform for Multimodal Data Fusion: Discovery of One-to-Many Associations in Structural and Functional Imaging Data
Brain signals can be measured using multiple imaging modalities, such as magnetic resonance imaging (MRI)-based techniques. Different modalities convey distinct yet complementary information; thus, their joint analyses can provide valuable insight into how the brain functions in both healthy and diseased conditions. Data-driven approaches have proven most useful for multimodal fusion as they minimize assumptions imposed on the data, and there are a number of methods that have been developed to uncover relationships across modalities. However, none of these methods, to the best of our knowledge, can discover “one-to-many associations”, meaning one component from one modality is linked with more than one component from another modality. However, such “one-to-many associations” are likely to exist, since the same brain region can be involved in multiple neurological processes. Additionally, most existing data fusion methods require the signal subspace order to be identical for all modalities—a severe restriction for real-world data of different modalities. Here, we propose a new fusion technique—the consecutive independence and correlation transform (C-ICT) model—which successively performs independent component analysis and independent vector analysis and is uniquely flexible in terms of the number of datasets, signal subspace order, and the opportunity to find “one-to-many associations”. We apply C-ICT to fuse diffusion MRI, structural MRI, and functional MRI datasets collected from healthy controls (HCs) and patients with schizophrenia (SZs). We identify six interpretable triplets of components, each of which consists of three associated components from the three modalities. Besides, components from these triplets that show significant group differences between the HCs and SZs are identified, which could be seen as putative biomarkers in schizophrenia.
more »
« less
- Award ID(s):
- 1631838
- NSF-PAR ID:
- 10388431
- Date Published:
- Journal Name:
- Applied Sciences
- Volume:
- 11
- Issue:
- 18
- ISSN:
- 2076-3417
- Page Range / eLocation ID:
- 8382
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract There is growing evidence that rather than using a single brain imaging modality to study its association with physiological or symptomatic features, the field is paying more attention to fusion of multimodal information. However, most current multimodal fusion approaches that incorporate functional magnetic resonance imaging (fMRI) are restricted to second‐level 3D features, rather than the original 4D fMRI data. This trade‐off is that the valuable temporal information is not utilized during the fusion step. Here we are motivated to propose a novel approach called “parallel group ICA+ICA” that incorporates temporal fMRI information from group independent component analysis (GICA) into a parallel independent component analysis (ICA) framework, aiming to enable direct fusion of first‐level fMRI features with other modalities (e.g., structural MRI), which thus can detect linked functional network variability and structural covariations. Simulation results show that the proposed method yields accurate intermodality linkage detection regardless of whether it is strong or weak. When applied to real data, we identified one pair of significantly associated fMRI‐sMRI components that show group difference between schizophrenia and controls in both modalities, and this linkage can be replicated in an independent cohort. Finally, multiple cognitive domain scores can be predicted by the features identified in the linked component pair by our proposed method. We also show these multimodal brain features can predict multiple cognitive scores in an independent cohort. Overall, results demonstrate the ability of parallel GICA+ICA to estimate joint information from 4D and 3D data without discarding much of the available information up front, and the potential for using this approach to identify imaging biomarkers to study brain disorders.
-
more » « less
Abstract Acquisition of multimodal brain imaging data for the same subject has become more common leading to a growing interest in determining the intermodal relationships between imaging modalities to further elucidate the pathophysiology of schizophrenia. Multimodal data have previously been individually analyzed and subsequently integrated; however, these analysis techniques lack the ability to examine true modality inter‐relationships. The utilization of a multiset canonical correlation and joint independent component analysis (mCCA + jICA) model for data fusion allows shared or distinct abnormalities between modalities to be examined. In this study, first‐episode schizophrenia patients (nSZ=19) and matched controls (nHC=21) completed a resting‐state functional magnetic resonance imaging (fMRI) scan at 7 T. Grey matter (GM), white matter (WM), cerebrospinal fluid (CSF), and amplitude of low frequency fluctuation (ALFF) maps were used as features in a mCCA + jICA model. Results of the mCCA + jICA model indicated three joint group‐discriminating components (GM‐CSF, WM‐ALFF, GM‐ALFF) and two modality‐unique group‐discriminating components (GM, WM). The joint component findings are highlighted by GM basal ganglia, somatosensory, parietal lobe, and thalamus abnormalities associated with ventricular CSF volume; WM occipital and frontal lobe abnormalities associated with temporal lobe function; and GM frontal, temporal, parietal, and occipital lobe abnormalities associated with caudate function. These results support and extend major findings throughout the literature using independent single modality analyses. The multimodal fusion of 7 T data in this study provides a more comprehensive illustration of the relationships between underlying neuronal abnormalities associated with schizophrenia than examination of imaging data independently.
-
Background: Schizophrenia affects around 1% of the global population. Functional connectivity extracted from resting-state functional magnetic resonance imaging (rs-fMRI) has previously been used to study schizophrenia and has great potential to provide novel insights into the disorder. Some studies have shown abnormal functional connectivity in the default mode network (DMN) of individuals with schizophrenia, and more recent studies have shown abnormal dynamic functional connectivity (dFC) in individuals with schizophrenia. However, DMN dFC and the link between abnormal DMN dFC and symptom severity have not been well-characterized. Method: Resting-state fMRI data from subjects with schizophrenia (SZ) and healthy controls (HC) across two datasets were analyzed independently. We captured seven maximally independent subnodes in the DMN by applying group independent component analysis and estimated dFC between subnode time courses using a sliding window approach. A clustering method separated the dFCs into five reoccurring brain states. A feature selection method modeled the difference between SZs and HCs using the state-specific FC features. Finally, we used the transition probability of a hidden Markov model to characterize the link between symptom severity and dFC in SZ subjects. Results: We found decreases in the connectivity of the anterior cingulate cortex (ACC) and increases in the connectivity between the precuneus (PCu) and the posterior cingulate cortex (PCC) (i.e., PCu/PCC) of SZ subjects. In SZ, the transition probability from a state with weaker PCu/PCC and stronger ACC connectivity to a state with stronger PCu/PCC and weaker ACC connectivity increased with symptom severity. Conclusions: To our knowledge, this was the first study to investigate DMN dFC and its link to schizophrenia symptom severity. We identified reproducible neural states in a data-driven manner and demonstrated that the strength of connectivity within those states differed between SZs and HCs. Additionally, we identified a relationship between SZ symptom severity and the dynamics of DMN functional connectivity. We validated our results across two datasets. These results support the potential of dFC for use as a biomarker of schizophrenia and shed new light upon the relationship between schizophrenia and DMN dynamics.more » « less
-
Background Schizophrenia is a brain disorder characterized by diffuse, diverse, and wide-spread changes in gray matter volume (GM) and white matter structure (fractional anisotropy, FA), as well as cognitive impairments that greatly impact an individual’s quality of life. While the relationship of each of these image modalities and their links to schizophrenia status and cognitive impairment has been investigated separately, a multimodal fusion via parallel independent component analysis (pICA) affords the opportunity to explore the relationships between the changes in GM and FA, and the implications these network changes have on cognitive performance. Methods Images from 73 subjects with schizophrenia (SZ) and 82 healthy controls (HC) were drawn from an existing dataset. We investigated 12 components from each feature (FA and GM). Loading coefficients from the images were used to identify pairs of features that were significantly correlated and showed significant group differences between HC and SZ. MANCOVA analysis uncovered the relationships the identified spatial maps had with age, gender, and a global cognitive performance score. Results Three component pairs showed significant group differences (HC > SZ) in both gray and white matter measurements. Two of the component pairs identified networks of gray matter that drove significant relationships with cognition (HC > SZ) after accounting for age and gender. The gray and white matter structural networks identified in these three component pairs pull broadly from many regions, including the right and left thalamus, lateral occipital cortex, multiple regions of the middle temporal gyrus, precuneus cortex, postcentral gyrus, cingulate gyrus/cingulum, lingual gyrus, and brain stem. Conclusion The results of this multimodal analysis adds to our understanding of how the relationship between GM, FA, and cognition differs between HC and SZ by highlighting the correlated intermodal covariance of these structural networks and their differential relationships with cognitive performance. Previous unimodal research has found similar areas of GM and FA differences between these groups, and the cognitive deficits associated with SZ have been well documented. This study allowed us to evaluate the intercorrelated covariance of these structural networks and how these networks are involved the differences in cognitive performance between HC and SZ.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/app11188382
