Brain signals can be measured using multiple imaging modalities, such as magnetic resonance imaging (MRI)-based techniques. Different modalities convey distinct yet complementary information; thus, their joint analyses can provide valuable insight into how the brain functions in both healthy and diseased conditions. Data-driven approaches have proven most useful for multimodal fusion as they minimize assumptions imposed on the data, and there are a number of methods that have been developed to uncover relationships across modalities. However, none of these methods, to the best of our knowledge, can discover “one-to-many associations”, meaning one component from one modality is linked with more than one component from another modality. However, such “one-to-many associations” are likely to exist, since the same brain region can be involved in multiple neurological processes. Additionally, most existing data fusion methods require the signal subspace order to be identical for all modalities—a severe restriction for real-world data of different modalities. Here, we propose a new fusion technique—the consecutive independence and correlation transform (C-ICT) model—which successively performs independent component analysis and independent vector analysis and is uniquely flexible in terms of the number of datasets, signal subspace order, and the opportunity to find “one-to-many associations”. We apply C-ICT to fuse diffusion MRI, structural MRI, and functional MRI datasets collected from healthy controls (HCs) and patients with schizophrenia (SZs). We identify six interpretable triplets of components, each of which consists of three associated components from the three modalities. Besides, components from these triplets that show significant group differences between the HCs and SZs are identified, which could be seen as putative biomarkers in schizophrenia.
more »
« less
Association of Neuroimaging Data with Behavioral Variables: A Class of Multivariate Methods and Their Comparison Using Multi-Task FMRI Data
It is becoming increasingly common to collect multiple related neuroimaging datasets either from different modalities or from different tasks and conditions. In addition, we have non-imaging data such as cognitive or behavioral variables, and it is through the association of these two sets of data—neuroimaging and non-neuroimaging—that we can understand and explain the evolution of neural and cognitive processes, and predict outcomes for intervention and treatment. Multiple methods for the joint analysis or fusion of multiple neuroimaging datasets or modalities exist; however, methods for the joint analysis of imaging and non-imaging data are still in their infancy. Current approaches for identifying brain networks related to cognitive assessments are still largely based on simple one-to-one correlation analyses and do not use the cross information available across multiple datasets. This work proposes two approaches based on independent vector analysis (IVA) to jointly analyze the imaging datasets and behavioral variables such that multivariate relationships across imaging data and behavioral features can be identified. The simulation results show that our proposed methods provide better accuracy in identifying associations across imaging and behavioral components than current approaches. With functional magnetic resonance imaging (fMRI) task data collected from 138 healthy controls and 109 patients with schizophrenia, results reveal that the central executive network (CEN) estimated in multiple datasets shows a strong correlation with the behavioral variable that measures working memory, a result that is not identified by traditional approaches. Most of the identified fMRI maps also show significant differences in activations across healthy controls and patients potentially providing a useful signature of mental disorders.
more »
« less
- NSF-PAR ID:
- 10331816
- Date Published:
- Journal Name:
- Sensors
- Volume:
- 22
- Issue:
- 3
- ISSN:
- 1424-8220
- Page Range / eLocation ID:
- 1224
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Analysis of time-evolving data is crucial to understand the functioning of dynamic systems such as the brain. For instance, analysis of functional magnetic resonance imaging (fMRI) data collected during a task may reveal spatial regions of interest, and how they evolve during the task. However, capturing underlying spatial patterns as well as their change in time is challenging. The traditional approach in fMRI data analysis is to assume that underlying spatial regions of interest are static. In this article, using fractional amplitude of low-frequency fluctuations (fALFF) as an effective way to summarize the variability in fMRI data collected during a task, we arrange time-evolving fMRI data as a subjects by voxels by time windows tensor, and analyze the tensor using a tensor factorization-based approach called a PARAFAC2 model to reveal spatial dynamics. The PARAFAC2 model jointly analyzes data from multiple time windows revealing subject-mode patterns, evolving spatial regions (also referred to as networks) and temporal patterns. We compare the PARAFAC2 model with matrix factorization-based approaches relying on independent components, namely, joint independent component analysis (ICA) and independent vector analysis (IVA), commonly used in neuroimaging data analysis. We assess the performance of the methods in terms of capturing evolving networks through extensive numerical experiments demonstrating their modeling assumptions. In particular, we show that (i) PARAFAC2 provides a compact representation in all modes, i.e., subjects, time , and voxels , revealing temporal patterns as well as evolving spatial networks, (ii) joint ICA is as effective as PARAFAC2 in terms of revealing evolving networks but does not reveal temporal patterns, (iii) IVA's performance depends on sample size, data distribution and covariance structure of underlying networks. When these assumptions are satisfied, IVA is as accurate as the other methods, (iv) when subject-mode patterns differ from one time window to another, IVA is the most accurate. Furthermore, we analyze real fMRI data collected during a sensory motor task, and demonstrate that a component indicating statistically significant group difference between patients with schizophrenia and healthy controls is captured, which includes primary and secondary motor regions, cerebellum, and temporal lobe, revealing a meaningful spatial map and its temporal change.more » « less
-
more » « less
Abstract There is growing evidence that rather than using a single brain imaging modality to study its association with physiological or symptomatic features, the field is paying more attention to fusion of multimodal information. However, most current multimodal fusion approaches that incorporate functional magnetic resonance imaging (fMRI) are restricted to second‐level 3D features, rather than the original 4D fMRI data. This trade‐off is that the valuable temporal information is not utilized during the fusion step. Here we are motivated to propose a novel approach called “parallel group ICA+ICA” that incorporates temporal fMRI information from group independent component analysis (GICA) into a parallel independent component analysis (ICA) framework, aiming to enable direct fusion of first‐level fMRI features with other modalities (e.g., structural MRI), which thus can detect linked functional network variability and structural covariations. Simulation results show that the proposed method yields accurate intermodality linkage detection regardless of whether it is strong or weak. When applied to real data, we identified one pair of significantly associated fMRI‐sMRI components that show group difference between schizophrenia and controls in both modalities, and this linkage can be replicated in an independent cohort. Finally, multiple cognitive domain scores can be predicted by the features identified in the linked component pair by our proposed method. We also show these multimodal brain features can predict multiple cognitive scores in an independent cohort. Overall, results demonstrate the ability of parallel GICA+ICA to estimate joint information from 4D and 3D data without discarding much of the available information up front, and the potential for using this approach to identify imaging biomarkers to study brain disorders.
-
more » « less
Abstract Face perception is a fundamental aspect of human social interaction, yet most research on this topic has focused on single modalities and specific aspects of face perception. Here, we present a comprehensive multimodal dataset for examining facial emotion perception and judgment. This dataset includes EEG data from 97 unique neurotypical participants across 8 experiments, fMRI data from 19 neurotypical participants, single-neuron data from 16 neurosurgical patients (22 sessions), eye tracking data from 24 neurotypical participants, behavioral and eye tracking data from 18 participants with ASD and 15 matched controls, and behavioral data from 3 rare patients with focal bilateral amygdala lesions. Notably, participants from all modalities performed the same task. Overall, this multimodal dataset provides a comprehensive exploration of facial emotion perception, emphasizing the importance of integrating multiple modalities to gain a holistic understanding of this complex cognitive process. This dataset serves as a key missing link between human neuroimaging and neurophysiology literature, and facilitates the study of neuropsychiatric populations.
-
In a number of applications, one has access to high-dimensional time series data on several related subjects. A motivating application area comes from the neuroimaging field, such as brain fMRI time series data, obtained from various groups of subjects (cases/controls) with a specific neurological disorder. The problem of regularized joint estimation of multiple related Vector Autoregressive (VAR) models is discussed, leveraging a group lasso penalty in addition to a regular lasso one, so as to increase statistical efficiency of the estimates by borrowing strength across the models. A modeling framework is developed that it allows for both group-level and subject-specific effects for related subjects, using a group lasso penalty to estimate the former. An estimation procedure is introduced, whose performance is illustrated on synthetic data and compared to other state-of-the-art methods. Moreover, the proposed approach is employed for the analysis of resting state fMRI data. In particular, a group-level descriptive analysis is conducted for brain inter-regional temporal effects of Attention Deficit Hyperactive Disorder (ADHD) patients as opposed to controls, with the data available from the ADHD-200 Global Competition repository.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/s22031224
