An official website of the United States government
It is becoming increasingly common to collect multiple related neuroimaging datasets either from different modalities or from different tasks and conditions. In addition, we have non-imaging data such as cognitive or behavioral variables, and it is through the association of these two sets of data—neuroimaging and non-neuroimaging—that we can understand and explain the evolution of neural and cognitive processes, and predict outcomes for intervention and treatment. Multiple methods for the joint analysis or fusion of multiple neuroimaging datasets or modalities exist; however, methods for the joint analysis of imaging and non-imaging data are still in their infancy. Current approaches for identifying brain networks related to cognitive assessments are still largely based on simple one-to-one correlation analyses and do not use the cross information available across multiple datasets. This work proposes two approaches based on independent vector analysis (IVA) to jointly analyze the imaging datasets and behavioral variables such that multivariate relationships across imaging data and behavioral features can be identified. The simulation results show that our proposed methods provide better accuracy in identifying associations across imaging and behavioral components than current approaches. With functional magnetic resonance imaging (fMRI) task data collected from 138 healthy controls and 109 patients with schizophrenia, results reveal that the central executive network (CEN) estimated in multiple datasets shows a strong correlation with the behavioral variable that measures working memory, a result that is not identified by traditional approaches. Most of the identified fMRI maps also show significant differences in activations across healthy controls and patients potentially providing a useful signature of mental disorders.
more »
« less
Dual-Attention Recurrent Networks for Affine Registration of Neuroimaging Data
Neuroimaging data typically undergoes several preprocessing steps before further analysis and mining can be done. Affine image registration is one of the important tasks during preprocessing. Recently, several image registration methods which are based on Convolutional Neural Networks have been proposed. However, due to the high computational and memory requirements of CNNs, these methods cannot be used in real-time for large neuroimaging data like fMRI. In this paper, we propose a Dual-Attention Recurrent Network (DRN) which uses a hard attention mechanism to allow the model to focus on small, but task-relevant, parts of the input image – thus reducing computational and memory costs. Furthermore, DRN naturally supports inhomogeneity between the raw input image (e.g., functional MRI) and the image we want to align it to (e.g., anatomical MRI) so it can be applied to harder registration tasks such as fMRI coregistration and normalization. Extensive experiments on two different datasets demonstrate that DRN significantly reduces the computational and memory costs compared with other neural network-based methods without sacrificing the quality of image registration
more »
« less
- Award ID(s):
- 1718310
- PAR ID:
- 10215776
- Date Published:
- Journal Name:
- Proceedings of the 2020 SIAM International Conference on Data Mining, SDM 2020
- Page Range / eLocation ID:
- 379-387
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Traditional tests of concept knowledge generate scores to assess how well a learner understands a concept. Here, we investigated whether patterns of brain activity collected during a concept knowledge task could be used to compute a neural ‘score’ to complement traditional scores of an individual’s conceptual understanding. Using a novel data-driven multivariate neuroimaging approach—informational network analysis—we successfully derived a neural score from patterns of activity across the brain that predicted individual differences in multiple concept knowledge tasks in the physics and engineering domain. These tasks include an fMRI paradigm, as well as two other previously validated concept inventories. The informational network score outperformed alternative neural scores computed using data-driven neuroimaging methods, including multivariate representational similarity analysis. This technique could be applied to quantify concept knowledge in a wide range of domains, including classroom-based education research, machine learning, and other areas of cognitive science.more » « less
-
Abstract Magnetic resonance imaging (MRI) is a technique that scans the anatomical structure of the brain, whereas functional magnetic resonance imaging (fMRI) uses the same basic principles of atomic physics as MRI scans but image metabolic function. A major goal of MRI and fMRI study is to precisely delineate various types of tissues, anatomical structure, pathologies, and detect the brain regions that react to outer stimuli (e.g., viewing an image). As a key feature of these MRI‐based neuroimaging data, voxels (cubic pixels of the brain volume) are highly correlated. However, the associations between voxels are often overlooked in the statistical analysis. We adapt a recently proposed dimension reduction method called the envelope method to analyze neuoimaging data taking into account correlation among voxels. We refer to the modified procedure the envelope chain procedure. Because the envelope chain procedure has not been employed before, we demonstrate in simulations the empirical performance of estimator, and examine its sensitivity when our assumptions are violated. We use the estimator to analyze the MRI data from ADHD‐200 study. Data analyses demonstrate that leveraging the correlations among voxels can significantly increase the efficiency of the regression analysis, thus achieving higher detection power with small sample sizes.more » « less
-
Deformable image registration (DIR) is an active research topic in biomedical imaging. There is a growing interest in developing DIR methods based on deep learning (DL). A traditional DL approach to DIR is based on training a convolutional neural network (CNN) to estimate the registration field between two input images. While conceptually simple, this approach comes with a limitation that it exclusively relies on a pre-trained CNN without explicitly enforcing fidelity between the registered image and the reference. We present plug-and-play image registration network (PIRATE) as a new DIR method that addresses this issue by integrating an explicit data-fidelity penalty and a CNN prior. PIRATE pre-trains a CNN denoiser on the registration field and "plugs" it into an iterative method as a regularizer. We additionally present PIRATE+ that fine-tunes the CNN prior in PIRATE using deep equilibrium models (DEQ). PIRATE+ interprets the fixed-point iteration of PIRATE as a network with effectively infinite layers and then trains the resulting network end-to-end, enabling it to learn more task-specific information and boosting its performance. Our numerical results on OASIS and CANDI datasets show that our methods achieve state-of-the-art performance on DIR.more » « less
-
Brain large-scale dynamics is constrained by the heterogeneity of intrinsic anatomical substrate. Little is known how the spatiotemporal dynamics adapt for the heterogeneous structural connectivity (SC). Modern neuroimaging modalities make it possible to study the intrinsic brain activity at the scale of seconds to minutes. Diffusion magnetic resonance imaging (dMRI) and functional MRI reveals the large-scale SC across different brain regions. Electrophysiological methods (i.e. MEG/EEG) provide direct measures of neural activity and exhibits complex neurobiological temporal dynamics which could not be solved by fMRI. However, most of existing multimodal analytical methods collapse the brain measurements either in space or time domain and fail to capture the spatio-temporal circuit dynamics. In this paper, we propose a novel spatio-temporal graph Transformer model to integrate the structural and functional connectivity in both spatial and temporal domain. The proposed method learns the heterogeneous node and graph representation via contrastive learning and multi-head attention based graph Transformer using multimodal brain data (i.e. fMRI, MRI, MEG and behavior performance). The proposed contrastive graph Transformer representation model incorporates the heterogeneity map constrained by T1-to-T2-weighted (T1w/T2w) to improve the model fit to structurefunction interactions. The experimental results with multimodal resting state brain measurements demonstrate the proposed method could highlight the local properties of large-scale brain spatio-temporal dynamics and capture the dependence strength between functional connectivity and behaviors. In summary, the proposed method enables the complex brain dynamics explanation for different modal variants.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1137/1.9781611976236.43
