An official website of the United States government
Brain large-scale dynamics is constrained by the heterogeneity of intrinsic anatomical substrate. Little is known how the spatiotemporal dynamics adapt for the heterogeneous structural connectivity (SC). Modern neuroimaging modalities make it possible to study the intrinsic brain activity at the scale of seconds to minutes. Diffusion magnetic resonance imaging (dMRI) and functional MRI reveals the large-scale SC across different brain regions. Electrophysiological methods (i.e. MEG/EEG) provide direct measures of neural activity and exhibits complex neurobiological temporal dynamics which could not be solved by fMRI. However, most of existing multimodal analytical methods collapse the brain measurements either in space or time domain and fail to capture the spatio-temporal circuit dynamics. In this paper, we propose a novel spatio-temporal graph Transformer model to integrate the structural and functional connectivity in both spatial and temporal domain. The proposed method learns the heterogeneous node and graph representation via contrastive learning and multi-head attention based graph Transformer using multimodal brain data (i.e. fMRI, MRI, MEG and behavior performance). The proposed contrastive graph Transformer representation model incorporates the heterogeneity map constrained by T1-to-T2-weighted (T1w/T2w) to improve the model fit to structurefunction interactions. The experimental results with multimodal resting state brain measurements demonstrate the proposed method could highlight the local properties of large-scale brain spatio-temporal dynamics and capture the dependence strength between functional connectivity and behaviors. In summary, the proposed method enables the complex brain dynamics explanation for different modal variants.
more »
« less
DSAM: A deep learning framework for analyzing temporal and spatial dynamics in brain networks
Resting-state functional magnetic resonance imaging (rs-fMRI) is a noninvasive technique pivotal for understanding human neural mechanisms of intricate cognitive processes. Most rs-fMRI studies compute a single static functional connectivity matrix across brain regions of interest, or dynamic functional connectivity matrices with a sliding window approach. These approaches are at risk of oversimplifying brain dynamics and lack proper consideration of the goal at hand. While deep learning has gained substantial popularity for modeling complex relational data, its application to uncovering the spatiotemporal dynamics of the brain is still limited. In this study we propose a novel interpretable deep learning framework that learns goal-specific functional connectivity matrix directly from time series and employs a specialized graph neural network for the final classification. Our model, DSAM, leverages temporal causal convolutional networks to capture the temporal dynamics in both low- and high-level feature representations, a temporal attention unit to identify important time points, a self-attention unit to construct the goal-specific connectivity matrix, and a novel variant of graph neural network to capture the spatial dynamics for downstream classification. To validate our approach, we conducted experiments on the Human Connectome Project dataset with 1075 samples to build and interpret the model for the classification of sex group, and the Adolescent Brain Cognitive Development Dataset with 8520 samples for independent testing. Compared our proposed framework with other state-of-art models, results suggested this novel approach goes beyond the assumption of a fixed connectivity matrix, and provides evidence of goal-specific brain connectivity patterns, which opens up potential to gain deeper insights into how the human brain adapts its functional connectivity specific to the task at hand.
more »
« less
- Award ID(s):
- 2112455
- PAR ID:
- 10569590
- Publisher / Repository:
- Elsevier
- Date Published:
- Journal Name:
- Medical Image Analysis
- Volume:
- 101
- Issue:
- C
- ISSN:
- 1361-8415
- Page Range / eLocation ID:
- 103462
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Identifying Structural Brain Networks from Functional Connectivity: A Network Deconvolution ApproachWe address the problem of identifying structural brain networks from signals measured by resting-state functional magnetic resonance imaging (fMRI). To this end, we model functional brain activity as graph signals generated through a linear diffusion process on the unknown structural network. While this is admittedly an oversimplification of the complex mechanisms at work in the brain, recent studies have shown it is an accurate generative model for the second-order statistics of functional signals. We show the diffusion model implies that the signal covariance matrix (a.k.a. functional connectivity) is an unknown polynomial function of the structural network’s adjacency matrix. Accordingly, we advocate a network deconvolution approach whereby we: (i) use the fMRI signals to estimate the eigenvectors of the structural network from those of the empirical covariance; and (ii) solve a convex, sparsity-regularized inverse problem to recover the eigenvalues that were obscured by diffusion. The inferred structural networks capture some key patterns that match known pathology in attention deficit/hyper activity disorder. We also offer preliminary evidence supporting their role as potential biomarkers for subject diagnosis and classification.more » « less
-
Abstract In neuroimaging research, understanding the intricate dynamics of brain networks over time is paramount for unraveling the complexities of brain function. One approach commonly used to explore the dynamic nature of brain networks is functional connectivity analysis. However, while functional connectivity offers valuable insights, it fails to consider the diverse timescales of coupling between different brain regions. This gap in understanding leaves a significant aspect of brain dynamics unexplored in neuroimaging research. We propose an innovative approach that delves into the dynamic coupling/connectivity timescales of brain regions relative to one another, focusing on how brain region couplings stretch or shrink over time, rather than relying solely on functional connectivity measures. Our method introduces a novel metric called “warping elasticity,” which utilizes dynamic time warping (DTW) to capture the temporal nuances of connectivity. Unlike traditional methods, our approach allows for (potentially nonlinear) dynamic compression and expansion of the time series, offering a more intricate understanding of how coupling between brain regions evolves. Through the adaptive windows employed by the DTW method, we can effectively capture transient couplings within varying connectivity timescales of brain network pairs. In extensive evaluations, our method exhibits high replicability across subjects and diverse datasets, showcasing robustness against noise. More importantly, it uncovers statistically significant distinctions between healthy control (HC) and schizophrenia (SZ) groups through the identification of warp elasticity states. These states are cluster centroids, representing the warp elasticity across subjects and time, offering a novel perspective on the dynamic nature of brain connectivity, distinct from conventional metrics focused solely on functional connectivity. For instance, controls spend more time in a warp elasticity state characterized by timescale stretching of the visual domain relative to other domains, suggesting disruptions in the visual cortex. Conversely, patients show increased time spent in a warp elasticity state with stretching timescales in higher cognitive areas relative to sensory regions, indicative of prolonged cognitive processing of sensory input. Overall, our approach presents a promising avenue for investigating the temporal dynamics of brain network interactions in functional magnetic resonance imaging (fMRI) data. By focusing on the elasticity of connectivity timescales, rather than adhering to functional connectivity metrics, we pave the way for a deeper understanding of neuropsychiatric disorders in neuroscience research.more » « less
-
We propose a novel matrix autoencoder to map functional connectomes from resting state fMRI (rs-fMRI) to structural connectomes from Diffusion Tensor Imaging (DTI), as guided by subject-level phenotypic measures. Our specialized autoencoder infers a low dimensional manifold embedding for the rs-fMRI correlation matrices that mimics a canonical outer-product decomposition. The embedding is simultaneously used to reconstruct DTI tractography matrices via a second manifold alignment decoder and to predict inter-subject phenotypic variability via an artificial neural network. We validate our framework on a dataset of 275 healthy individuals from the Human Connectome Project database and on a second clinical dataset consisting of 57 subjects with Autism Spectrum Disorder. We demonstrate that the model reliably recovers structural connectivity patterns across individuals, while robustly extracting predictive and interpretable brain biomarkers in a cross-validated setting. Finally, our framework outperforms several baselines at predicting behavioral phenotypes in both real-world datasets.more » « less
-
Patterns of estimated neural activity derived from resting state functional magnetic resonance imaging (rs-fMRI) have been shown to predict a wide range of cognitive and behavioral outcomes in both normative and clinical populations. Yet, without links to established cognitive processes, the functional brain states associated with the resting brain will remain unexplained, and potentially confounded, markers of individual differences. In this work we demonstrate the application of multivoxel pattern classifiers (MVPCs) to predict the valence and arousal properties of spontaneous affect processing in the task-non-engaged resting state. rs-fMRI data were acquired from subjects that were held out from a subject set that underwent image-based affect induction concurrent with fMRI to train the MVPCs. We also validated these affective predictions against a well-established, independent measure of autonomic arousal, skin conductance response. These findings suggest a new neuroimaging methodology for resting state analysis in which models, trained on cognition-specific task-based fMRI acquired from well-matched cohorts, capably predict hidden cognitive processes operating within the resting brain.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1016/j.media.2025.103462
