This content will become publicly available on October 4, 2025
- PAR ID:
- 10548265
- Publisher / Repository:
- Springer, Cham
- Date Published:
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
Brain large-scale dynamics is constrained by the heterogeneity of intrinsic anatomical substrate. Little is known how the spatiotemporal dynamics adapt for the heterogeneous structural connectivity (SC). Modern neuroimaging modalities make it possible to study the intrinsic brain activity at the scale of seconds to minutes. Diffusion magnetic resonance imaging (dMRI) and functional MRI reveals the large-scale SC across different brain regions. Electrophysiological methods (i.e. MEG/EEG) provide direct measures of neural activity and exhibits complex neurobiological temporal dynamics which could not be solved by fMRI. However, most of existing multimodal analytical methods collapse the brain measurements either in space or time domain and fail to capture the spatio-temporal circuit dynamics. In this paper, we propose a novel spatio-temporal graph Transformer model to integrate the structural and functional connectivity in both spatial and temporal domain. The proposed method learns the heterogeneous node and graph representation via contrastive learning and multi-head attention based graph Transformer using multimodal brain data (i.e. fMRI, MRI, MEG and behavior performance). The proposed contrastive graph Transformer representation model incorporates the heterogeneity map constrained by T1-to-T2-weighted (T1w/T2w) to improve the model fit to structurefunction interactions. The experimental results with multimodal resting state brain measurements demonstrate the proposed method could highlight the local properties of large-scale brain spatio-temporal dynamics and capture the dependence strength between functional connectivity and behaviors. In summary, the proposed method enables the complex brain dynamics explanation for different modal variants.more » « less
-
We propose a novel matrix autoencoder to map functional connectomes from resting state fMRI (rs-fMRI) to structural connectomes from Diffusion Tensor Imaging (DTI), as guided by subject-level phenotypic measures. Our specialized autoencoder infers a low dimensional manifold embedding for the rs-fMRI correlation matrices that mimics a canonical outer-product decomposition. The embedding is simultaneously used to reconstruct DTI tractography matrices via a second manifold alignment decoder and to predict inter-subject phenotypic variability via an artificial neural network. We validate our framework on a dataset of 275 healthy individuals from the Human Connectome Project database and on a second clinical dataset consisting of 57 subjects with Autism Spectrum Disorder. We demonstrate that the model reliably recovers structural connectivity patterns across individuals, while robustly extracting predictive and interpretable brain biomarkers in a cross-validated setting. Finally, our framework outperforms several baselines at predicting behavioral phenotypes in both real-world datasets.more » « less
-
Abstract There are a growing number of neuroimaging studies motivating joint structural and functional brain connectivity. Brain connectivity of different modalities provides insight into brain functional organization by leveraging complementary information, especially for brain disorders such as schizophrenia. In this paper, we propose a multi-modal independent component analysis (ICA) model that utilizes information from both structural and functional brain connectivity guided by spatial maps to estimate intrinsic connectivity networks (ICNs). Structural connectivity is estimated through whole-brain tractography on diffusion-weighted MRI (dMRI), while functional connectivity is derived from resting-state functional MRI (rs-fMRI). The proposed structural-functional connectivity and spatially constrained ICA (sfCICA) model estimates ICNs at the subject level using a multi-objective optimization framework. We evaluated our model using synthetic and real datasets (including dMRI and rs-fMRI from 149 schizophrenia patients and 162 controls). Multi-modal ICNs revealed enhanced functional coupling between ICNs with higher structural connectivity, improved modularity, and network distinction, particularly in schizophrenia. Statistical analysis of group differences showed more significant differences in the proposed model compared to the unimodal model. In summary, the sfCICA model showed benefits from being jointly informed by structural and functional connectivity. These findings suggest advantages in simultaneously learning effectively and enhancing connectivity estimates using structural connectivity.
-
Summary The human brain is a directional network system, in which brain regions are network nodes and the influence exerted by one region on another is a network edge. We refer to this directional information flow from one region to another as directional connectivity. Seizures arise from an epileptic directional network; abnormal neuronal activities start from a seizure onset zone and propagate via a network to otherwise healthy brain regions. As such, effective epilepsy diagnosis and treatment require accurate identification of directional connections among regions, i.e., mapping of epileptic patients’ brain networks. This article aims to understand the epileptic brain network using intracranial electroencephalographic data—recordings of epileptic patients’ brain activities in many regions. The most popular models for directional connectivity use ordinary differential equations (ODE). However, ODE models are sensitive to data noise and computationally costly. To address these issues, we propose a high-dimensional state-space multivariate autoregression (SSMAR) model for the brain’s directional connectivity. Different from standard multivariate autoregression and SSMAR models, the proposed SSMAR features a cluster structure, where the brain network consists of several clusters of densely connected brain regions. We develop an expectation–maximization algorithm to estimate the proposed model and use it to map the interregional networks of epileptic patients in different seizure stages. Our method reveals the evolution of brain networks during seizure development.
-
Resting-state functional magnetic resonance imaging (rs-fMRI) is a noninvasive technique pivotal for understanding human neural mechanisms of intricate cognitive processes. Most rs-fMRI studies compute a single static functional connectivity matrix across brain regions of interest, or dynamic functional connectivity matrices with a sliding window approach. These approaches are at risk of oversimplifying brain dynamics and lack proper consideration of the goal at hand. While deep learning has gained substantial popularity for modeling complex relational data, its application to uncovering the spatiotemporal dynamics of the brain is still limited. In this study we propose a novel interpretable deep learning framework that learns goal-specific functional connectivity matrix directly from time series and employs a specialized graph neural network for the final classification. Our model, DSAM, leverages temporal causal convolutional networks to capture the temporal dynamics in both low- and high-level feature representations, a temporal attention unit to identify important time points, a self-attention unit to construct the goal-specific connectivity matrix, and a novel variant of graph neural network to capture the spatial dynamics for downstream classification. To validate our approach, we conducted experiments on the Human Connectome Project dataset with 1075 samples to build and interpret the model for the classification of sex group, and the Adolescent Brain Cognitive Development Dataset with 8520 samples for independent testing. Compared our proposed framework with other state-of-art models, results suggested this novel approach goes beyond the assumption of a fixed connectivity matrix, and provides evidence of goal-specific brain connectivity patterns, which opens up potential to gain deeper insights into how the human brain adapts its functional connectivity specific to the task at hand.more » « less