skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Unified Embeddings of Structural and Functional Connectome via a Function-Constrained Structural Graph Variational Auto-Encoder
Title: Unified Embeddings of Structural and Functional Connectome via a Function-Constrained Structural Graph Variational Auto-Encoder
Graph theoretical analyses have become standard tools in modeling functional and anatomical connectivity in the brain. With the advent of connectomics, the primary graphs or networks of interest are structural connectome (derived from DTI tractography) and functional connectome (derived from resting-state fMRI). However, most published connectome studies have focused on either structural or functional connectome, yet complementary information between them, when available in the same dataset, can be jointly leveraged to improve our understanding of the brain. To this end, we propose a function-constrained structural graph variational autoencoder (FCS-GVAE) capable of incorporating information from both functional and structural connectome in an unsupervised fashion. This leads to a joint low-dimensional embedding that establishes a unified spatial coordinate system for comparing across different subjects. We evaluate our approach using the publicly available OASIS-3 Alzheimer’s disease (AD) dataset and show that a variational formulation is necessary to optimally encode functional brain dynamics. Further, the proposed joint embedding approach can more accurately distinguish different patient sub-populations than approaches that do not use complementary connectome information.  more » « less
Award ID(s):
2045848
PAR ID:
10359063
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. Lecture Notes in Computer Science.
Volume:
13431
Page Range / eLocation ID:
406–415
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; (, Network Neuroscience)
    Abstract There are a growing number of neuroimaging studies motivating joint structural and functional brain connectivity. Brain connectivity of different modalities provides insight into brain functional organization by leveraging complementary information, especially for brain disorders such as schizophrenia. In this paper, we propose a multi-modal independent component analysis (ICA) model that utilizes information from both structural and functional brain connectivity guided by spatial maps to estimate intrinsic connectivity networks (ICNs). Structural connectivity is estimated through whole-brain tractography on diffusion-weighted MRI (dMRI), while functional connectivity is derived from resting-state functional MRI (rs-fMRI). The proposed structural-functional connectivity and spatially constrained ICA (sfCICA) model estimates ICNs at the subject level using a multi-objective optimization framework. We evaluated our model using synthetic and real datasets (including dMRI and rs-fMRI from 149 schizophrenia patients and 162 controls). Multi-modal ICNs revealed enhanced functional coupling between ICNs with higher structural connectivity, improved modularity, and network distinction, particularly in schizophrenia. Statistical analysis of group differences showed more significant differences in the proposed model compared to the unimodal model. In summary, the sfCICA model showed benefits from being jointly informed by structural and functional connectivity. These findings suggest advantages in simultaneously learning effectively and enhancing connectivity estimates using structural connectivity. 
    more » « less
  2. ; ; ; ; ; (, The journal of machine learning for biomedical imaging)
    The structural network of the brain, or structural connectome, can be represented by fiber bundles generated by a variety of tractography methods. While such methods give qualitative insights into brain structure, there is controversy over whether they can provide quantitative information, especially at the population level. In order to enable population-level statistical analysis of the structural connectome, we propose representing a connectome as a Riemannian metric, which is a point on an infinite-dimensional manifold. We equip this manifold with the Ebin metric, a natural metric structure for this space, to get a Riemannian manifold along with its associated geometric properties. We then use this Riemannian framework to apply object-oriented statistical analysis to define an atlas as the Fréchet mean of a population of Riemannian metrics. This formulation ties into the existing framework for diffeomorphic construction of image atlases, allowing us to construct a multimodal atlas by simultaneously integrating complementary white matter structure details from DWMRI and cortical details from T1-weighted MRI. We illustrate our framework with 2D data examples of connectome registration and atlas formation. Finally, we build an example 3D multimodal atlas using T1 images and connectomes derived from diffusion tensors estimated from a subset of subjects from the Human Connectome Project. 
    more » « less
  3. ; ; ; (, 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP))
    In this paper, the relationship between functional and structural brain networks is investigated by training a graph encoder-decoder system to learn the mapping from brain structural connectivity (SC) to functional connectivity (FC). Our work leverages a graph convolutional network (GCN) model in the encoder which integrates both nodal attributes and the network topology information to generate new graph representations in lower dimensions. Using brain SC graphs as inputs, the novel GCN-based encoder-decoder system manages to account for both direct and indirect interactions between brain regions to reconstruct the empirical FC networks. In doing so, the latent variables within the system (i.e., the learnt low-dimensional embeddings) capture important information regarding the relation between functional and structural networks. By decomposing the reconstructed functional networks in terms of the output of each graph convolution filter, we can extract those brain regions which contribute most to the generation of FC networks from their SC counterparts. Experiments on a large population of healthy subjects from the Human Connectome Project show our model can learn a generalizable and interpretable SC-FC relationship. Overall, results here support the promising prospect of using GCNs to discover more about the complex nature of human brain activity and function. 
    more » « less
  4. ; ; ; ; ; ; ; ; ; ; et al (, Springer, Cham)
    The MRI-derived brain network serves as a pivotal instrument in elucidating both the structural and functional aspects of the brain, encompassing the ramifications of diseases and developmental processes. However, prevailing methodologies, often focusing on synchronous BOLD signals from functional MRI (fMRI), may not capture directional influences among brain regions and rarely tackle temporal functional dynamics. In this study, we first construct the brain-effective network via the dynamic causal model. Subsequently, we introduce an interpretable graph learning framework termed Spatio-Temporal Embedding ODE (STE-ODE). This framework incorporates specifically designed directed node embedding layers, aiming at capturing the dynamic interplay between structural and effective networks via an ordinary differential equation (ODE) model, which characterizes spatial-temporal brain dynamics. Our framework is validated on several clinical phenotype prediction tasks using two independent publicly available datasets (HCP and OASIS). The experimental results clearly demonstrate the advantages of our model compared to several state-of-the-art methods. 
    more » « less
  5. ; (, Human Brain Mapping)
    Abstract The study of human brain connectivity, including structural connectivity (SC) and functional connectivity (FC), provides insights into the neurophysiological mechanism of brain function and its relationship to human behavior and cognition. Both types of connectivity measurements provide crucial yet complementary information. However, integrating these two modalities into a single framework remains a challenge, because of the differences in their quantitative interdependencies as well as their anatomical representations due to distinctive imaging mechanisms. In this study, we introduced a new method, joint connectivity matrix independent component analysis (cmICA), which provides a data‐driven parcellation and automated‐linking of SC and FC information simultaneously using a joint analysis of functional magnetic resonance imaging (MRI) and diffusion‐weighted MRI data. We showed that these two connectivity modalities produce common cortical segregation, though with various degrees of (dis)similarity. Moreover, we show conjoint FC networks and structural white matter tracts that directly link these cortical parcellations/sources, within one analysis. Overall, data‐driven joint cmICA provides a new approach for integrating or fusing structural connectivity and FC systematically and conveniently, and provides an effective tool for connectivity‐based multimodal data fusion in brain. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email