More Like this

1. Integrating Neural Networks and Dictionary Learning for Multidimensional Clinical Characterizations from Functional Connectomics Data

   Niharika Shimona D’Souza, Mary Beth (October 2019, Medical Image Computing and Computer Assisted Intervention)

   We propose a unified optimization framework that combines neural networks with dictionary learning to model complex interactions between resting state functional MRI and behavioral data. The dictionary learning objective decomposes patient correlation matrices into a collection of shared basis networks and subject-specific loadings. These subject-specific features are simultaneously input into a neural network that predicts multidimensional clinical information. Our novel optimization framework combines the gradient information from the neural network with that of a conventional matrix factorization objective. This procedure collectively estimates the basis networks, subject loadings, and neural network weights most informative of clinical severity. We evaluate our combined model on a multi-score prediction task using 52 patients diagnosed with Autism Spectrum Disorder (ASD). Our integrated framework outperforms state-of-the-art methods in a ten-fold cross validated setting to predict three different measures of clinical severity. 

   more » « less
2. A spatially constrained independent component analysis jointly informed by structural and functional network connectivity

   https://doi.org/10.1162/netn_a_00398  

   Fouladivanda, Mahshid; Iraji, Armin; Wu, Lei; van_Erp, Theodorus_G_M; Belger, Aysenil; Hawamdeh, Faris; Pearlson, Godfrey_D; Calhoun, Vince_D (June 2024, Network Neuroscience)

   Abstract There are a growing number of neuroimaging studies motivating joint structural and functional brain connectivity. Brain connectivity of different modalities provides insight into brain functional organization by leveraging complementary information, especially for brain disorders such as schizophrenia. In this paper, we propose a multi-modal independent component analysis (ICA) model that utilizes information from both structural and functional brain connectivity guided by spatial maps to estimate intrinsic connectivity networks (ICNs). Structural connectivity is estimated through whole-brain tractography on diffusion-weighted MRI (dMRI), while functional connectivity is derived from resting-state functional MRI (rs-fMRI). The proposed structural-functional connectivity and spatially constrained ICA (sfCICA) model estimates ICNs at the subject level using a multi-objective optimization framework. We evaluated our model using synthetic and real datasets (including dMRI and rs-fMRI from 149 schizophrenia patients and 162 controls). Multi-modal ICNs revealed enhanced functional coupling between ICNs with higher structural connectivity, improved modularity, and network distinction, particularly in schizophrenia. Statistical analysis of group differences showed more significant differences in the proposed model compared to the unimodal model. In summary, the sfCICA model showed benefits from being jointly informed by structural and functional connectivity. These findings suggest advantages in simultaneously learning effectively and enhancing connectivity estimates using structural connectivity. 

   more » « less
3. Re-Think and Re-Design Graph Neural Networks in Spaces of Continuous Graph Diffusion Functionals

   Dan, T; Ding, J; Wei, Z; Kovalsky, S; Kim, M; Kim, WH; Wu, G (September 2023, Advances in Neural Information Processing Systems 36 (NeurIPS 2023))

   Graphs are ubiquitous in various domains, such as social networks and biological systems. Despite the great successes of graph neural networks (GNNs) in modeling and analyzing complex graph data, the inductive bias of locality assumption, which involves exchanging information only within neighboring connected nodes, restricts GNNs in capturing long-range dependencies and global patterns in graphs. Inspired by the classic Brachistochrone problem, we seek how to devise a new inductive bias for cutting-edge graph application and present a general framework through the lens of variational analysis. The backbone of our framework is a two-way mapping between the discrete GNN model and continuous diffusion functional, which allows us to design application-specific objective function in the continuous domain and engineer discrete deep model with mathematical guarantees. First, we address over-smoothing in current GNNs. Specifically, our inference reveals that the existing layer-by-layer models of graph embedding learning are equivalent to a ℓ 2 -norm integral functional of graph gradients, which is the underlying cause of the over-smoothing problem. Similar to edge-preserving filters in image denoising, we introduce the total variation (TV) to promote alignment of the graph diffusion pattern with the global information present in community topologies. On top of this, we devise a new selective mechanism for inductive bias that can be easily integrated into existing GNNs and effectively address the trade-off between model depth and over-smoothing. Second, we devise a novel generative adversarial network (GAN) to predict the spreading flows in the graph through a neural transport equation. To avoid the potential issue of vanishing flows, we tailor the objective function to minimize the transportation within each community while maximizing the inter-community flows. Our new GNN models achieve state-of-the-art (SOTA) performance on graph learning benchmarks such as Cora, Citeseer, and Pubmed. 

   more » « less
4. From Data to Design: Leveraging Frequency Statistics for Efficient Neural Network Architectures

   Munir, Mustafa; Li, Guihong; Rahman, Mostafijur; Zhang, Alex; Marculescu, Radu (June 2025, IEEE)

   This paper delves into the frequency analysis of image datasets and neural networks, particularly Vision Transformers (ViTs) and Convolutional Neural Networks (CNNs), and reveals the alignment property between datasets and network architecture design. Our analysis suggests that the frequency statistics of image datasets and the learning behavior of neural networks are intertwined. Based on this observation, our main contribution consists of a new framework for network optimization that guides the design process by adjusting the network’s depth and width to align the frequency characteristics of untrained models with those of trained models. Our frequency analysis framework can be used to design better neural networks with better performance-model size trade-offs. Our results on ImageNet-1k image classification, CIFAR-100 image classification, and MS-COCO object detection and instance segmentation benchmarks show that our method is broadly applicable and can improve network architecture performance. Our investigation into the alignment between the frequency characteristics of image datasets and network architecture opens up a new direction in model analysis that can be used to design more efficient networks. 

   more » « less
5. Drop, Swap, and Generate: A Self-Supervised Approach for Generating Neural Activity

   Ran Liu, Mehdi Azabou (December 2021, Advances in neural information processing systems)

   Meaningful and simplified representations of neural activity can yield insights into how and what information is being processed within a neural circuit. However, without labels, finding representations that reveal the link between the brain and behavior can be challenging. Here, we introduce a novel unsupervised approach for learning disentangled representations of neural activity called Swap-VAE. Our approach combines a generative modeling framework with an instance-specific alignment loss that tries to maximize the representational similarity between transformed views of the input (brain state). These transformed (or augmented) views are created by dropping out neurons and jittering samples in time, which intuitively should lead the network to a representation that maintains both temporal consistency and invariance to the specific neurons used to represent the neural state. Through evaluations on both synthetic data and neural recordings from hundreds of neurons in different primate brains, we show that it is possible to build representations that disentangle neural datasets along relevant latent dimensions linked to behavior. 

   more » « less