An official website of the United States government
Brain large-scale dynamics is constrained by the heterogeneity of intrinsic anatomical substrate. Little is known how the spatiotemporal dynamics adapt for the heterogeneous structural connectivity (SC). Modern neuroimaging modalities make it possible to study the intrinsic brain activity at the scale of seconds to minutes. Diffusion magnetic resonance imaging (dMRI) and functional MRI reveals the large-scale SC across different brain regions. Electrophysiological methods (i.e. MEG/EEG) provide direct measures of neural activity and exhibits complex neurobiological temporal dynamics which could not be solved by fMRI. However, most of existing multimodal analytical methods collapse the brain measurements either in space or time domain and fail to capture the spatio-temporal circuit dynamics. In this paper, we propose a novel spatio-temporal graph Transformer model to integrate the structural and functional connectivity in both spatial and temporal domain. The proposed method learns the heterogeneous node and graph representation via contrastive learning and multi-head attention based graph Transformer using multimodal brain data (i.e. fMRI, MRI, MEG and behavior performance). The proposed contrastive graph Transformer representation model incorporates the heterogeneity map constrained by T1-to-T2-weighted (T1w/T2w) to improve the model fit to structurefunction interactions. The experimental results with multimodal resting state brain measurements demonstrate the proposed method could highlight the local properties of large-scale brain spatio-temporal dynamics and capture the dependence strength between functional connectivity and behaviors. In summary, the proposed method enables the complex brain dynamics explanation for different modal variants.
more »
« less
This content will become publicly available on December 6, 2026
Lightweight Fourier Neural Operator for Time-Dependent Partial Differential Equations
Fourier Neural Operators (FNOs) have shown strong performance in solving time-dependent problems partial differential equations (PDEs). However, accurately modeling complex spatio-temporal dynamics remains challenging and is typically addressed in one of two ways: (i) by applying spectral convolutions over the spatial domain with temporal dynamics handled autoregressively, or (ii) by applying spectral convolutions over the entire spatio-temporal domain. While the former is more computationally efficient, it fails to capture true spatio-temporal interactions. The latter, though more accurate, becomes computationally prohibitive when scaling to larger datasets. We propose LITEFNO, a novel FNO framework that achieves both numerical accuracy and computational efficiency for time-dependent PDEs. Specifically, we first model spatial dynamics by learning a low-rank spatial basis of spectral convolutional weights space. We then incorporate temporal dynamics by learning a new temporal basis through transduction. This factorized formulation enables efficient learning of full spatio-temporal dynamics with significantly fewer parameters (99.9% reduction) and superior performance (44% improvement in VRMSE) compared to the variants of FNO models. The source code and the dataset are available at https://anonymous.4open.science/r/LFNO.
more »
« less
- Award ID(s):
- 2112650
- PAR ID:
- 10712508
- Publisher / Repository:
- Machine Learning and the Physical Sciences Workshop, NeurIPS 2025
- Date Published:
- Format(s):
- Medium: X
- Location:
- San Diego, California
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Operator learning has become a powerful tool in machine learning for modeling complex physical systems governed by partial differential equations (PDEs). Although Deep Operator Networks (DeepONet) show promise, they require extensive data acquisition. Physics-informed DeepONets (PI-DeepONet) mitigate data scarcity but suffer from inefficient training processes. We introduce Separable Operator Networks (SepONet), a novel framework that significantly enhances the efficiency of physics-informed operator learning. SepONet uses independent trunk networks to learn basis functions separately for different coordinate axes, enabling faster and more memory-efficient training via forward-mode automatic differentiation. We provide a universal approximation theorem for SepONet proving the existence of a separable approximation to any nonlinear continuous operator. Then, we comprehensively benchmark its representational capacity and computational performance against PI-DeepONet. Our results demonstrate SepONet's superior performance across various nonlinear and inseparable PDEs, with SepONet's advantages increasing with problem complexity, dimension, and scale. For 1D time-dependent PDEs, SepONet achieves up to 112× faster training and 82× reduction in GPU memory usage compared to PI-DeepONet, while maintaining comparable accuracy. For the 2D time-dependent nonlinear diffusion equation, SepONet efficiently handles the complexity, achieving a 6.44\% mean relative $$\el_2$$ test error, while PI-DeepONet fails due to memory constraints. This work paves the way for extreme-scale learning of continuous mappings between infinite-dimensional function spaces.more » « less
-
Operator learning has become a powerful tool in machine learning for modeling complex physical systems governed by partial differential equations (PDEs). Although Deep Operator Networks (DeepONet) show promise, they require extensive data acquisition. Physics-informed DeepONets (PI-DeepONet) mitigate data scarcity but suffer from inefficient training processes. We introduce Separable Operator Networks (SepONet), a novel framework that significantly enhances the efficiency of physics-informed operator learning. SepONet uses independent trunk networks to learn basis functions separately for different coordinate axes, enabling faster and more memory-efficient training via forward-mode automatic differentiation. We provide a universal approximation theorem for SepONet proving the existence of a separable approximation to any nonlinear continuous operator. Then, we comprehensively benchmark its representational capacity and computational performance against PI-DeepONet. Our results demonstrate SepONet's superior performance across various nonlinear and inseparable PDEs, with SepONet's advantages increasing with problem complexity, dimension, and scale. For 1D time-dependent PDEs, SepONet achieves up to 112× faster training and 82× reduction in GPU memory usage compared to PI-DeepONet, while maintaining comparable accuracy. For the 2D time-dependent nonlinear diffusion equation, SepONet efficiently handles the complexity, achieving a 6.44\% mean relative $$\ell_{2}$$ test error, while PI-DeepONet fails due to memory constraints. This work paves the way for extreme-scale learning of continuous mappings between infinite-dimensional function spaces.more » « less
-
Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent FlowsRecent advancements in operator-type neural networks have shown promising results in approximating the solutions of spatiotemporal Partial Differential Equations (PDEs). However, these neural networks often entail considerable training expenses, and may not always achieve the desired accuracy required in many scientific and engineering disciplines. In this paper, we propose a new learning framework to address these issues. A new spatiotemporal adaptation is proposed to generalize any Fourier Neural Operator (FNO) variant to learn maps between Bochner spaces, which can perform an arbitrary-length temporal super-resolution for the first time. To better exploit this capacity, a new paradigm is proposed to refine the commonly adopted end-to-end neural operator training and evaluations with the help from the wisdom from traditional numerical PDE theory and techniques. Specifically, in the learning problems for the turbulent flow modeled by the Navier-Stokes Equations (NSE), the proposed paradigm trains an FNO only for a few epochs. Then, only the newly proposed spatiotemporal spectral convolution layer is fine-tuned without the frequency truncation. The spectral fine-tuning loss function uses a negative Sobolev norm for the first time in operator learning, defined through a reliable functional-type a posteriori error estimator whose evaluation is exact thanks to the Parseval identity. Moreover, unlike the difficult nonconvex optimization problems in the end-to-end training, this fine-tuning loss is convex. Numerical experiments on commonly used NSE benchmarks demonstrate significant improvements in both computational efficiency and accuracy, compared to end-to-end evaluation and traditional numerical PDE solvers under certain conditions. The source code is publicly available at https://github.com/scaomath/torch-cfd.more » « less
-
Background and aims of study Forecasting the onset of epidemics is a challenging problem due to the scarcity and inconsistency of reported cases across different spatial regions. Epidemic data is often incomplete and limited in certain areas, while other regions provide more comprehensive reporting. This work investigates improving epidemic onset prediction across communities at the state and county level by leveraging domain disentanglement and knowledge transfer techniques. The approach transfers insights from data-rich regions to improve prediction accuracy in data-poor regions by learning domain-invariant and domain-specific features. Methods and results We develop a machine learning framework for predicting epidemic onset using spatio-temporal multivariate time series data. Demographic features are incorporated to analyze how domain disentanglement enables forecasting by separating domain-invariant and domain-specific characteristics. We further examine how correlations between demographic features and case counts evolve over time. Initial results indicate that the impact of demographic characteristics becomes increasingly significant over time. Experimental findings also demonstrate how performance varies with changes in historical data length, forecasting window, and target prediction day across models of different scales. Implications This framework addresses challenges arising from scarce and irregular epidemic data and provides a systematic approach for leveraging cross-regional information. It enables epidemiologists and policymakers to better analyze complex data and make informed decisions by distinguishing between shared and region-specific patterns in epidemic dynamics.more » « less
