Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Spectral-Refiner: Accurate Fine-Tuning of Spatiotemporal Fourier Neural Operator for Turbulent Flows
Recent advancements in operator-type neural networks have shown promising results in approximating the solutions of spatiotemporal Partial Differential Equations (PDEs). However, these neural networks often entail considerable training expenses, and may not always achieve the desired accuracy required in many scientific and engineering disciplines. In this paper, we propose a new learning framework to address these issues. A new spatiotemporal adaptation is proposed to generalize any Fourier Neural Operator (FNO) variant to learn maps between Bochner spaces, which can perform an arbitrary-length temporal super-resolution for the first time. To better exploit this capacity, a new paradigm is proposed to refine the commonly adopted end-to-end neural operator training and evaluations with the help from the wisdom from traditional numerical PDE theory and techniques. Specifically, in the learning problems for the turbulent flow modeled by the Navier-Stokes Equations (NSE), the proposed paradigm trains an FNO only for a few epochs. Then, only the newly proposed spatiotemporal spectral convolution layer is fine-tuned without the frequency truncation. The spectral fine-tuning loss function uses a negative Sobolev norm for the first time in operator learning, defined through a reliable functional-type a posteriori error estimator whose evaluation is exact thanks to the Parseval identity. Moreover, unlike the difficult nonconvex optimization problems in the end-to-end training, this fine-tuning loss is convex. Numerical experiments on commonly used NSE benchmarks demonstrate significant improvements in both computational efficiency and accuracy, compared to end-to-end evaluation and traditional numerical PDE solvers under certain conditions. The source code is publicly available at https://github.com/scaomath/torch-cfd.  more » « less
Award ID(s):
2309778
PAR ID:
10574739
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
ICLR 2025
Date Published:
Format(s):
Medium: X
Location:
Singapore
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; (, Communications Physics)
    Abstract Traditional data-driven deep learning models often struggle with high training costs, error accumulation, and poor generalizability in complex physical processes. Physics-informed deep learning (PiDL) addresses these challenges by incorporating physical principles into the model. Most PiDL approaches regularize training by embedding governing equations into the loss function, yet this depends heavily on extensive hyperparameter tuning to weigh each loss term. To this end, we propose to leverage physics prior knowledge by “baking” the discretized governing equations into the neural network architecture via the connection between the partial differential equations (PDE) operators and network structures, resulting in a PDE-preserved neural network (PPNN). This method, embedding discretized PDEs through convolutional residual networks in a multi-resolution setting, largely improves the generalizability and long-term prediction accuracy, outperforming conventional black-box models. The effectiveness and merit of the proposed methods have been demonstrated across various spatiotemporal dynamical systems governed by spatiotemporal PDEs, including reaction-diffusion, Burgers’, and Navier-Stokes equations. 
    more » « less
  2. ; ; (, American Society of Mechanical Engineers (ASME))
    Electrical machines traditionally rely on Finite Element Analysis (FEA) to evaluate or simulate their properties by solving the associated partial differential equations (PDEs). However, FEA is computationally costly, which limits its capability for rapid design iteration and real-time simulations. While recent surrogate models such as Physics-Informed Neural Networks (PINNs) have shown promise, they often suffer from slow convergence and scalability issues in complex geometries. In this paper, we propose the use of the Fourier Neural Operator (FNO) as a resolution-invariant surrogate model to significantly reduce the computation time required for FEA-based PDE solutions in electric machines. Previous research has demonstrated the FNO’s ability to learn mappings for time-sequence problems by approximating operators between function spaces. Building on this, we present a methodology to directly predict the later-state electromagnetic fields of a rotating interior permanent magnet (IPM) motor based on its earlier-stage data by approximating the underlying operator that governs these transitions. Our framework enables full-geometry modeling without relying on segmentation, preserving accuracy while dramatically improving computational efficiency. The model was trained and validated on an FEA dataset with multiple boundary conditions and motor configurations, demonstrating strong generalization across different designs and resolutions. Experimental results show that the proposed FNO method achieves a significant reduction in computational time compared to traditional FEA simulations while maintaining an acceptable level of accuracy. This study highlights the potential of neural operators for accelerating electromagnetic simulations, enabling faster design iterations and offering new possibilities for real-time and optimization-based applications in electric machine design. 
    more » « less
  3. ; ; (, Computer Methods in Applied Mechanics and Engineering)
    Minimizing partial differential equation (PDE)-residual losses is a common strategy to promote physical consistency in neural operators. However, standard formulations often lack variational correctness, meaning that small residuals do not guarantee small solution errors due to the use of non-compliant norms or ad hoc penalty terms for boundary conditions. This work develops a variationally correct operator learning framework by constructing first-order system least- squares (FOSLS) objectives whose values are provably equivalent to the solution errors in PDE-compliant norms. We demonstrate this framework on the stationary diffusion and linear elasticity equations, incorporating mixed Dirichlet-Neumann boundary conditions via variational lifts to preserve norm equivalence without inconsistent penalties. To ensure the function space conformity required by the FOSLS loss, we propose a Reduced Basis Neural Operator (RBNO). The RBNO predicts coefficients for a pre-computed, conforming reduced basis, thereby ensuring variational stability by design while enabling efficient training. We provide a rigorous convergence analysis that bounds the total error by the sum of finite element discretization error, reduced basis projection error, neural network approximation error, statistical estimation error, and optimization error. Numerical benchmarks validate these theoretical bounds and demonstrate that the proposed approach achieves superior accuracy in PDE-compliant norms compared to standard baselines, while the residual loss serves as a reliable and computable a posteriori error estimator. 
    more » « less
  4. ; (, Computer Methods in Applied Mechanics and Engineering)
    Accurate temporal extrapolation presents a fundamental challenge for neural operators in modeling dynamical systems, where reliable predictions must extend significantly beyond the training time horizon. Conventional deep operator network (DeepONet) approaches rely on two inherently limited training paradigms: fixed-horizon rollouts, which predict complete spatiotemporal solutions while disregarding temporal causality, and autoregressive formulations, which accumulate errors through sequential predictions in time. We introduce TI-DeepONet (Time-Integratorembedded Deep Operator Network), a framework that integrates neural operators with adaptive numerical time-stepping techniques to preserve the underlying Markovian structure of dynamical systems while substantially mitigating error propagation in extended temporal forecasting. Our approach reformulates the learning objective from direct state prediction to the approximation of instantaneous time-derivative fields, which are subsequently integrated using established numerical schemes. This design naturally supports continuous-time prediction and enables the use of higher-precision integrators at inference than those employed during training, striking a balance between computational efficiency and predictive accuracy. We further develop TI(L)-DeepONet (Learnable Time-Integrator-embedded Deep Operator Network), which incorporates learnable coefficients for intermediate stages in a multi-stage numerical integration scheme, thereby adapting to solution-specific variations and enhancing predictive fidelity. Rigorous evaluation across six canonical partial differential equations (PDEs) spanning diverse, high-dimensional, chaotic, dissipative, and dispersive dynamics shows that TI(L)-DeepONet marginally outperforms TIDeepONet, with both frameworks achieving significant reductions in relative 𝐿2 extrapolation error: approximately 96.3% compared to autoregressive implementations and 83.6% compared to fixed-horizon approaches. Notably, both methods maintain stable predictions over temporal domains extending to nearly twice the training interval. This research establishes a physics-aware operator learning paradigm that bridges neural approximation with numerical analysis principles, preserving the causal structure of dynamical systems while addressing a critical gap in long-term forecasting of complex physical phenomena. 
    more » « less
  5. ; ; ; ; ; (, Machine Learning and the Physical Sciences Workshop, NeurIPS 2025)
    Fourier Neural Operators (FNOs) have shown strong performance in solving time-dependent problems partial differential equations (PDEs). However, accurately modeling complex spatio-temporal dynamics remains challenging and is typically addressed in one of two ways: (i) by applying spectral convolutions over the spatial domain with temporal dynamics handled autoregressively, or (ii) by applying spectral convolutions over the entire spatio-temporal domain. While the former is more computationally efficient, it fails to capture true spatio-temporal interactions. The latter, though more accurate, becomes computationally prohibitive when scaling to larger datasets. We propose LITEFNO, a novel FNO framework that achieves both numerical accuracy and computational efficiency for time-dependent PDEs. Specifically, we first model spatial dynamics by learning a low-rank spatial basis of spectral convolutional weights space. We then incorporate temporal dynamics by learning a new temporal basis through transduction. This factorized formulation enables efficient learning of full spatio-temporal dynamics with significantly fewer parameters (99.9% reduction) and superior performance (44% improvement in VRMSE) compared to the variants of FNO models. The source code and the dataset are available at https://anonymous.4open.science/r/LFNO. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Text Encoded Conversion
  • XML
  • Save / Share this Record
  • Send to Email