Accurate temporal extrapolation presents a fundamental challenge for neural operators in modeling dynamical systems, where reliable predictions must extend significantly beyond the training time horizon. Conventional deep operator network (DeepONet) approaches rely on two inherently limited training paradigms: fixed-horizon rollouts, which predict complete spatiotemporal solutions while disregarding temporal causality, and autoregressive formulations, which accumulate errors through sequential predictions in time. We introduce TI-DeepONet (Time-Integratorembedded Deep Operator Network), a framework that integrates neural operators with adaptive numerical time-stepping techniques to preserve the underlying Markovian structure of dynamical systems while substantially mitigating error propagation in extended temporal forecasting. Our approach reformulates the learning objective from direct state prediction to the approximation of instantaneous time-derivative fields, which are subsequently integrated using established numerical schemes. This design naturally supports continuous-time prediction and enables the use of higher-precision integrators at inference than those employed during training, striking a balance between computational efficiency and predictive accuracy. We further develop TI(L)-DeepONet (Learnable Time-Integrator-embedded Deep Operator Network), which incorporates learnable coefficients for intermediate stages in a multi-stage numerical integration scheme, thereby adapting to solution-specific variations and enhancing predictive fidelity. Rigorous evaluation across six canonical partial differential equations (PDEs) spanning diverse, high-dimensional, chaotic, dissipative, and dispersive dynamics shows that TI(L)-DeepONet marginally outperforms TIDeepONet, with both frameworks achieving significant reductions in relative 𝐿2 extrapolation error: approximately 96.3% compared to autoregressive implementations and 83.6% compared to fixed-horizon approaches. Notably, both methods maintain stable predictions over temporal domains extending to nearly twice the training interval. This research establishes a physics-aware operator learning paradigm that bridges neural approximation with numerical analysis principles, preserving the causal structure of dynamical systems while addressing a critical gap in long-term forecasting of complex physical phenomena.
more »
« less
This content will become publicly available on June 1, 2027
Physics-informed time-integrated DeepONet: Temporal tangent space operator learning for high-accuracy inference
Accurately modeling and inferring solutions to time-dependent partial differential equations (PDEs) over extended temporal horizons remains a core challenge in scientific machine learning. Traditional full rollout (FR) methods, predicting entire trajectories in a single pass, often fail to capture the causal dependencies inherent to dynamical systems and exhibit poor generalization outside the training time horizon. In contrast, autoregressive (AR) approaches, which evolve the system step by step, are prone to error accumulation, as predictions at each time step depend on potentially erroneous prior outputs. These shortcomings limit the long-term accuracy and reliability of both strategies. To overcome these issues, we introduce Physics-Informed Time-Integrated Deep Operator Network (PITI-DeepONet), an operator learning framework designed for stable and accurate long-term time evolution, well beyond the training time horizon. PITI-DeepONet employs a dual-output DeepONet architecture trained via either fully physics-informed or hybrid physics- and data-driven objectives. The training enforces consistency between the learned temporal derivative and its counterpart obtained via automatic differentiation. Rather than directly forecasting future states, the network learns the time-derivative operator from the current state, which is then integrated using classical time-stepping schemes - such as explicit Euler, fourthorder Runge-Kutta, second-order Adams-Bashforth-Moulton, or implicit Euler - to advance the solution sequentially in time. Additionally, the framework supports residual monitoring during inference to estimate prediction quality and flags when the learned temporal tangent becomes unreliable, e.g., outside the training domain. Applied to benchmark problems, PITI-DeepONet demonstrates enhanced accuracy and stability over extended inference time horizons when compared to traditional methods. Mean relative L2 errors reduced by 84% (versus FR) and 79% (versus AR) for the one-dimensional heat equation; by 87% (versus FR) and 98% (versus AR) for the onedimensional Burgers equation; by 42% (versus FR) and 89% (versus AR) for the two-dimensional Allen-Cahn equation; and by 58% (vs. FR) and 61% (vs. AR) for the one-dimensional Kuramoto- Sivashinsky equation. By moving beyond classic FR and AR schemes, PITI-DeepONet paves the way for more reliable, long-term integration of complex, time-dependent PDEs.
more »
« less
- PAR ID:
- 10712390
- Publisher / Repository:
- Elsevier
- Date Published:
- Journal Name:
- Computer Methods in Applied Mechanics and Engineering
- Volume:
- 455
- Issue:
- C
- ISSN:
- 0045-7825
- Page Range / eLocation ID:
- 118917
- Subject(s) / Keyword(s):
- Operator learning, Physics-informed, Extrapolation, Time integration, Partial differential equations, DeepONet
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Operator learning has become a powerful tool in machine learning for modeling complex physical systems governed by partial differential equations (PDEs). Although Deep Operator Networks (DeepONet) show promise, they require extensive data acquisition. Physics-informed DeepONets (PI-DeepONet) mitigate data scarcity but suffer from inefficient training processes. We introduce Separable Operator Networks (SepONet), a novel framework that significantly enhances the efficiency of physics-informed operator learning. SepONet uses independent trunk networks to learn basis functions separately for different coordinate axes, enabling faster and more memory-efficient training via forward-mode automatic differentiation. We provide a universal approximation theorem for SepONet proving the existence of a separable approximation to any nonlinear continuous operator. Then, we comprehensively benchmark its representational capacity and computational performance against PI-DeepONet. Our results demonstrate SepONet's superior performance across various nonlinear and inseparable PDEs, with SepONet's advantages increasing with problem complexity, dimension, and scale. For 1D time-dependent PDEs, SepONet achieves up to 112× faster training and 82× reduction in GPU memory usage compared to PI-DeepONet, while maintaining comparable accuracy. For the 2D time-dependent nonlinear diffusion equation, SepONet efficiently handles the complexity, achieving a 6.44\% mean relative $$\el_2$$ test error, while PI-DeepONet fails due to memory constraints. This work paves the way for extreme-scale learning of continuous mappings between infinite-dimensional function spaces.more » « less
-
Operator learning has become a powerful tool in machine learning for modeling complex physical systems governed by partial differential equations (PDEs). Although Deep Operator Networks (DeepONet) show promise, they require extensive data acquisition. Physics-informed DeepONets (PI-DeepONet) mitigate data scarcity but suffer from inefficient training processes. We introduce Separable Operator Networks (SepONet), a novel framework that significantly enhances the efficiency of physics-informed operator learning. SepONet uses independent trunk networks to learn basis functions separately for different coordinate axes, enabling faster and more memory-efficient training via forward-mode automatic differentiation. We provide a universal approximation theorem for SepONet proving the existence of a separable approximation to any nonlinear continuous operator. Then, we comprehensively benchmark its representational capacity and computational performance against PI-DeepONet. Our results demonstrate SepONet's superior performance across various nonlinear and inseparable PDEs, with SepONet's advantages increasing with problem complexity, dimension, and scale. For 1D time-dependent PDEs, SepONet achieves up to 112× faster training and 82× reduction in GPU memory usage compared to PI-DeepONet, while maintaining comparable accuracy. For the 2D time-dependent nonlinear diffusion equation, SepONet efficiently handles the complexity, achieving a 6.44\% mean relative $$\ell_{2}$$ test error, while PI-DeepONet fails due to memory constraints. This work paves the way for extreme-scale learning of continuous mappings between infinite-dimensional function spaces.more » « less
-
In the realm of computational science and engineering, constructing models that reflect real-world phenomena requires solving partial differential equations (PDEs) with different conditions. Recent advancements in neural operators, such as deep operator network (DeepONet), which learn mappings between infinite-dimensional function spaces, promise efficient computation of PDE solutions for a new condition in a single forward pass. However, classical DeepONet entails quadratic complexity concerning input dimensions during evaluation. Given the progress in quantum algorithms and hardware, here we propose to utilize quantum computing to accelerate DeepONet evaluations, yielding complexity that is linear in input dimensions. Our proposed quantum DeepONet integrates unary encoding and orthogonal quantum layers. We benchmark our quantum DeepONet using a variety of PDEs, including the antiderivative operator, advection equation, and Burgers' equation. We demonstrate the method's efficacy in both ideal and noisy conditions. Furthermore, we show that our quantum DeepONet can also be informed by physics, minimizing its reliance on extensive data collection. Quantum DeepONet will be particularly advantageous in applications in outer loop problems which require exploring parameter space and solving the corresponding PDEs, such as uncertainty quantification and optimal experimental design.more » « less
-
Physics-informed neural networks (PINNs) have been widely used to solve partial differential equations (PDEs) in a forward and inverse manner using neural networks. However, balancing individual loss terms can be challenging, mainly when training these networks for stiff PDEs and scenarios requiring enforcement of numerous constraints. Even though statistical methods can be applied to assign relative weights to the regression loss for data, assigning relative weights to equation-based loss terms remains a formidable task. This paper proposes a method for assigning relative weights to the mean squared loss terms in the objective function used to train PINNs. Due to the presence of temporal gradients in the governing equation, the physics-informed loss can be recast using numerical integration through backward Euler discretization. The physics-uninformed and physics-informed networks should yield identical predictions when assessed at corresponding spatiotemporal positions. We refer to this consistency as “temporal consistency.” This approach introduces a unique method for training physics-informed neural networks (PINNs), redefining the loss function to allow for assigning relative weights with statistical properties of the observed data. In this work, we consider the two- and three-dimensional Navier–Stokes equations and determine the kinematic viscosity using the spatiotemporal data on the velocity and pressure fields. We consider numerical datasets to test our method. We test the sensitivity of our method to the timestep size, the number of timesteps, noise in the data, and spatial resolution. Finally, we use the velocity field obtained using particle image velocimetry experiments to generate a reference pressure field and test our framework using the velocity and pressure fields.more » « less
An official website of the United States government
