More Like this

1. Causal Deep Operator Networks for Data-Driven Modeling of Dynamical Systems

   https://doi.org/10.1109/SMC53992.2023.10394294  

   Nghiem, Truong X.; Nguyen, Thang; Nguyen, Binh T.; Nguyen, Linh (October 2023, Proceeding of the IEEE International Conference on Systems, Man, and Cybernetics)

   The deep operator network (DeepONet) architecture is a promising approach for learning functional operators, that can represent dynamical systems described by ordinary or partial differential equations. However, it has two major limitations, namely its failures to account for initial conditions and to guarantee the temporal causality – a fundamental property of dynamical systems. This paper proposes a novel causal deep operator network (Causal-DeepONet) architecture for incorporating both the initial condition and the temporal causality into data-driven learning of dynamical systems, overcoming the limitations of the original DeepONet approach. This is achieved by adding an independent root network for the initial condition and independent branch networks conditioned, or switched on/off, by time-shifted step functions or sigmoid functions for expressing the temporal causality. The proposed architecture was evaluated and compared with two baseline deep neural network methods and the original DeepONet method on learning the thermal dynamics of a room in a building using real data. It was shown to not only achieve the best overall prediction accuracy but also enhance substantially the accuracy consistency in multistep predictions, which is crucial for predictive control. 

   more » « less
2. Physics-informed time-integrated DeepONet: Temporal tangent space operator learning for high-accuracy inference

   https://doi.org/10.1016/j.cma.2026.118917  

   Mandl, Luis; Nayak, Dibyajyoti; Ricken, Tim; Goswami, Somdatta (June 2026, Computer Methods in Applied Mechanics and Engineering)

   Accurately modeling and inferring solutions to time-dependent partial differential equations (PDEs) over extended temporal horizons remains a core challenge in scientific machine learning. Traditional full rollout (FR) methods, predicting entire trajectories in a single pass, often fail to capture the causal dependencies inherent to dynamical systems and exhibit poor generalization outside the training time horizon. In contrast, autoregressive (AR) approaches, which evolve the system step by step, are prone to error accumulation, as predictions at each time step depend on potentially erroneous prior outputs. These shortcomings limit the long-term accuracy and reliability of both strategies. To overcome these issues, we introduce Physics-Informed Time-Integrated Deep Operator Network (PITI-DeepONet), an operator learning framework designed for stable and accurate long-term time evolution, well beyond the training time horizon. PITI-DeepONet employs a dual-output DeepONet architecture trained via either fully physics-informed or hybrid physics- and data-driven objectives. The training enforces consistency between the learned temporal derivative and its counterpart obtained via automatic differentiation. Rather than directly forecasting future states, the network learns the time-derivative operator from the current state, which is then integrated using classical time-stepping schemes - such as explicit Euler, fourthorder Runge-Kutta, second-order Adams-Bashforth-Moulton, or implicit Euler - to advance the solution sequentially in time. Additionally, the framework supports residual monitoring during inference to estimate prediction quality and flags when the learned temporal tangent becomes unreliable, e.g., outside the training domain. Applied to benchmark problems, PITI-DeepONet demonstrates enhanced accuracy and stability over extended inference time horizons when compared to traditional methods. Mean relative L2 errors reduced by 84% (versus FR) and 79% (versus AR) for the one-dimensional heat equation; by 87% (versus FR) and 98% (versus AR) for the onedimensional Burgers equation; by 42% (versus FR) and 89% (versus AR) for the two-dimensional Allen-Cahn equation; and by 58% (vs. FR) and 61% (vs. AR) for the one-dimensional Kuramoto- Sivashinsky equation. By moving beyond classic FR and AR schemes, PITI-DeepONet paves the way for more reliable, long-term integration of complex, time-dependent PDEs. 

   more » « less
3. Non-Linear Operator Approximations for Initial Value Problems

   Gupta, Gaurav and (May 2022, International Conference on Learning Representations)

   Time-evolution of partial differential equations is the key to model several dynamical processes, events forecasting but the operators associated with such problems are non-linear. We propose a Padé approximation based exponential neural operator scheme for efficiently learning the map between a given initial condition and activities at a later time. The multiwavelets bases are used for space discretization. By explicitly embedding the exponential operators in the model, we reduce the training parameters and make it more data-efficient which is essential in dealing with scarce real-world datasets. The Padé exponential operator uses a to model the non-linearity compared to recent neural operators that rely on using multiple linear operator layers in succession. We show theoretically that the gradients associated with the recurrent Padé network are bounded across the recurrent horizon. We perform experiments on non-linear systems such as Korteweg-de Vries (KdV) and Kuramoto–Sivashinsky (KS) equations to show that the proposed approach achieves the best performance and at the same time is data-efficient. We also show that urgent real-world problems like Epidemic forecasting (for example, COVID-19) can be formulated as a 2D time-varying operator problem. The proposed Padé exponential operators yield better prediction results ( better MAE than best neural operator (non-neural operator deep learning model)) compared to state-of-the-art forecasting models. 

   more » « less
4. Non-linear operator approximations for initial value problems

   Gaurav Gupta, Xiongye Xiao (June 2022, International Conference on Learning Representations)

   Time-evolution of partial differential equations is the key to model several dynamical processes, events forecasting but the operators associated with such problems are non-linear. We propose a Padé approximation based exponential neural operator scheme for efficiently learning the map between a given initial condition and activities at a later time. The multiwavelets bases are used for space discretization. By explicitly embedding the exponential operators in the model, we reduce the training parameters and make it more data-efficient which is essential in dealing with scarce real-world datasets. The Padé exponential operator uses a to model the non-linearity compared to recent neural operators that rely on using multiple linear operator layers in succession. We show theoretically that the gradients associated with the recurrent Padé network are bounded across the recurrent horizon. We perform experiments on non-linear systems such as Korteweg-de Vries (KdV) and Kuramoto–Sivashinsky (KS) equations to show that the proposed approach achieves the best performance and at the same time is data-efficient. We also show that urgent real-world problems like Epidemic forecasting (for example, COVID-19) can be formulated as a 2D time-varying operator problem. The proposed Padé exponential operators yield better prediction results ( better MAE than best neural operator (non-neural operator deep learning model)) compared to state-of-the-art forecasting models. 

   more » « less
5. NON-LINEAR OPERATOR APPROXIMATIONS FOR INITIAL VALUE PROBLEMS

   Gupta, Gaurav; Xiao, Xiongye; Balan, Radu; Bogdan, Paul (April 2022, International Conference on Learning Representations (ICLR))

   Time-evolution of partial differential equations is fundamental for modeling several complex dynamical processes and events forecasting, but the operators associated with such problems are non-linear. We propose a Pad´e approximation based exponential neural operator scheme for efficiently learning the map between a given initial condition and the activities at a later time. The multiwavelets bases are used for space discretization. By explicitly embedding the exponential operators in the model, we reduce the training parameters and make it more data-efficient which is essential in dealing with scarce and noisy real-world datasets. The Pad´e exponential operator uses a recurrent structure with shared parameters to model the non-linearity compared to recent neural operators that rely on using multiple linear operator layers in succession. We show theoretically that the gradients associated with the recurrent Pad´e network are bounded across the recurrent horizon. We perform experiments on non-linear systems such as Korteweg-de Vries (KdV) and Kuramoto–Sivashinsky (KS) equations to show that the proposed approach achieves the best performance and at the same time is data-efficient. We also show that urgent real-world problems like epidemic forecasting (for example, COVID- 19) can be formulated as a 2D time-varying operator problem. The proposed Pad´e exponential operators yield better prediction results (53% (52%) better MAE than best neural operator (non-neural operator deep learning model)) compared to state-of-the-art forecasting models. 

   more » « less