An official website of the United States government
Deep Learning (DL), in particular deep neural networks (DNN), by default is purely data-driven and in general does not require physics. This is the strength of DL but also one of its key limitations when applied to science and engineering problems in which underlying physical properties—such as stability, conservation, and positivity—and accuracy are required. DL methods in their original forms are not capable of respecting the underlying mathematical models or achieving desired accuracy even in big-data regimes. On the other hand, many data-driven science and engineering problems, such as inverse problems, typically have limited experimental or observational data, and DL would overfit the data in this case. Leveraging information encoded in the underlying mathematical models, we argue, not only compensates missing information in low data regimes but also provides opportunities to equip DL methods with the underlying physics, and hence promoting better generalization. This paper develops a model-constrained deep learning approach and its variant TNet—a Tikhonov neural network—that are capable of learning not only information hidden in the training data but also in the underlying mathematical models to solve inverse problems governed by partial differential equations in low data regimes. We provide the constructions and some theoretical results for the proposed approaches for both linear and nonlinear inverse problems. Since TNet is designed to learn inverse solution with Tikhonov regularization, it is interpretable: in fact it recovers Tikhonov solutions for linear cases while potentially approximating Tikhonov solutions in any desired accuracy for nonlinear inverse problems. We also prove that data randomization can enhance not only the smoothness of the networks but also their generalizations. Comprehensive numerical results confirm the theoretical findings and show that with even as little as 1 training data sample for 1D deconvolution, 5 for inverse 2D heat conductivity problem, 100 for inverse initial conditions for time-dependent 2D Burgers’ equation, and 50 for inverse initial conditions for 2D Navier-Stokes equations, TNet solutions can be as accurate as Tikhonov solutions while being several orders of magnitude faster. This is possible owing to the model-constrained term, replications, and randomization.
more »
« less
This content will become publicly available on November 1, 2026
TAEN: a model-constrained Tikhonov autoencoder network for forward and inverse problems
Efficient real-time solvers for forward and inverse problems are essential in engineering and science applications. Machine learning surrogate models have emerged as promising alter- natives to traditional methods, offering substantially reduced computational time. Never- theless, these models typically demand extensive training datasets to achieve robust gen- eralization across diverse scenarios. While physics-based approaches can partially mitigate this data dependency and ensure physics-interpretable solutions, addressing scarce data regimes remains a challenge. Both purely data-driven and physics-based machine learning approaches demonstrate severe overfitting issues when trained with insufficient data. We propose a novel model-constrained Tikhonov autoencoder neural network framework, called TAEN, capable of learning both forward and inverse surrogate models using a single arbitrary observational sample. We develop comprehensive theoretical foundations including forward and inverse inference error bounds for the proposed approach for linear cases. For compara- tive analysis, we derive equivalent formulations for pure data-driven and model-constrained approach counterparts. At the heart of our approach is a data randomization strategy with theoretical justification, which functions as a generative mechanism for exploring the train- ing data space, enabling effective training of both forward and inverse surrogate models even with a single observation, while regularizing the learning process. We validate our approach through extensive numerical experiments on two challenging inverse problems: 2D heat conductivity inversion and initial condition reconstruction for time-dependent 2D Navier–Stokes equations. Results demonstrate that TAEN achieves accuracy comparable to traditional Tikhonov solvers and numerical forward solvers for both inverse and forward problems, respectively, while delivering orders of magnitude computational speedups.
more »
« less
- PAR ID:
- 10632451
- Publisher / Repository:
- ScienceDirect
- Date Published:
- Journal Name:
- Computer Methods in Applied Mechanics and Engineering
- Volume:
- 446
- Issue:
- PA
- ISSN:
- 0045-7825
- Page Range / eLocation ID:
- 118245
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Predicting the future contributions of the ice sheets to sea level rise remains a significant challenge due to our limited understanding of key physical processes (e.g., basal friction, ice rheology) and the lack of observations of critical model inputs (e.g., bed topography). Traditional numerical models typically rely on data assimilation methods to estimate these variables by solving inverse problems based on conservation laws of mass, momentum, and energy. However, these methods are not versatile and require extensive code development to incorporate new physics. Moreover, their dependence on data alignment within computational grids hampers their adaptability, especially in the context of sparse data availability in space and time. To address these limitations, we developed PINNICLE (Physics-Informed Neural Networks for Ice and CLimatE), an open-source Python library dedicated to ice sheet modeling. PINNICLE seamlessly integrates observational data and physical laws, facilitating the solution of both forward and inverse problems within a single framework. PINNICLE currently supports a variety of conservation laws, including the Shelfy-Stream Approximation (SSA), Mono-Layer Higher-Order (MOLHO) models, and mass conservation equations, for both time-independent and time-dependent simulations. The library is user-friendly, requiring only the setting of a few hyperparameters for standard modeling tasks, while advanced users can define custom models within the framework. Additionally, PINNICLE is based on the DeepXDE library, which supports widely-used machine learning packages such as TensorFlow, PyTorch, and JAX, enabling users to select the backend that best fits their hardware. We describe here the implementation of PINNICLE and showcase this library with examples across the Greenland and Antarctic ice sheets for a range of forward and inverse problems.more » « less
-
Predicting the future contributions of the ice sheets to sea-level rise remains a significant challenge due to our limited understanding of key physical processes (e.g., basal friction, ice rheology) and the lack of observations of critical model inputs (e.g., bed topography). Traditional numerical models typically rely on data assimilation methods to estimate these variables by solving inverse problems based on conservation laws of mass, momentum, and energy. However, these methods are not versatile and require extensive code development to incorporate new physics. Moreover, their dependence on data alignment within computational grids hampers their adaptability, especially in the context of sparse data availability in space and time. To address these limitations, we developed PINNICLE (Physics-Informed Neural Networks for Ice and CLimatE), an open-source Python library dedicated to ice sheet modeling. PINNICLE seamlessly integrates observational data and physical laws, facilitating the solution of both forward and inverse problems within a single framework. PINNICLE currently supports a variety of conservation laws, including the Shelfy-Stream Approximation (SSA), MOno-Layer Higher-Order (MOLHO) models, and mass conservation equations, for both time-independent and time-dependent simulations. The library is user-friendly, requiring only the setting of a few hyperparameters for standard modeling tasks, while advanced users can define custom models within the framework. Additionally, PINNICLE is based on the DeepXDE library, which supports widely used machine learning packages such as TensorFlow, PyTorch, and JAX, enabling users to select the backend that best fits their hardware. We describe here the implementation of PINNICLE and showcase this library with examples across the Greenland and Antarctic ice sheets for a range of forward and inverse problems.more » « less
-
Real-time accurate solutions of large-scale complex dynamical systems are critically needed for control, optimization, uncertainty quantification, and decision-making in practical engineering and science applications, particularly in digital twin contexts. Recent research on hybrid approaches combining numerical methods and machine learning in end-to-end training has shown significant improvements over either approach alone. However, using neural networks as surrogate models generally exhibits limitations in generalizability over different settings and in capturing the evolution of solution discontinuities. In this work, we develop a model-constrained discontinuous Galerkin Network DGNet approach, a significant extension to our previous work, for compressible Euler equations with out-of-distribution generalization. The core of DGNet is the synergy of several key strategies: (i) leveraging time integration schemes to capture temporal correlation and taking advantage of neural network speed for computation time reduction. This is the key to the temporal discretization-invariant property of DGNet; (ii) employing a model-constrained approach to ensure the learned tangent slope satisfies governing equations; (iii) utilizing a DG-inspired architecture for GNN where edges represent Riemann solver surrogate models and nodes represent volume integration correction surrogate models, enabling capturing discontinuity capability, aliasing error reduction, and mesh discretization generalizability. Such a design allows DGNet to learn the DG spatial discretization accurately; (iv) developing an input normalization strategy that allows surrogate models to generalize across different initial conditions, geometries, meshes, boundary conditions, and solution orders. In fact, the normalization is the key to spatial discretization-invariance for DGNet; and (v) incorporating a data randomization technique that not only implicitly promotes agreement between surrogate models and true numerical models up to second-order derivatives, ensuring long-term stability and prediction capacity, but also serves as a data generation engine during training, leading to enhanced generalization on unseen data. To validate the theoretical results, effectiveness, stability, and generalizability of our novel DGNet approach, we present comprehensive numerical results for 1D and 2D compressible Euler equation problems, including Sod Shock Tube, Lax Shock Tube, Isentropic Vortex, Forward Facing Step, Scramjet, Airfoil, Euler Benchmarks, Double Mach Reflection, and a Hypersonic Sphere Cone benchmark.more » « less
-
The inception of physics-constrained or physics-informed machine learning represents a paradigm shift, addressing the challenges associated with data scarcity and enhancing model interpretability. This innovative approach incorporates the fundamental laws of physics as constraints, guiding the training process of machine learning models. In this work, the physics-constrained convolutional recurrent neural network is further extended for solving spatial-temporal partial differential equations with arbitrary boundary conditions. Two notable advancements are introduced: the implementation of boundary conditions as soft constraints through finite difference-based differentiation, and the establishment of an adaptive weighting mechanism for the optimal allocation of weights to various losses. These enhancements significantly augment the network's ability to manage intricate boundary conditions and expedite the training process. The efficacy of the proposed model is validated through its application to two-dimensional phase transition, fluid dynamics, and reaction-diffusion problems, which are pivotal in materials modeling. Compared to traditional physics-constrained neural networks, the physics-constrained convolutional recurrent neural network demonstrates a tenfold increase in prediction accuracy within a similar computational budget. Moreover, the model's exceptional performance in extrapolating solutions for the Burgers' equation underscores its utility. Therefore, this research establishes the physics-constrained recurrent neural network as a viable surrogate model for sophisticated spatial-temporal PDE systems, particularly beneficial in scenarios plagued by sparse and noisy datasets.more » « less
