In this work, we propose a joint operator learning method for reconstructing images of conductivity coefficients from boundary data. Inspired by the idea of using partial differential equation (PDE) solvers as preconditioners for this inverse problem, we investigate a “solver-in-the-loop” training mechanism. This mechanism allows learnable parameters in a PDE solver module to interact with those in neural networks for image reconstruction. Specifically, we employ a fractional Laplace--Beltrami operator with a learnable fractional order to transform boundary data into high-dimensional features. These features are then used as input to a neural network, significantly improving reconstruction accuracy. For this purpose, we develop a Learning-Automated FEM (LA-FEM) package with PyTorch as the backend to support this “solver-in-the-loop” property. The new LA-FEM module allows automatic differentiation of an objective function to propagate through both the PDE solver for the forward problem and the coupled neural networks for the inverse problem.
more »
« less
Elliptic PDE learning is provably data-efficient
Partial differential equations (PDE) learning is an emerging field that combines physics and machine learning to recover unknown physical systems from experimental data. While deep learning models traditionally require copious amounts of training data, recent PDE learning techniques achieve spectacular results with limited data availability. Still, these results are empirical. Our work provides theoretical guarantees on the number of input–output training pairs required in PDE learning. Specifically, we exploit randomized numerical linear algebra and PDE theory to derive a provably data-efficient algorithm that recovers solution operators of three-dimensional uniformly elliptic PDEs from input–output data and achieves an exponential convergence rate of the error with respect to the size of the training dataset with an exceptionally high probability of success.
more »
« less
- Award ID(s):
- 2045646
- PAR ID:
- 10508645
- Publisher / Repository:
- PNAS
- Date Published:
- Journal Name:
- Proceedings of the National Academy of Sciences
- Volume:
- 120
- Issue:
- 39
- ISSN:
- 0027-8424
- Subject(s) / Keyword(s):
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Recently, a broad class of linear delayed and ODE-PDEs systems was shown to have an equivalent representation using Partial Integral Equations (PIEs). In this paper, we use this PIE representation, combined with algorithms for convex optimization of Partial Integral (PI) operators to bound the H2-norm for input-output systems of this class. Specifically, the methods proposed here apply to delayed and ODE-PDE systems (including delayed PDE systems) in one or two spatial variables where the disturbance does not enter through the boundary. For such systems, we define a notion of H2-norm using an initial state-to-output framework and show that this notion reduces to more traditional concepts under the assumption of existence of a strongly continuous semigroup. Next, we consider input-output systems for which there exists a PIE representation and for such systems show that computing a minimal upper bound on the H2-norm of delayed and PDE systems can be equivalently formulated as a convex optimization problem subject to linear PI operator inequalities (LPIs). We convert, then, these optimization problems to Semi-Definite Programming (SDP) problems using the PIETOOLS toolbox. Finally, we apply the results to several numerical examples – focusing on time-delay systems (TDS) for which comparable H2 approximation results are available in the literature. The numerical results demonstrate the accuracy of the computed upper bound on the H2-norm.more » « less
-
Abstract. We propose a novel machine learning framework for solving optimization problems governed by large-scale partial differential equations (PDEs) with high-dimensional random parameters. Such optimization under uncertainty (OUU) problems may be computationally prohibitive using classical methods, as the estimation of risk measures may require a large number of solutions of an expensive-to-solve PDE at every iteration of an optimization algorithm. To address this challenge, we propose the multi-input reduced basis derivative-informed neural operator (MR-DINO), as an approximation to the PDE solution operator. The MR-DINO has the combined merits of (1) being able to accurately approximate not only the mapping from the joint inputs of random parameters and optimization variables to the PDE state, but also its derivative with respect to the optimization variables, (2) using a reduced basis architecture that can be efficiently constructed and is scalable to high-dimensional OUU problems, and (3) requiring only a limited amount of training data to achieve high accuracy for both the PDE solution and the OUU solution. We demonstrate the capability of MR-DINOs through several numerical experiments, including the risk-averse control of a semilinear elliptic PDE and the steady-state Navier–Stokes equations in two and three spatial dimensions. Across the examples, MR-DINOs offer [Formula: see text]–[Formula: see text] reductions in execution times and produce OUU solutions of comparable accuracies to PDE-based OUU solutions using standard quasi-Newton methods (BFGS/L-BFGS-B) while being one to two orders of magnitude more cost-efficient after accounting for construction costs. Moreover, the construction cost of MR-DINOs can be amortized over many optimization runs with differing objectives and constraints.more » « less
-
Abstract We automate the process of machine learning correlations between knot invariants. For nearly 200 000 distinct sets of input knot invariants together with an output invariant, we attempt to learn the output invariant by training a neural network on the input invariants. Correlation between invariants is measured by the accuracy of the neural network prediction, and bipartite or tripartite correlations are sequentially filtered from the input invariant sets so that experiments with larger input sets are checking for true multipartite correlation. We rediscover several known relationships between polynomial, homological, and hyperbolic knot invariants, while also finding novel correlations which are not explained by known results in knot theory. These unexplained correlations strengthen previous observations concerning links between Khovanov and knot Floer homology. Our results also point to a new connection between quantum algebraic and hyperbolic invariants, similar to the generalized volume conjecture.more » « less
-
In this paper, we propose a systematic approach for accelerating finite element-type methods by machine learning for the numerical solution of partial differential equations (PDEs). The main idea is to use a neural network to learn the solution map of the PDEs and to do so in an element-wise fashion. This map takes input of the element geometry and the PDE’s parameters on that element, and gives output of two operators: (1) the in2out operator for inter-element communication, and (2) the in2sol operator (Green’s function) for element-wise solution recovery. A significant advantage of this approach is that, once trained, this network can be used for the numerical solution of the PDE for any domain geometry and any parameter distribution without retraining. Also, the training is significantly simpler since it is done on the element level instead on the entire domain. We call this approach element learning. This method is closely related to hybridizable discontinuous Galerkin (HDG) methods in the sense that the local solvers of HDG are replaced by machine learning approaches. Numerical tests are presented for an example PDE, the radiative transfer or radiation transport equation, in a variety of scenarios with idealized or realistic cloud fields, with smooth or sharp gradient in the cloud boundary transition. Under a fixed accuracy level of $$10^{−3}$$ in the relative $L^2$ error, and polynomial degree $p = 6$ in each element, we observe an approximately $$5$$ to $10$ times speed-up by element learning compared to a classical finite element-type method.more » « less
An official website of the United States government

