More Like this

1. Data-Driven Model Reduction for a Class of Semi-Explicit DAEs Using the Loewner Framewor

   https://doi.org/10.1007/978-3-030-53905-4_7  

   Antoulas, A. C. ; Gosea, I. V. ; Heinkenschloss, M. ( October 2020 , Progress in Differential-Algebraic Equations II)

   Reis, T. ; Grundel, S. ; Schöps, S. (Ed.)

   This paper introduces a modified version of the recent data-driven Loewner framework to compute reduced order models (ROMs) for a class of semi-explicit differential algebraic equation (DAE) systems, which include the semi-discretized linearized Navier-Stokes /Oseen equations. The modified version estimates the polynomial part of the original transfer function from data and incorporate this estimate into the Loewner ROM construction. Without this proposed modification the transfer function of the Loewner ROM is strictly proper, i.e., goes to zero as the magnitude of the frequency goes to infinity, and therefore may have a different behavior for large frequencies than the transfer function of the original system. The modification leads to a Loewner ROM with a transfer function that has a strictly proper and a polynomial part, just as the original model. This leads to better approximations for transfer functioncomponents in which the coefficients in the polynomial part are not too small. The construction of the improved Loewner ROM is described and the improvement is demonstrated on a large-scale system governed by the semi-discretized Oseen equations. 

   more » « less
2. Waveform inversion via reduced order modeling

   https://doi.org/10.1190/GEO2022-0070.1  

   Borcea, Liliana ; Garnier, Josselin ; Mamonov, Alexander V. ; Zimmerling, Jörn ( March 2023 , GEOPHYSICS)

   We introduce a novel approach to waveform inversion based on a data-driven reduced order model (ROM) of the wave operator. The presentation is for the acoustic wave equation, but the approach can be extended to elastic or electromagnetic waves. The data are time resolved measurements of the pressure wave gathered by an acquisition system that probes the unknown medium with pulses and measures the generated waves. We propose to solve the inverse problem of velocity estimation by minimizing the square misfit between the ROM computed from the recorded data and the ROM computed from the modeled data, at the current guess of the velocity. We give a step by step computation of the ROM, which depends nonlinearly on the data and yet can be obtained from them in a noniterative fashion, using efficient methods from linear algebra. We also explain how to make the ROM robust to data inaccuracy. The ROM computation requires the full array response matrix gathered with colocated sources and receivers. However, we find that the computation can deal with an approximation of this matrix, obtained from towed-streamer data using interpolation and reciprocity on-the-fly. Although the full-waveform inversion approach of nonlinear least-squares data fitting is challenging without low-frequency information, due to multiple minima of the data fit objective function, we find that the ROM misfit objective function has better behavior, even for a poor initial guess. We also find by explicit computation of the objective functions in a simple setting that the ROM misfit objective function has convexity properties, whereas the least-squares data fit objective function displays multiple local minima. 

   more » « less
3. Understanding the acceleration phenomenon via high-resolution differential equations

   https://doi.org/10.1007/s10107-021-01681-8  

   Shi, Bin ; Du, Simon S. ; Jordan, Michael I. ; Su, Weijie J. ( July 2021 , Mathematical Programming)

   Gradient-based optimization algorithms can be studied from the perspective of limiting ordinary differential equations (ODEs). Motivated by the fact that existing ODEs do not distinguish between two fundamentally different algorithms—Nesterov’s accelerated gradient method for strongly convex functions (NAG-) and Polyak’s heavy-ball method—we study an alternative limiting process that yieldshigh-resolution ODEs. We show that these ODEs permit a general Lyapunov function framework for the analysis of convergence in both continuous and discrete time. We also show that these ODEs are more accurate surrogates for the underlying algorithms; in particular, they not only distinguish between NAG- and Polyak’s heavy-ball method, but they allow the identification of a term that we refer to as “gradient correction” that is present in NAG- but not in the heavy-ball method and is responsible for the qualitative difference in convergence of the two methods. We also use the high-resolution ODE framework to study Nesterov’s accelerated gradient method for (non-strongly) convex functions, uncovering a hitherto unknown result—that NAG- minimizes the squared gradient norm at an inverse cubic rate. Finally, by modifying the high-resolution ODE of NAG-, we obtain a family of new optimization methods that are shown to maintain the accelerated convergence rates of NAG- for smooth convex functions.

    

   more » « less
4. Reachability Analysis of a General Class of Neural Ordinary Differential Equations

   https://doi.org/10.1007/978-3-031-15839-1_15  

   Manzanas Lopez, D. ; Musau, P. ; Hamilton, N.P. ; Johnson, T.T. ( August 2022 , Formal Modeling and Analysis of Timed Systems. FORMATS 2022)

   Bogomolov, S. ; Parker, D. (Ed.)

   Continuous deep learning models, referred to as Neural Ordinary Differential Equations (Neural ODEs), have received considerable attention over the last several years. Despite their burgeoning impact, there is a lack of formal analysis techniques for these systems. In this paper, we consider a general class of neural ODEs with varying architectures and layers, and introduce a novel reachability framework that allows for the formal analysis of their behavior. The methods developed for the reachability analysis of neural ODEs are implemented in a new tool called NNVODE. Specifically, our work extends an existing neural network verification tool to support neural ODEs. We demonstrate the capabilities and efficacy of our methods through the analysis of a set of benchmarks that include neural ODEs used for classification, and in control and dynamical systems, including an evaluation of the efficacy and capabilities of our approach with respect to existing software tools within the continuous-time systems reachability literature, when it is possible to do so. 

   more » « less
5. Solving inverse scattering problems via reduced-order model embedding procedures

   https://doi.org/10.1088/1361-6420/ad149d  

   Zimmerling, Jörn ; Druskin, Vladimir ; Guddati, Murthy ; Cherkaev, Elena ; Remis, Rob ( December 2023 , Inverse Problems)

   We present a reduced-order model (ROM) methodology for inverse scattering problems in which the ROMs are data-driven, i.e. they are constructed directly from data gathered by sensors. Moreover, the entries of the ROM contain localised information about the coefficients of the wave equation. We solve the inverse problem by embedding the ROM in physical space. Such an approach is also followed in the theory of ‘optimal grids,’ where the ROMs are interpreted as two-point finite-difference discretisations of an underlying set of equations of a first-order continuous system on this special grid. Here, we extend this line of work to wave equations and introduce a new embedding technique, which we callKrein embedding, since it is inspired by Krein’s seminal work on vibrations of a string. In this embedding approach, an adaptive grid and a set of medium parameters can be directly extracted from a ROM and we show that several limitations of optimal grid embeddings can be avoided. Furthermore, we show how Krein embedding is connected to classical optimal grid embedding and that convergence results for optimal grids can be extended to this novel embedding approach. Finally, we also briefly discuss Krein embedding for open domains, that is, semi-infinite domains that extend to infinity in one direction.

    

   more » « less