We provide a short introduction to the devising of a special type of methods for numerically approximating the solution of Hamiltonian partial differential equations. These methods use Galerkin space-discretizations which result in a system of ODEs displaying a discrete version of the Hamiltonian structure of the original system. The resulting system of ODEs is then discretized by a symplectic time-marching method. This combination results in high-order accurate, fully discrete methods which can preserve the invariants of the Hamiltonian defining the ODE system. We restrict our attention to linear Hamiltonian systems, as the main results can be obtained easily and directly, and are applicable to many Hamiltonian systems of practical interest including acoustics, elastodynamics, and electromagnetism. After a brief description of the Hamiltonian systems of our interest, we provide a brief introduction to symplectic time-marching methods for linear systems of ODEs which does not require any background on the subject. We describe then the case in which finite-difference space-discretizations are used and focus on the popular Yee scheme (1966) for electromagnetism. Finally, we consider the case of finite-element space discretizations. The emphasis is placed on the conservation properties of the fully discrete schemes. We end by describing ongoing work.
more »
« less
Time discretizations of Wasserstein–Hamiltonian flows
We study discretizations of Hamiltonian systems on the probability density manifold equipped with the L 2 L^2 -Wasserstein metric. Based on discrete optimal transport theory, several Hamiltonian systems on a graph (lattice) with different weights are derived, which can be viewed as spatial discretizations of the original Hamiltonian systems. We prove consistency of these discretizations. Furthermore, by regularizing the system using the Fisher information, we deduce an explicit lower bound for the density function, which guarantees that symplectic schemes can be used to discretize in time. Moreover, we show desirable long time behavior of these symplectic schemes, and demonstrate their performance on several numerical examples. Finally, we compare the present approach with the standard viscosity methodology.
more »
« less
- Award ID(s):
- 1830225
- NSF-PAR ID:
- 10341580
- Date Published:
- Journal Name:
- Mathematics of computation
- ISSN:
- 1088-6842
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Arguably, the two most popular accelerated or momentum-based optimization methods in machine learning are Nesterov’s accelerated gradient and Polyaks’s heavy ball, both corresponding to different discretizations of a particular second order differential equation with friction. Such connections with continuous-time dynamical systems have been instrumental in demystifying acceleration phenomena in optimization. Here we study structure-preserving discretizations for a certain class of dissipative (conformal) Hamiltonian systems, allowing us to analyse the symplectic structure of both Nesterov and heavy ball, besides providing several new insights into these methods. Moreover, we propose a new algorithm based on a dissipative relativistic system that normalizes the momentum and may result in more stable/faster optimization. Importantly, such a method generalizes both Nesterov and heavy ball, each being recovered as distinct limiting cases, and has potential advantages at no additional cost.more » « less
-
more » « less
Summary While system dynamics are usually derived in continuous time, respective model‐based optimal control problems can only be solved numerically, ie, as discrete‐time approximations. Thus, the performance of control methods depends on the choice of numerical integration scheme. In this paper, we present a first‐order discretization of linear quadratic optimal control problems for mechanical systems that is structure preserving and hence preferable to standard methods. Our approach is based on symplectic integration schemes and thereby inherits structure from the original continuous‐time problem. Starting from a symplectic discretization of the system dynamics, modified discrete‐time Riccati equations are derived, which preserve the Hamiltonian structure of optimal control problems in addition to the mechanical structure of the control system. The method is extended to optimal tracking problems for nonlinear mechanical systems and evaluated in several numerical examples. Compared to standard discretization, it improves the approximation quality by orders of magnitude. This enables low‐bandwidth control and sensing in real‐time autonomous control applications.
-
more » « less
Abstract A variational formulation for accelerated optimization on normed vector spaces was recently introduced in Wibisono et al. (PNAS 113:E7351–E7358, 2016), and later generalized to the Riemannian manifold setting in Duruisseaux and Leok (SJMDS, 2022a). This variational framework was exploited on normed vector spaces in Duruisseaux et al. (SJSC 43:A2949–A2980, 2021) using time-adaptive geometric integrators to design efficient explicit algorithms for symplectic accelerated optimization, and it was observed that geometric discretizations which respect the time-rescaling invariance and symplecticity of the Lagrangian and Hamiltonian flows were substantially less prone to stability issues, and were therefore more robust, reliable, and computationally efficient. As such, it is natural to develop time-adaptive Hamiltonian variational integrators for accelerated optimization on Riemannian manifolds. In this paper, we consider the case of Riemannian manifolds embedded in a Euclidean space that can be characterized as the level set of a submersion. We will explore how holonomic constraints can be incorporated in discrete variational integrators to constrain the numerical discretization of the Riemannian Hamiltonian system to the Riemannian manifold, and we will test the performance of the resulting algorithms by solving eigenvalue and Procrustes problems formulated as optimization problems on the unit sphere and Stiefel manifold.
-
Geometric numerical integration has recently been exploited to design symplectic accelerated optimization algorithms by simulating the Bregman Lagrangian and Hamiltonian systems from the variational framework introduced by Wibisono et al. In this paper, we discuss practical considerations which can significantly boost the computational performance of these optimization algorithms and considerably simplify the tuning process. In particular, we investigate how momentum restarting schemes ameliorate computational efficiency and robustness by reducing the undesirable effect of oscillations and ease the tuning process by making time-adaptivity superfluous. We also discuss how temporal looping helps avoiding instability issues caused by numerical precision, without harming the computational efficiency of the algorithms. Finally, we compare the efficiency and robustness of different geometric integration techniques and study the effects of the different parameters in the algorithms to inform and simplify tuning in practice. From this paper emerge symplectic accelerated optimization algorithms whose computational efficiency, stability and robustness have been improved, and which are now much simpler to use and tune for practical applications.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1090/mcom/3726
