Adjoint systems are widely used to inform control, optimization, and design in systems described by ordinary differential equations or differentialalgebraic equations. In this paper, we explore the geometric properties and develop methods for such adjoint systems. In particular, we utilize symplectic and presymplectic geometry to investigate the properties of adjoint systems associated with ordinary differential equations and differentialalgebraic equations, respectively. We show that the adjoint variational quadratic conservation laws, which are key to adjoint sensitivity analysis, arise from (pre)symplecticity of such adjoint systems. We discuss various additional geometric properties of adjoint systems, such as symmetries and variational characterizations. For adjoint systems associated with a differentialalgebraic equation, we relate the index of the differentialalgebraic equation to the presymplectic constraint algorithm of Gotay et al. (J Math Phys 19(11):2388–2399, 1978). As an application of this geometric framework, we discuss how the adjoint variational quadratic conservation laws can be used to compute sensitivities of terminal or running cost functions. Furthermore, we develop structurepreserving numerical methods for such systems using Galerkin Hamiltonian variational integrators (Leok and Zhang in IMA J. Numer. Anal. 31(4):1497–1532, 2011) which admit discrete analogues of these quadratic conservation laws. We additionally show that such methods are natural, in the sense that reduction, forming the adjoint system, and discretization all commute, for suitable choices of these processes. We utilize this naturality to derive a variational error analysis result for the presymplectic variational integrator that we use to discretize the adjoint DAE system. Finally, we discuss the application of adjoint systems in the context of optimal control problems, where we prove a similar naturality result.
Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher.
Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?
Some links on this page may take you to nonfederal websites. Their policies may differ from this site.

Abstract 
Abstract M. Kruskal showed that each continuoustime nearly periodic dynamical system admits a formal
U (1)symmetry, generated by the socalled rotorate. When the nearly periodic system is also Hamiltonian, Noether’s theorem implies the existence of a corresponding adiabatic invariant. We develop a discretetime analog of Kruskal’s theory. Nearly periodic maps are defined as parameterdependent diffeomorphisms that limit to rotations along aU (1)action. When the limiting rotation is nonresonant, these maps admit formalU (1)symmetries to all orders in perturbation theory. For Hamiltonian nearly periodic maps on exact presymplectic manifolds, we prove that the formalU (1)symmetry gives rise to a discretetime adiabatic invariant using a discretetime extension of Noether’s theorem. When the unperturbedU (1)orbits are contractible, we also find a discretetime adiabatic invariant for mappings that are merely presymplectic, rather than Hamiltonian. As an application of the theory, we use it to develop a novel technique for geometric integration of noncanonical Hamiltonian systems on exact symplectic manifolds. 
Abstract A variational formulation for accelerated optimization on normed vector spaces was recently introduced in Wibisono et al. (PNAS 113:E7351–E7358, 2016), and later generalized to the Riemannian manifold setting in Duruisseaux and Leok (SJMDS, 2022a). This variational framework was exploited on normed vector spaces in Duruisseaux et al. (SJSC 43:A2949–A2980, 2021) using timeadaptive geometric integrators to design efficient explicit algorithms for symplectic accelerated optimization, and it was observed that geometric discretizations which respect the timerescaling invariance and symplecticity of the Lagrangian and Hamiltonian flows were substantially less prone to stability issues, and were therefore more robust, reliable, and computationally efficient. As such, it is natural to develop timeadaptive Hamiltonian variational integrators for accelerated optimization on Riemannian manifolds. In this paper, we consider the case of Riemannian manifolds embedded in a Euclidean space that can be characterized as the level set of a submersion. We will explore how holonomic constraints can be incorporated in discrete variational integrators to constrain the numerical discretization of the Riemannian Hamiltonian system to the Riemannian manifold, and we will test the performance of the resulting algorithms by solving eigenvalue and Procrustes problems formulated as optimization problems on the unit sphere and Stiefel manifold.

Geometric numerical integration has recently been exploited to design symplectic accelerated optimization algorithms by simulating the Bregman Lagrangian and Hamiltonian systems from the variational framework introduced by Wibisono et al. In this paper, we discuss practical considerations which can significantly boost the computational performance of these optimization algorithms and considerably simplify the tuning process. In particular, we investigate how momentum restarting schemes ameliorate computational efficiency and robustness by reducing the undesirable effect of oscillations and ease the tuning process by making timeadaptivity superfluous. We also discuss how temporal looping helps avoiding instability issues caused by numerical precision, without harming the computational efficiency of the algorithms. Finally, we compare the efficiency and robustness of different geometric integration techniques and study the effects of the different parameters in the algorithms to inform and simplify tuning in practice. From this paper emerge symplectic accelerated optimization algorithms whose computational efficiency, stability and robustness have been improved, and which are now much simpler to use and tune for practical applications.more » « lessFree, publiclyaccessible full text available November 2, 2024

Riemannian submanifold optimization with momentum is computationally challenging because, to ensure that the iterates remain on the submanifold, we often need to solve difficult differential equations. Here, we simplify such difficulties for a class of structured symmetric positivedefinite matrices with the affineinvariant metric. We do so by proposing a generalized version of the Riemannian normal coordinates that dynamically orthonormalizes the metric and locally converts the problem into an unconstrained problem in the Euclidean space. We use our approach to simplify existing approaches for structured covariances and develop matrixinversefree 2ndorder optimizers for deep learning in low precision settings.more » « less

Incorporating prior knowledge of physics laws and structural properties of dynamical systems into the design of deep learning architectures has proven to be a powerful technique for improving their computational efficiency and generalization capacity. Learning accurate models of robot dynamics is critical for safe and stable control. Autonomous mobile robots, including wheeled, aerial, and underwater vehicles, can be modeled as controlled Lagrangian or Hamiltonian rigidbody systems evolving on matrix Lie groups. In this paper, we introduce a new structurepreserving deep learning architecture, the Lie group Forced Variational Integrator Network (LieFVIN), capable of learning controlled Lagrangian or Hamiltonian dynamics on Lie groups, either from positionvelocity or positiononly data. By design, LieFVINs preserve both the Lie group structure on which the dynamics evolve and the symplectic structure underlying the Hamiltonian or Lagrangian systems of interest. The proposed architecture learns surrogate discretetime flow maps allowing accurate and fast prediction without numericalintegrator, neuralODE, or adjoint techniques, which are needed for vector fields. Furthermore, the learnt discretetime dynamics can be utilized with computationally scalable discretetime (optimal) control strategies.more » « less

This paper develops the theory of discrete Dirac reduction of discrete Lagrange–Dirac systems with an abelian symmetry group acting on the configuration space. We begin with the linear theory and, then, we extend it to the nonlinear setting using retraction compatible charts. We consider the reduction of both the discrete Dirac structure and the discrete Lagrange–Pontryagin principle, and show that they both lead to the same discrete Lagrange–Poincaré–Dirac equations. The coordinatization of the discrete reduced spaces relies on the notion of discrete connections on principal bundles. At last, we demonstrate the method obtained by applying it to a charged particle in a magnetic field, and to the double spherical pendulum.more » « less

A variational framework for accelerated optimization was recently introduced on normed vector spaces and Riemannian manifolds in [1] and [2]. It was observed that a careful combination of timeadaptivity and symplecticity in the numerical integration can result in a significant gain in computational efficiency. It is however well known that symplectic integrators lose their nearenergy preservation properties when variable timesteps are used. The most common approach to circumvent this problem involves the Poincaré transformation on the Hamiltonian side, and was used in [3] to construct efficient explicit algorithms for symplectic accelerated optimization. However, the current formulations of Hamiltonian variational integrators do not make intrinsic sense on more general spaces such as Riemannian manifolds and Lie groups. In contrast, Lagrangian variational integrators are welldefined on manifolds, so we develop here a framework for timeadaptivity in Lagrangian variational integrators and use the resulting geometric integrators to solve optimization problems on vector spaces and Lie groups.more » « less