skip to main content

Title: Time-inconsistent stochastic optimal control problems and backward stochastic volterra integral equations
An optimal control problem is considered for a stochastic differential equation with the cost functional determined by a backward stochastic Volterra integral equation (BSVIE, for short). This kind of cost functional can cover the general discounting (including exponential and non-exponential) situations with a recursive feature. It is known that such a problem is time-inconsistent in general. Therefore, instead of finding a global optimal control, we look for a time-consistent locally near optimal equilibrium strategy. With the idea of multi-person differential games, a family of approximate equilibrium strategies is constructed associated with partitions of the time intervals. By sending the mesh size of the time interval partition to zero, an equilibrium Hamilton–Jacobi–Bellman (HJB, for short) equation is derived, through which the equilibrium value function and an equilibrium strategy are obtained. Under certain conditions, a verification theorem is proved and the well-posedness of the equilibrium HJB is established. As a sort of Feynman–Kac formula for the equilibrium HJB equation, a new class of BSVIEs (containing the diagonal value Z ( r , r ) of Z (⋅ , ⋅)) is naturally introduced and the well-posedness of such kind of equations is briefly presented.
Authors:
;
Editors:
Buttazzo, G.; Casas, E.; de Teresa, L.; Glowinski, R.; Leugering, G.; Trélat, E.; Zhang, X.
Award ID(s):
1812921
Publication Date:
NSF-PAR ID:
10220031
Journal Name:
ESAIM: Control, Optimisation and Calculus of Variations
Volume:
27
Page Range or eLocation-ID:
22
ISSN:
1292-8119
Sponsoring Org:
National Science Foundation
More Like this
  1. This paper studies an optimal stochastic impulse control problem in a finite time horizon with a decision lag, by which we mean that after an impulse is made, a fixed number units of time has to be elapsed before the next impulse is allowed to be made. The continuity of the value function is proved. A suitable version of dynamic programming principle is established, which takes into account the dependence of state process on the elapsed time. The corresponding Hamilton-Jacobi-Bellman (HJB) equation is derived, which exhibits some special feature of the problem. The value function of this optimal impulse controlmore »problem is characterized as the unique viscosity solution to the corresponding HJB equation. An optimal impulse control is constructed provided the value function is given. Moreover, a limiting case with the waiting time approaching 0 is discussed.« less
  2. In magnetic confinement fusion devices, the equilibrium configuration of a plasma is determined by the balance between the hydrostatic pressure in the fluid and the magnetic forces generated by an array of external coils and the plasma itself. The location of the plasma is not known a priori and must be obtained as the solution to a free boundary problem. The partial differential equation that determines the behavior of the combined magnetic field depends on a set of physical parameters (location of the coils, intensity of the electric currents going through them, magnetic permeability, etc.) that are subject to uncertaintymore »and variability. The confinement region is in turn a function of these stochastic parameters as well. In this work, we consider variations on the current intensities running through the external coils as the dominant source of uncertainty. This leads to a parameter space of dimension equal to the number of coils in the reactor. With the aid of a surrogate function built on a sparse grid in parameter space, a Monte Carlo strategy is used to explore the effect that stochasticity in the parameters has on important features of the plasma boundary such as the location of the x-point, the strike points, and shaping attributes such as triangularity and elongation. The use of the surrogate function reduces the time required for the Monte Carlo simulations by factors that range between 7 and over 30.« less
  3. This paper investigates optimal power management of a fuel cell hybrid small unmanned aerial vehicle (sUAV) from the perspective of endurance (time of flight) maximization in a stochastic environment. Stochastic drift counteraction optimal control is exploited to obtain an optimal policy for power management that coordinates the operation of the fuel cell and battery to maximize the expected flight time while accounting for the limits on the rate of change of fuel cell power output and the orientation dependence of fuel cell efficiency. The proposed power management strategy accounts for known statistics in transitions of propeller power and climb anglemore »during the mission, but does not require the exact preview of their time histories. The optimal control policy is generated offline using value iterations implemented in Cython, demonstrating an order of magnitude speedup as compared to MATLAB. It is also shown that the value iterations can be further sped up using a discount factor, but at the cost of decreased performance. Simulation results for a 1.5 kg sUAV are reported that illustrate the optimal coordination between the fuel cell and the battery during aircraft maneuvers, including a turnpike in the battery state of charge (SOC) trajectory. As the fuel cell is not able to support fast changes in power output, the optimal policy is shown to charge the battery to the turnpike value if starting from a low initial SOC value. If starting from a high SOC value, the battery energy is used till a turnpike value of the SOC is reached with further discharge delayed to later in the flight. For the specific scenarios and simulated sUAV parameters considered, the results indicate the capability of up to 2.7 h of flight time.« less
  4. We consider particles obeying Langevin dynamics while being at known positions and having known velocities at the two end-points of a given interval. Their motion in phase space can be modeled as an Ornstein–Uhlenbeck process conditioned at the two end-points—a generalization of the Brownian bridge. Using standard ideas from stochastic optimal control we construct a stochastic differential equation (SDE) that generates such a bridge that agrees with the statistics of the conditioned process, as a degenerate diffusion. Higher order linear diffusions are also considered. In general, a time-varying drift is sufficient to modify the prior SDE and meet the end-pointmore »conditions. When the drift is obtained by solving a suitable differential Lyapunov equation, the SDE models correctly the statistics of the bridge. These types of models are relevant in controlling and modeling distribution of particles and the interpolation of density functions.« less
  5. Abstract

    We consider an optimal control problem where the state equations are a coupled hyperbolic–elliptic system. This system arises in elastodynamics with piezoelectric effects—the elastic stress tensor is a function of elastic displacement and electric potential. The electric flux acts as the control variable and bound constraints on the control are considered. We develop a complete analysis for the state equations and the control problem. The requisite regularity on the control, to show the well-posedness of the state equations, is enforced using the cost functional. We rigorously derive the first-order necessary and sufficient conditions using adjoint equations and further studymore »their well-posedness. For spatially discrete (time-continuous) problems, we show the convergence of our numerical scheme. Three-dimensional numerical experiments are provided showing convergence properties of a fully discrete method and the practical applicability of our approach.

    « less