An official website of the United States government
This paper is concerned with two-person mean-field linear-quadratic non-zero sum stochastic differential games in an infinite horizon. Both open-loop and closed-loop Nash equilibria are introduced. The existence of an open-loop Nash equilibrium is characterized by the solvability of a system of mean-field forward-backward stochastic differential equations in an infinite horizon and the convexity of the cost functionals, and the closed-loop representation of an open-loop Nash equilibrium is given through the solution to a system of two coupled non-symmetric algebraic Riccati equations. The existence of a closed-loop Nash equilibrium is characterized by the solvability of a system of two coupled symmetric algebraic Riccati equations. Two-person mean-field linear-quadratic zero-sum stochastic differential games in an infinite horizon are also considered. Both the existence of open-loop and closed-loop saddle points are characterized by the solvability of a system of two coupled generalized algebraic Riccati equations with static stabilizing solutions. Mean-field linear-quadratic stochastic optimal control problems in an infinite horizon are discussed as well, for which it is proved that the open-loop solvability and closed-loop solvability are equivalent.
more »
« less
Linear-Quadratic Optimal Controls for Stochastic Volterra Integral Equations: Causal State Feedback and Path-Dependent Riccati Equations
A linear-quadratic optimal control problem for a forward stochastic Volterra integral equation (FSVIE) is considered. Under the usual convexity conditions, open-loop optimal control exists, which can be characterized by the optimality system, a coupled system of an FSVIE and a type-II backward SVIE (BSVIE). To obtain a causal state feedback representation for the open-loop optimal control, a path-dependent Riccati equation for an operator-valued function is introduced, via which the optimality system can be decoupled. In the process of decoupling, a type-III BSVIE is introduced whose adapted solution can be used to represent the adapted M-solution of the corresponding type-II BSVIE. Under certain conditions, it is proved that the path-dependent Riccati equation admits a unique solution, which means that the decoupling field for the optimality system is found. Therefore, a causal state feedback representation of the open-loop optimal control is constructed. An additional interesting finding is that when the control only appears in the diffusion term, not in the drift term of the state system, the causal state feedback reduces to a Markovian state feedback.
more »
« less
- Award ID(s):
- 2305475
- PAR ID:
- 10499995
- Publisher / Repository:
- SIAM
- Date Published:
- Journal Name:
- SIAM Journal on Control and Optimization
- Volume:
- 61
- Issue:
- 4
- ISSN:
- 0363-0129
- Page Range / eLocation ID:
- 2595 to 2629
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
A general problem encompassing output regulation and pattern generation can be formulated as the design of controllers to achieve convergence to a persistent trajectory within the zero dynamics on which an output vanishes. We develop an optimal control theory for such design by adding the requirement to minimize the H2 norm of a closed-loop transfer function. Within the framework of eigenstructure assignment, the optimal control is proven identical to the standard H2 control in form. However, the solution to the Riccati equation for the linear quadratic regulator is not stabilizing. Instead it partially stabilizes the closed-loop dynamics excluding the zero dynamics. The optimal control architecture is shown to have the feedback of the deviation from the subspace of the zero dynamics and the feedforward of the control input to remain in the subspace.more » « less
-
Abstract The Moore-Gibson-Thompson [MGT] dynamics is considered. This third order in time evolution arises within the context of acoustic wave propagation with applications in high frequency ultrasound technology. The optimal boundary feedback control is constructed in order to have on-line regulation. The above requires wellposedness of the associated Algebraic Riccati Equation. The paper by Lasiecka and Triggiani (2022) recently contributed a comprehensive study of the Optimal Control Problem for the MGT-third order dynamics with boundary control, over an infinite time-horizon. A critical missing point in such a study is the issue of uniqueness (within a specific class) of the corresponding highly non-standard Algebraic Riccati Equation. The present note resolves this problem in the positive, thus completing the study of Lasiecka and Triggiani (2022) with the final goal of having on line feedback control, which is also optimal.more » « less
-
Editor-in-Chief: George Yin (Ed.)This paper presents approaches to mean-field control, motivated by distributed control of multi-agent systems. Control solutions are based on a convex optimization problem, whose domain is a convex set of probability mass functions (pmfs). The main contributions follow: 1. Kullback-Leibler-Quadratic (KLQ) optimal control is a special case, in which the objective function is composed of a control cost in the form of Kullback-Leibler divergence between a candidate pmf and the nominal, plus a quadratic cost on the sequence of marginals. Theory in this paper extends prior work on deterministic control systems, establishing that the optimal solution is an exponential tilting of the nominal pmf. Transform techniques are introduced to reduce complexity of the KLQ solution, motivated by the need to consider time horizons that are much longer than the inter-sampling times required for reliable control. 2. Infinite-horizon KLQ leads to a state feedback control solution with attractive properties. It can be expressed as either state feedback, in which the state is the sequence of marginal pmfs, or an open loop solution is obtained that is more easily computed. 3. Numerical experiments are surveyed in an application of distributed control of residential loads to provide grid services, similar to utility-scale battery storage. The results show that KLQ optimal control enables the aggregate power consumption of a collection of flexible loads to track a time-varying reference signal, while simultaneously ensuring each individual load satisfies its own quality of service constraints.more » « less
-
Achieving optimal steady-state performance in real-time is an increasingly necessary requirement of many critical infrastructure systems. In pursuit of this goal, this paper builds a systematic design framework of feedback controllers for Linear Time-Invariant (LTI) systems that continuously track the optimal solution of some predefined optimization problem. We logically divide the proposed solution into three components. The first component estimates the system state from the output measurements. The second component uses the estimated state and computes a drift direction based on an optimization algorithm. The third component calculates an input to the LTI system that aims to drive the system toward the optimal steady-state. We analyze the equilibrium characteristics of the closed-loop system and provide conditions for optimality and stability. Our analysis shows that the proposed solution guarantees optimal steady-state performance, even in the presence of constant disturbances. Furthermore, by leveraging recent results on the analysis of optimization algorithms using Integral Quadratic Constraints (IQCs), the proposed framework can translate input-output properties of our optimization component into sufficient conditions, based on linear matrix inequalities (LMIs), for global exponential asymptotic stability of the closed-loop system. We illustrate several resulting controller designs using a numerical example.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1137/22M1492696
