More Like this

1. Adaptive Dynamic Programming for Decentralized Stabilization of Uncertain Nonlinear Large-Scale Systems With Mismatched Interconnections

   https://doi.org/10.1109/TSMC.2018.2837899  

   Yang, Xiong; He, Haibo (October 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems)

   This paper presents a novel decentralized control strategy for a class of uncertain nonlinear large-scale systems with mismatched interconnections. First, it is shown that the decentralized controller for the overall system can be represented by an array of optimal control policies of auxiliary subsystems. Then, within the framework of adaptive dynamic programming, a simultaneous policy iteration (SPI) algorithm is developed to solve the Hamilton–Jacobi–Bellman equations associated with auxiliary subsystem optimal control policies. The convergence of the SPI algorithm is guaranteed by an equivalence relationship. To implement the present SPI algorithm, actor and critic neural networks are applied to approximate the optimal control policies and the optimal value functions, respectively. Meanwhile, both the least squares method and the Monte Carlo integration technique are employed to derive the unknown weight parameters. Furthermore, by using Lyapunov’s direct method, the overall system with the obtained decentralized controller is proved to be asymptotically stable. Finally, the effectiveness of the proposed decentralized control scheme is illustrated via simulations for nonlinear plants and unstable power systems. 

   more » « less
2. Path-Dependent Hamilton–Jacobi Equations with u-Dependence and Time-Measurable Hamiltonians

   https://doi.org/10.1007/s00245-025-10230-0  

   Bandini, Elena; Keller, Christian (February 2025, Applied Mathematics & Optimization)

   We establish existence and uniqueness of minimax solutions for a fairly general class of path-dependent Hamilton-Jacobi equations. In particular, the relevant Hamiltonians can contain the solution and they only need to be measurable with respect to time. We apply our results to optimal control problems of (delay) functional differential equations with cost functionals that have discount factors and with time-measurable data. Our main results are also crucial for our companion paper Bandini and Keller [arXiv preprint arXiv:2408.02147 (2024)], where non-local path-dependent Hamilton-Jacobi-Bellman equations associated to the stochastic optimal control of non-Markovian piecewise deterministic processes are studied. 

   more » « less
3. Non-local Hamilton–Jacobi–Bellman equations for the stochastic optimal control of path-dependent piecewise deterministic processes

   https://doi.org/10.1016/j.spa.2025.104813  

   Bandini, Elena; Keller, Christian (October 2025, Stochastic Processes and their Applications)

   We study the optimal control of path-dependent piecewise deterministic processes. An appropriate dynamic programming principle is established. We prove that the associated value function is the unique minimax solution of the corresponding non-local path-dependent Hamilton-Jacobi-Bellman equation. This is the first well-posedness result for nonsmooth solutions of fully nonlinear non-local path-dependent partial differential equations. 

   more » « less
4. Event-Triggered Robust Stabilization of Nonlinear Input-Constrained Systems Using Single Network Adaptive Critic Designs

   https://doi.org/10.1109/TSMC.2018.2853089  

   Xiong Yang, Haibo He (January 2018, IEEE transactions on systems, man, and cybernetics. Systems)

   In this paper, we study the event-triggered robust stabilization problem of nonlinear systems subject to mismatched perturbations and input constraints. First, with the introduction of an infinite-horizon cost function for the auxiliary system, we transform the robust stabilization problem into a constrained optimal control problem. Then, we prove that the solution of the event-triggered Hamilton–Jacobi–Bellman (ETHJB) equation, which arises in the constrained optimal control problem, guarantees original system states to be uniformly ultimately bounded (UUB). To solve the ETHJB equation, we present a single network adaptive critic design (SN-ACD). The critic network used in the SN-ACD is tuned through the gradient descent method. By using Lyapunov method, we demonstrate that all the signals in the closed-loop auxiliary system are UUB. Finally, we provide two examples, including the pendulum system, to validate the proposed event-triggered control strategy. 

   more » « less
5. Continuity of the value function for deterministic optimal impulse control with terminal state constraint

   https://doi.org/10.1051/cocv/2021101  

   Zhou, Yue; Feng, Xinwei; Yong, Jiongmin (January 2021, ESAIM: Control, Optimisation and Calculus of Variations)

   Deterministic optimal impulse control problem with terminal state constraint is considered. Due to the appearance of the terminal state constraint, the value function might be discontinuous in general. The main contribution of this paper is the introduction of an intrinsic condition under which the value function is proved to be continuous. Then by a Bellman dynamic programming principle, the corresponding Hamilton-Jacobi-Bellman type quasi-variational inequality (QVI, for short) is derived. The value function is proved to be a viscosity solution to such a QVI. The issue of whether the value function is characterized as the unique viscosity solution to this QVI is carefully addressed and the answer is left open challengingly. 

   more » « less