More Like this

1. Robust decentralized frequency control: A leaky integrator approach

   https://doi.org/10.23919/ECC.2018.8550060  

   Weitenberg, Erik; Jiang, Yan; Zhao, Changhong; Mallada, Enrique; Dorfler, Florian; De Persis, Claudio (June 2018, European Control Conference)

   We investigate the robustness of the so-called leaky integral frequency controller for the power network. In particular, using a strict Lyapunov function, we show the closed-loop system is robust in the input-to-state stability sense to measurement noise in the controller. Moreover, an interesting and explicit trade-o between controller performance and robustness is discussed and illustrated using a bench- mark study of the 39-bus New England reference network. 

   more » « less
2. Robust Decentralized Secondary Frequency Control in Power Systems: Merits and Trade-Offs

   https://doi.org/10.1109/TAC.2018.2884650  

   Weitenberg, Erik; Jiang, Yan; Zhao, Changhong; Mallada, Enrique; De Persis, Claudio; Dorfler, Florian (December 2018, IEEE Transactions on Automatic Control)

   Frequency restoration in power systems is conventionally performed by broadcasting a centralized signal to local controllers. As a result of the energy transition, technological advances, and the scientific interest in distributed control and optimization methods, a plethora of distributed frequency control strategies have been proposed recently that rely on communication amongst local controllers. In this paper, we propose a fully decentralized leaky integral controller for frequency restoration that is derived from a classic lag element. We study steady-state, asymptotic optimality, nominal stability, input-to-state stability, noise rejection, transient performance, and robustness properties of this controller in closed loop with a nonlinear and multivariable power system model. We demonstrate that the leaky integral controller can strike an acceptable trade-off between performance and robustness as well as between asymptotic disturbance rejection and transient convergence rate by tuning its DC gain and time constant. We compare our findings to conventional decentralized integral control and distributed- averaging-based integral control in theory and simulations. 

   more » « less
3. Scalable Robust Adaptive Control from the System Level Perspective

   https://doi.org/10.23919/ACC.2019.8814896  

   Ho, Dimitar; Doyle, John C. (July 2019, 2019 American Control Conference (ACC))

   We will present a new general framework for robust and adaptive control that allows for distributed and scalable learning and control of large systems of interconnected linear subsystems. The control method is demonstrated for a linear time-invariant system with bounded parameter uncertainties, disturbances and noise. The presented scheme continuously collects measurements to reduce the uncertainty about the system parameters and adapts dynamic robust controllers online in a stable and performance-improving way. A key enabler for our approach is choosing a time-varying dynamic controller implementation, inspired by recent work on System Level Synthesis [1]. We leverage a new robustness result for this implementation to propose a general robust adaptive control algorithm. In particular, the algorithm allows us to impose communication and delay constraints on the controller implementation and is formulated as a sequence of robust optimization problems that can be solved in a distributed manner. The proposed control methodology performs particularly well when the interconnection between systems is sparse and the dynamics of local regions of subsystems depend only on a small number of parameters. As we will show on a five-dimensional exemplary chain-system, the algorithm can utilize system structure to efficiently learn and control the entire system while respecting communication and implementation constraints. Moreover, although current theoretical results require the assumption of small initial uncertainties to guarantee robustness, we will present simulations that show good closed-loop performance even in the case of large uncertainties, which suggests that this assumption is not critical for the presented technique and future work will focus on providing less conservative guarantees. 

   more » « less
4. A Reinforcement Learning Look at Risk-Sensitive Linear Quadratic Gaussian Control

   Cui, Leilei; Basar, Tamer Basar; Jiang, Zhong-Ping (August 2023, Proceedings of Machine Learning Research)

   N. Matni, M. Morari (Ed.)

   In this paper, we propose a robust reinforcement learning method for a class of linear discrete-time systems to handle model mismatches that may be induced by sim-to-real gap. Under the formulation of risk-sensitive linear quadratic Gaussian control, a dual-loop policy optimization algorithm is proposed to iteratively approximate the robust and optimal controller. The convergence and robustness of the dual-loop policy optimization algorithm are rigorously analyzed. It is shown that the dual-loop policy optimization algorithm uniformly converges to the optimal solution. In addition, by invoking the concept of small-disturbance input-to-state stability, it is guaranteed that the dual-loop policy optimization algorithm still converges to a neighborhood of the optimal solution when the algorithm is subject to a sufficiently small disturbance at each step. When the system matrices are unknown, a learning-based off-policy policy optimization algorithm is proposed for the same class of linear systems with additive Gaussian noise. The numerical simulation is implemented to demonstrate the efficacy of the proposed algorithm. 

   more » « less
5. A Reinforcement Learning Look at Risk-Sensitive Linear Quadratic Gaussian Control

   Cui, Leilei; Basar, Tamer; Jiang, Zhong-Ping (June 2023, Proceedings of Machine Learning Research)

   Matni, N; Morari, M; Pappas, G J (Ed.)

   In this paper, we propose a robust reinforcement learning method for a class of linear discrete-time systems to handle model mismatches that may be induced by sim-to-real gap. Under the formulation of risk-sensitive linear quadratic Gaussian control, a dual-loop policy optimization algorithm is proposed to iteratively approximate the robust and optimal controller. The convergence and robustness of the dual-loop policy optimization algorithm are rigorously analyzed. It is shown that the dual-loop policy optimization algorithm uniformly converges to the optimal solution. In addition, by invoking the concept of small-disturbance input-to-state stability, it is guaranteed that the dual-loop policy optimization algorithm still converges to a neighborhood of the optimal solution when the algorithm is subject to a sufficiently small disturbance at each step. When the system matrices are unknown, a learning-based off-policy policy optimization algorithm is proposed for the same class of linear systems with additive Gaussian noise. The numerical simulation is implemented to demonstrate the efficacy of the proposed algorithm. 

   more » « less