skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach With Safe Gradient Flow
Title: Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach With Safe Gradient Flow
Deep reinforcement learning approaches are becoming appealing for the design of nonlinear controllers for voltage control problems, but the lack of stability guarantees hinders their real-world deployment. This letter constructs a decentralized RL-based controller for inverter-based real-time voltage control in distribution systems. It features two components: a transient control policy and a steady-state performance optimizer. The transient policy is parameterized as a neural network, and the steady-state optimizer represents the gradient of the long-term operating cost function. The two parts are synthesized through a safe gradient flow framework, which prevents the violation of reactive power capacity constraints. We prove that if the output of the transient controller is bounded and monotonically decreasing with respect to its input, then the closed-loop system is asymptotically stable and converges to the optimal steady-state solution. We demonstrate the effectiveness of our method by conducting experiments with IEEE 13-bus and 123-bus distribution system test feeders.  more » « less
Award ID(s):
2200692
PAR ID:
10493715
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
IEEE
Date Published:
Journal Name:
IEEE Control Systems Letters
Volume:
7
ISSN:
2475-1456
Page Range / eLocation ID:
2845 to 2850
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. (, Proceedings of IEEE Smartgridcom Conference)
    null (Ed.)
    Pronounced variability due to the growth of renewable energy sources, flexible loads, and distributed generation is challenging residential distribution systems. This context, motivates well fast, efficient, and robust reactive power control. Optimal reactive power control is possible in theory by solving a non-convex optimization problem based on the exact model of distribution flow. However, lack of high-precision instrumentation and reliable communications, as well as the heavy computational burden of non-convex optimization solvers render computing and implementing the optimal control challenging in practice. Taking a statistical learning viewpoint, the input-output relationship between each grid state and the corresponding optimal reactive power control (a.k.a., policy) is parameterized in the present work by a deep neural network, whose unknown weights are updated by minimizing the accumulated power loss over a number of historical and simulated training pairs, using the policy gradient method. In the inference phase, one just feeds the real-time state vector into the learned neural network to obtain the ‘optimal’ reactive power control decision with only several matrix-vector multiplications. The merits of this novel deep policy gradient approach include its computational efficiency as well as robustness to random input perturbations. Numerical tests on a 47-bus distribution network using real solar and consumption data corroborate these practical merits. 
    more » « less
  2. ; ; (, Proc. 24st International Conference on System Theory, Control and Computing)
    null (Ed.)
    A new learning methodology in terms of a discretization of a so-called Chen-Fliess series of a control affine nonlinear system was recently proposed, in part, for the purpose of systematically including system structure and expert knowledge into control strategies. The main objective of this paper is to appropriately embed this learning unit as a supporting predictive controller for power dynamical systems. In particular, an infinite bus system is used for the prototype design of a smart and active control policy to regulate voltage and frequency. It is demonstrated by simulation how a controller employing a Chen-Fliess learning unit can recover from a fault and address modeling mismatch. 
    more » « less
  3. ; (, IEEE Transactions on Power Systems)
    In this work, we investigate grid-forming control for power systems containing three-phase and single-phase converters connected to unbalanced distribution and transmission networks, investigate self-balancing between single-phase converters, and propose a novel balancing feedback for grid-forming control that explicitly allows to trade-off unbalances in voltage and power. We develop a quasi-steady-state power network model that allows to analyze the interactions between three-phase and single-phase power converters across transmission, distribution, and standard transformer interconnections. We first investigate conditions under which this general network admits a well-posed kron-reduced quasi-steady-state network model. Our main contribution leverages this reduced-order model to develop analytical conditions for stability of the overall network with grid-forming three-phase and single-phase converters connected through standard transformer interconnections. Specifically, we provide conditions on the network topology under which (i) single-phase converters autonomously self-synchronize to a phase-balanced operating point and (ii) single-phase converters phase-balance through synchronization with three-phase converters. Moreover, we establish that the conditions can be relaxed if a phase-balancing feedback control is used. Finally, case studies combining detailed models of transmission systems (i.e., IEEE 9-bus) and distribution systems (i.e., IEEE 13-bus) are used to illustrate the results for (i) a power system containing a mix of transmission and distribution connected converters and, (ii) a power system solely using distribution-connected converters at the grid edge. 
    more » « less
  4. ; ; (, Conference on Decision and Control)
    We consider the problem of designing a feedback controller that guides the input and output of a linear time-invariant system to a minimizer of a convex optimization problem. The system is subject to an unknown disturbance, piecewise constant in time, which shifts the feasible set defined by the system equilibrium constraints. Our proposed design combines proportional-integral control with gradient feedback, and enforces the Karush-Kuhn-Tucker optimality conditions in steady-state without incorporating dual variables into the controller. We prove that the input and output variables achieve optimality in steady-state, and provide a stability criterion based on absolute stability theory. The effectiveness of our approach is illustrated on a simple example system. 
    more » « less
  5. ; ; ; ; ; (, IEEE Transactions on Automatic Control)
    Frequency restoration in power systems is conventionally performed by broadcasting a centralized signal to local controllers. As a result of the energy transition, technological advances, and the scientific interest in distributed control and optimization methods, a plethora of distributed frequency control strategies have been proposed recently that rely on communication amongst local controllers. In this paper, we propose a fully decentralized leaky integral controller for frequency restoration that is derived from a classic lag element. We study steady-state, asymptotic optimality, nominal stability, input-to-state stability, noise rejection, transient performance, and robustness properties of this controller in closed loop with a nonlinear and multivariable power system model. We demonstrate that the leaky integral controller can strike an acceptable trade-off between performance and robustness as well as between asymptotic disturbance rejection and transient convergence rate by tuning its DC gain and time constant. We compare our findings to conventional decentralized integral control and distributed- averaging-based integral control in theory and simulations. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Text Encoded Conversion
  • XML
  • Save / Share this Record
  • Send to Email