skip to main content

This content will become publicly available on October 1, 2023

Title: Improved primary frequency response through deep reinforcement learning
This paper explores the application of deep reinforcement learning (DRL) to create a coordinating mechanism between synchronous generators (SGs) and distributed energy resources (DERs) for improved primary frequency regulation. Renewable energy sources, such as wind and solar, may be used to aid in frequency regulation of the grid. Without proper coordination between the sources, however, the participation only results in a delay of SG governor response and frequency deviation. The proposed DRL application uses a deep deterministic policy gradient (DDPG) agent to create a generalized coordinating signal for DERs. The coordinating signal communicates the degree of distributed participation to the SG governor, resolving delayed governor response and reducing system rate of change of frequency (ROCOF). The validity of the coordinating signal is presented with a single-machine finite bus system. The use of DRL for signal creation is explored in an under-frequency event. While further exploration is needed for validation in large systems, the development of this concept shows promising results towards increased power grid stabilization.
Mostafa Sahraei-Ardakani; Mingxi Liu
Award ID(s):
Publication Date:
Journal Name:
North American Power Symposium
Sponsoring Org:
National Science Foundation
More Like this
  1. With the advent of remarkable development of solar power panel and inverter technology and focus on reducing greenhouse emissions, there is increased migration from fossil fuels to carbon-free energy sources (e.g., solar, wind, and geothermal). A new paradigm called Transactive Energy (TE) [3] has emerged that utilizes economic and control techniques to effectively manage Distributed Energy Resources (DERs). Another goal of TE is to improve grid reliability and efficiency. However, to evaluate various TE approaches, a comprehensive simulation tool is needed that is easy to use and capable of simulating the power-grid along with various grid operational scenarios that occur in the transactive energy paradigm. In this research, we present a web-based design and simulation platform (called a design studio) targeted toward evaluation of power-grid distribution system and transactive energy approaches [1]. The design studio allows to edit and visualize existing power-grid models graphically, create new power-grid network models, simulate those networks, and inject various scenario-specific perturbations to evaluate specific configurations of transactive energy simulations. The design studio provides (i) a novel Domain-Specific Modeling Language (DSML) using the Web-based Generic Modeling Environment (WebGME [4]) for the graphical modeling of power-grid, cyber-physical attacks, and TE scenarios, and (ii) a reusable cloud-hostedmore »simulation backend using the Gridlab-D power-grid distribution system simulation tool [2].« less
  2. This paper presents one of the first real-life demonstrations of coordinated and distributed resource control for secondary frequency response in a power distribution grid. A series of tests involved up to 69 heterogeneous active distributed energy resources consisting of air handling units, unidirectional and bidirectional electric vehicle charging stations, a battery energy storage system, and 107 passive distributed energy resources consisting of building loads and solar photovoltaic systems. The distributed control setup consists of a set of Raspberry Pi end-points exchanging messages via an ethernet switch. Actuation commands for the distributed energy resources are obtained by solving a power allocation problem at every regulation instant using distributed ratio-consensus, primal-dual, and Newton-like algorithms. The problem formulation minimizes the sum of distributed energy resource costs while tracking the aggregate setpoint provided by the system operator. We demonstrate accurate and fast real-time distributed computation of the optimization solution and effective tracking of the regulation signal over 40 min time horizons. An economic benefit analysis confirms eligibility to participate in an ancillary services market and demonstrates up to $53k of potential annual revenue for the selected population of distributed energy resources.
  3. Distributed generation is gaining greater penetration levels in distribution grids due to government incentives for integrating distributed energy resources (DERs) and DER cost reductions. The frequency response of a grid-connected single inverter changes as other inverters are connected in parallel due to the couplings among grid inductance and/or inverter output filters. The selection of the inverter- or grid-side currents as feedback control signals is then not trivial because each one has tradeoffs. This paper analyses the system stability for multiple parallel- and grid-connected inverters using the inverter- or gridside currents as feedback signals. Modeling of both feedback signals is performed using the current separation technique. Moreover, the stability range for different conditions including active damping is analyzed through the root locus technique. The grid-side current has a wider range of stability, but the inverterside current allows for higher values of the proportional gain near the critical frequency and no extra sensors are needed since measurement of the inverter current is needed for protection in high-power applications.
  4. The interconnection of distributed energy resources (DERs) in microgrids (MGs) operating in both islanded and grid-connected modes require coordinated control strategies. DERs are interfaced with voltage source inverters (VSIs) enabling interconnection. This paper proposes a load demand sharing scheme for the parallel operation of VSIs in an islanded voltage source inverter-based microgrid (VSI-MG). The ride-through capability of a heavily loaded VSI-MG, where some of the VSIs are fully loaded due to the occurrence of an event is investigated. In developing analytical equations to model the VSI, the concept of virtual synchronous machines (VSM) is applied to enable the VSI mimic the inertia effect of synchronous machines. A power frame transformation (PFT) that takes the line ratios of the MG network into account is also incorporated to yield satisfactory transient responses of both network frequency and bus voltages in the MG network. A Jacobian-based method is then developed to take into account the operational capacity of each VSI in the VSI-MG. The resulting amendable droop control constrains the VSIs within their power capabilities when an event occurs. Simulation results presented within demonstrate the effectiveness of the proposed procedure which has great potential to facilitate efforts in maintaining system reliability and resiliency.
  5. Dispatching a large fleet of distributed energy resources (DERs) in response to wholesale energy market or regional grid signals requires solving a challenging disaggregation problem when the DERs are located within a distribution network. This manuscript presents a computationally tractable convex inner approximation for the optimal power flow (OPF) problem that characterizes a feeders aggregate DERs hosting capacity and enables a realtime, grid-aware dispatch of DERs for radial distribution networks. The inner approximation is derived by considering convex envelopes on the nonlinear terms in the AC power flow equations. The resulting convex formulation is then used to derive provable nodal injection limits, such that any combination of DER dispatches within their respective nodal limits is guaranteed to be AC admissible. These nodal injection limits are then used to construct a realtime, open-loop control policy for dispatching DERs at each location in the network to collectively deliver grid services. The IEEE-37 distribution network is used to validate the technical results and highlight various use-cases.