skip to main content


Title: Improved primary frequency response through deep reinforcement learning
This paper explores the application of deep reinforcement learning (DRL) to create a coordinating mechanism between synchronous generators (SGs) and distributed energy resources (DERs) for improved primary frequency regulation. Renewable energy sources, such as wind and solar, may be used to aid in frequency regulation of the grid. Without proper coordination between the sources, however, the participation only results in a delay of SG governor response and frequency deviation. The proposed DRL application uses a deep deterministic policy gradient (DDPG) agent to create a generalized coordinating signal for DERs. The coordinating signal communicates the degree of distributed participation to the SG governor, resolving delayed governor response and reducing system rate of change of frequency (ROCOF). The validity of the coordinating signal is presented with a single-machine finite bus system. The use of DRL for signal creation is explored in an under-frequency event. While further exploration is needed for validation in large systems, the development of this concept shows promising results towards increased power grid stabilization.  more » « less
Award ID(s):
2033910
NSF-PAR ID:
10351473
Author(s) / Creator(s):
;
Editor(s):
Mostafa Sahraei-Ardakani; Mingxi Liu
Date Published:
Journal Name:
North American Power Symposium
ISSN:
2163-4939
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. With the advent of remarkable development of solar power panel and inverter technology and focus on reducing greenhouse emissions, there is increased migration from fossil fuels to carbon-free energy sources (e.g., solar, wind, and geothermal). A new paradigm called Transactive Energy (TE) [3] has emerged that utilizes economic and control techniques to effectively manage Distributed Energy Resources (DERs). Another goal of TE is to improve grid reliability and efficiency. However, to evaluate various TE approaches, a comprehensive simulation tool is needed that is easy to use and capable of simulating the power-grid along with various grid operational scenarios that occur in the transactive energy paradigm. In this research, we present a web-based design and simulation platform (called a design studio) targeted toward evaluation of power-grid distribution system and transactive energy approaches [1]. The design studio allows to edit and visualize existing power-grid models graphically, create new power-grid network models, simulate those networks, and inject various scenario-specific perturbations to evaluate specific configurations of transactive energy simulations. The design studio provides (i) a novel Domain-Specific Modeling Language (DSML) using the Web-based Generic Modeling Environment (WebGME [4]) for the graphical modeling of power-grid, cyber-physical attacks, and TE scenarios, and (ii) a reusable cloud-hosted simulation backend using the Gridlab-D power-grid distribution system simulation tool [2]. 
    more » « less
  2. With the proliferation of distributed energy resources (DERs) in the distribution grid, it is a challenge to effectively control a large number of DERs resilient to the communication and security disruptions, as well as to provide the online grid services, such as voltage regulation and virtual power plant (VPP) dispatch. To this end, a hybrid feedback-based optimization algorithm along with deep learning forecasting technique is proposed to specifically address the cyber-related issues. The online decentralized feedback-based DER optimization control requires timely, accurate voltage measurement from the grid. However, in practice such information may not be received by the control center or even be corrupted. Therefore, the long short-term memory (LSTM) deep learning algorithm is employed to forecast delayed/missed/attacked messages with high accuracy. The IEEE 37-node feeder with high penetration of PV systems is used to validate the efficiency of the proposed hybrid algorithm. The results show that 1) the LSTM-forecasted lost voltage can effectively improve the performance of the DER control algorithm in the practical cyber-physical architecture; and 2) the LSTM forecasting strategy outperforms other strategies of using previous message and skipping dual parameter update. 
    more » « less
  3. null (Ed.)
    This paper presents one of the first real-life demonstrations of coordinated and distributed resource control for secondary frequency response in a power distribution grid. A series of tests involved up to 69 heterogeneous active distributed energy resources consisting of air handling units, unidirectional and bidirectional electric vehicle charging stations, a battery energy storage system, and 107 passive distributed energy resources consisting of building loads and solar photovoltaic systems. The distributed control setup consists of a set of Raspberry Pi end-points exchanging messages via an ethernet switch. Actuation commands for the distributed energy resources are obtained by solving a power allocation problem at every regulation instant using distributed ratio-consensus, primal-dual, and Newton-like algorithms. The problem formulation minimizes the sum of distributed energy resource costs while tracking the aggregate setpoint provided by the system operator. We demonstrate accurate and fast real-time distributed computation of the optimization solution and effective tracking of the regulation signal over 40 min time horizons. An economic benefit analysis confirms eligibility to participate in an ancillary services market and demonstrates up to $53k of potential annual revenue for the selected population of distributed energy resources. 
    more » « less
  4. The widespread use of distributed energy sources (DERs) raises significant challenges for power system design, planning, and operation, leading to wide adaptation of tools on hosting capacity analysis (HCA). Traditional HCA methods conduct extensive power flow analysis. Due to the computation burden, these time-consuming methods fail to provide online hosting capacity (HC) in large distribution systems. To solve the problem, we first propose a deep learning-based problem formulation for HCA, which conducts offline training and determines HC in real time. The used learning model, long short-term memory (LSTM), implements historical time-series data to capture periodical patterns in distribution systems. However, directly applying LSTMs suffers from low accuracy due to the lack of consideration on spatial information, where location information like feeder topology is critical in nodal HCA. Therefore, we modify the forget gate function to dual forget gates, to capture the spatial correlation within the grid. Such a design turns the LSTM into the Spatial-Temporal LSTM (ST-LSTM). Moreover, as voltage violations are the most vital constraints in HCA, we design a voltage sensitivity gate to increase accuracy further. The results of LSTMs and ST-LSTMs on feeders, such as IEEE 34-, 123-bus feeders, and utility feeders, validate our designs. 
    more » « less
  5. Distributed generation is gaining greater penetration levels in distribution grids due to government incentives for integrating distributed energy resources (DERs) and DER cost reductions. The frequency response of a grid-connected single inverter changes as other inverters are connected in parallel due to the couplings among grid inductance and/or inverter output filters. The selection of the inverter- or grid-side currents as feedback control signals is then not trivial because each one has tradeoffs. This paper analyses the system stability for multiple parallel- and grid-connected inverters using the inverter- or gridside currents as feedback signals. Modeling of both feedback signals is performed using the current separation technique. Moreover, the stability range for different conditions including active damping is analyzed through the root locus technique. The grid-side current has a wider range of stability, but the inverterside current allows for higher values of the proportional gain near the critical frequency and no extra sensors are needed since measurement of the inverter current is needed for protection in high-power applications. 
    more » « less