skip to main content

Attention:

The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, October 10 until 2:00 AM ET on Friday, October 11 due to maintenance. We apologize for the inconvenience.


Title: Improved primary frequency response through deep reinforcement learning
This paper explores the application of deep reinforcement learning (DRL) to create a coordinating mechanism between synchronous generators (SGs) and distributed energy resources (DERs) for improved primary frequency regulation. Renewable energy sources, such as wind and solar, may be used to aid in frequency regulation of the grid. Without proper coordination between the sources, however, the participation only results in a delay of SG governor response and frequency deviation. The proposed DRL application uses a deep deterministic policy gradient (DDPG) agent to create a generalized coordinating signal for DERs. The coordinating signal communicates the degree of distributed participation to the SG governor, resolving delayed governor response and reducing system rate of change of frequency (ROCOF). The validity of the coordinating signal is presented with a single-machine finite bus system. The use of DRL for signal creation is explored in an under-frequency event. While further exploration is needed for validation in large systems, the development of this concept shows promising results towards increased power grid stabilization.  more » « less
Award ID(s):
2033910
NSF-PAR ID:
10351473
Author(s) / Creator(s):
;
Editor(s):
Mostafa Sahraei-Ardakani; Mingxi Liu
Date Published:
Journal Name:
North American Power Symposium
ISSN:
2163-4939
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The growing adoption of residential distributed energy resources (DERs) introduces more uncertain variability in power grid operation. More importantly, the residential DERs operate behind customers’ energy meters, and therefore, the utility cannot “directly” monitor them. Prior approaches to enable visibility into behind-the-meter (BTM) DERs either depend on estimations or require intrusive instrumentation on the customer side. To address the critical need for direct real-time monitoring of BTM DERs, in this paper, we propose a novel approach for utility-side direct real-time monitoring of residential BTM DERs. We utilize high-frequency (> 10kHz) conducted electromagnetic interference (EMI) from residential DERs’ grid-tied inverters to monitor their power generation. We discuss the working principle of our approach and present supporting results using three of-the-shelf grid-tied inverters. 
    more » « less
  2. null (Ed.)
    This paper presents one of the first real-life demonstrations of coordinated and distributed resource control for secondary frequency response in a power distribution grid. A series of tests involved up to 69 heterogeneous active distributed energy resources consisting of air handling units, unidirectional and bidirectional electric vehicle charging stations, a battery energy storage system, and 107 passive distributed energy resources consisting of building loads and solar photovoltaic systems. The distributed control setup consists of a set of Raspberry Pi end-points exchanging messages via an ethernet switch. Actuation commands for the distributed energy resources are obtained by solving a power allocation problem at every regulation instant using distributed ratio-consensus, primal-dual, and Newton-like algorithms. The problem formulation minimizes the sum of distributed energy resource costs while tracking the aggregate setpoint provided by the system operator. We demonstrate accurate and fast real-time distributed computation of the optimization solution and effective tracking of the regulation signal over 40 min time horizons. An economic benefit analysis confirms eligibility to participate in an ancillary services market and demonstrates up to $53k of potential annual revenue for the selected population of distributed energy resources. 
    more » « less
  3. Distributed generation is gaining greater penetration levels in distribution grids due to government incentives for integrating distributed energy resources (DERs) and DER cost reductions. The frequency response of a grid-connected single inverter changes as other inverters are connected in parallel due to the couplings among grid inductance and/or inverter output filters. The selection of the inverter- or grid-side currents as feedback control signals is then not trivial because each one has tradeoffs. This paper analyses the system stability for multiple parallel- and grid-connected inverters using the inverter- or gridside currents as feedback signals. Modeling of both feedback signals is performed using the current separation technique. Moreover, the stability range for different conditions including active damping is analyzed through the root locus technique. The grid-side current has a wider range of stability, but the inverterside current allows for higher values of the proportional gain near the critical frequency and no extra sensors are needed since measurement of the inverter current is needed for protection in high-power applications. 
    more » « less
  4. Dispatching a large fleet of distributed energy resources (DERs) in response to wholesale energy market or regional grid signals requires solving a challenging disaggregation problem when the DERs are located within a distribution network. This manuscript presents a computationally tractable convex inner approximation for the optimal power flow (OPF) problem that characterizes a feeders aggregate DERs hosting capacity and enables a realtime, grid-aware dispatch of DERs for radial distribution networks. The inner approximation is derived by considering convex envelopes on the nonlinear terms in the AC power flow equations. The resulting convex formulation is then used to derive provable nodal injection limits, such that any combination of DER dispatches within their respective nodal limits is guaranteed to be AC admissible. These nodal injection limits are then used to construct a realtime, open-loop control policy for dispatching DERs at each location in the network to collectively deliver grid services. The IEEE-37 distribution network is used to validate the technical results and highlight various use-cases. 
    more » « less
  5. With the advent of remarkable development of solar power panel and inverter technology and focus on reducing greenhouse emissions, there is increased migration from fossil fuels to carbon-free energy sources (e.g., solar, wind, and geothermal). A new paradigm called Transactive Energy (TE) [3] has emerged that utilizes economic and control techniques to effectively manage Distributed Energy Resources (DERs). Another goal of TE is to improve grid reliability and efficiency. However, to evaluate various TE approaches, a comprehensive simulation tool is needed that is easy to use and capable of simulating the power-grid along with various grid operational scenarios that occur in the transactive energy paradigm. In this research, we present a web-based design and simulation platform (called a design studio) targeted toward evaluation of power-grid distribution system and transactive energy approaches [1]. The design studio allows to edit and visualize existing power-grid models graphically, create new power-grid network models, simulate those networks, and inject various scenario-specific perturbations to evaluate specific configurations of transactive energy simulations. The design studio provides (i) a novel Domain-Specific Modeling Language (DSML) using the Web-based Generic Modeling Environment (WebGME [4]) for the graphical modeling of power-grid, cyber-physical attacks, and TE scenarios, and (ii) a reusable cloud-hosted simulation backend using the Gridlab-D power-grid distribution system simulation tool [2]. 
    more » « less