skip to main content


Title: An Efficient Distributed Reinforcement Learning for Enhanced Multi-Microgrid Management
Economic dispatch in a multi-microgrid (MMG) system involves an increasing number of states from distributed energy resources (DERs) compared to a single microgrid. In these cases, traditional reinforcement learning (RL) approaches may become computationally expensive or less effective in finding the least-cost solution. This paper presents a novel RL approach that employs local learning agents to interact with individual microgrid environments in a distributed manner and a global agent to search for actions to minimize system cost at the MMG system level. The proposed distributed RL framework is more efficient in learning the dispatch policy compared to conventional approaches. Case studies are performed on a 3-microgrid system with different types of DERs. Results substantiate the effectiveness of the proposed approach in comparison with conventional methods in terms of operation costs, computation time, and peak-to-average ratio.  more » « less
Award ID(s):
2047064 1949921
NSF-PAR ID:
10391184
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
2022 International Joint Conference on Neural Networks (IJCNN)
Page Range / eLocation ID:
1 to 6
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    The mass deployment of distributed energy resources (DERs) to achieve clean energy objectives has become a major goal across several states in the U.S. However, the viability and reality of achieving these goals in dense urban areas, such as New York City, are challenged by several ‘Techno‐Economic’ barriers associated with available land space and the number of AC/direct current (DC) conversion stages that requires multiple electrical balance of plant (BOP) equipment for pairing/interconnecting these resources to the grid. The fundamental issue of interconnection is addressed by assessing the use of a common DC bus in a one‐of‐a‐kind configuration (to pair grid‐connected energy storage, photovoltaic, and electric vehicle chargers (EVC) systems) and reduce the number of BOP equipment needed for deployment. Building on similar work that has touched on distribution‐level DC interconnection, this paper will also address the intricacies of interconnecting third‐party and Utility DERs to a DC‐based point of common coupling. It will examine the requisite site controller configuration (control architecture) and requirements to coordinate the energy storage system's use between managing Utility and Third‐Party EVC demand while prioritising dispatch. The result shows that the DC‐coupled system is technologically feasible and hierarchical control architecture is recommended to maintain stability during various use cases proposed. This will inform a lab demonstration of this system that aims to test DC integration of the DERs with recommendations for the microgrid (MG) controllers and reduction in the BOP equipment. These learnings will then be applied to practical grid‐scale deployment of the systems at Con Edison's Cedar Street Substation. This system, if proven successful, has the potential to change the way community distributed generation and MGs are interconnected to the Utility System.

     
    more » « less
  2. This paper describes the development of a phasor-based campus microgrid model utilizing the Modelica language and the OpenIPSL library. The phasor-based modeling approach was chosen because the resulting microgrid model would yield faster simulation run times when compared to models developed using electromagnetic transient (EMT) methods. Beyond the benefits of simulation performance, this becomes necessary when attempting to understand dynamic phenomena arising under emergency conditions across time scales ranging from milliseconds to hours, which will aid in developing resiliency improvement plans for the real-world campus microgrid that the model represents. Considering the increasing number of distributed energy sources (DERs) being added to power grids across the world and the paradigm shift on how electrical grids can operate with more DERs, the implementation of such a microgrid campus model can help in the development and testing new control strategies to support new operational approaches while guaranteeing system stability and resiliency. The added benefit of having the microgrid model in Modelica is that it can be simulated in any Modelica complaint tool (both proprietary or not), preserving an open-source code, unlocked for the user to explore and adjust the implementation as well as observe and edit the mathematical formulation. This enables not only nonlinear time simulation, but also linear analysis techniques and other approaches to be applied. 
    more » « less
  3. In this paper, the cybersecurity of distributed secondary voltage control of AC microgrids is addressed. A resilient approach is proposed to mitigate the negative impacts of cyberthreats on the voltage and reactive power control of Distributed Energy Resources (DERs). The proposed secondary voltage control is inspired by the resilient flocking of a mobile robot team. This approach utilizes a virtual time-varying communication graph in which the quality of the communication links is virtualized and determined based on the synchronization behavior of DERs. The utilized control protocols on DERs ensure that the connectivity of the virtual communication graph is above a specific resilience threshold. Once the resilience threshold is satisfied the Weighted Mean Subsequence Reduced (WMSR) algorithm is applied to satisfy voltage restoration in the presence of malicious adversaries. A typical microgrid test system including 6 DERs is simulated to verify the validity of proposed resilient control approach. 
    more » « less
  4. In this paper, we propose the greedy smallest-cost-rate path first (GRASP) algorithm to route power from sources to loads in a digital microgrid (DMG). Routing of power from distributed energy resources (DERs) to loads of a DMG comprises matching loads to DERs and the selection of the smallest-cost-rate path from a load to its supplying DERs. In such a microgrid, one DER may supply power to one or many loads, and one or many DERs may supply the power requested by a load. Because the optimal method is NP-hard, GRASP addresses this high complexity by using heuristics to match sources and loads and to select the smallest-cost-rate paths in the DMG. We compare the cost achieved by GRASP and an optimal method based on integer linear programming on different IEEE test feeders and other test networks. The comparison shows the trade-offs between lowering complexity and achieving optimal-cost paths. The results show that the cost incurred by GRASP approaches that of the optimal solution by small margins. In the adopted networks, GRASP trades its lower complexity for up to 18% higher costs than those achieved by the optimal solution. 
    more » « less
  5. This paper presents a resilient control framework for distributed frequency and voltage control of AC microgrids under data manipulation attacks. In order for each distributed energy resource (DER) to detect any misbehavior on its neighboring DERs, an attack detection mechanism is first presented using a Kullback-Liebler (KL) divergence-based criterion. An attack mitigation technique is then proposed that utilizes the calculated KL divergence factors to determine trust values indicating the trustworthiness of the received information. Moreover, DERs continuously generate a self-belief factor and communicate it with their neighbors to inform them of the validity level of their own outgoing information. DERs incorporate their neighbors' self-belief and their own trust values in their control protocols to slow down and mitigate attacks. It is shown that the proposed cyber-secure control effectively distinguishes data manipulation attacks from legitimate events. The performance of proposed secure frequency and voltage control techniques is verified through the simulation of microgrid tests system implemented on IEEE 34-bus test feeder with six DERs. 
    more » « less