An official website of the United States government
The increasing penetration of Distributed Energy Resources (DERs) in modern distribution grids has created sig-nificant challenges in maintaining voltage profiles and minimizing reactive power losses. While Deep Reinforcement Learning (DRL) approaches have been explored for Volt/VAR Control (VVC), existing methods often rely on single-agent frameworks or lack adaptive mechanisms to balance competing grid objectives dy-namically. Multi-Agent (MA) DRL frameworks have emerged but typically require explicit inter-agent communication, limiting scalability and practicality. To address these limitations, we propose a novel MA-DRL framework that incorporates adaptive reward computation to implicitly coordinate special-ized agents, including a Voltage Control Agent for maintaining voltage profiles and a Reactive Power Agent for optimizing reactive power dispatch. Unlike prior approaches, our method dynamically adjusts local and global rewards based on real-time grid conditions, enabling scalable and efficient grid control without explicit inter-agent communication. We evaluated the framework on the IEEE 123-bus test system under high DER penetration. The results demonstrated that the proposed MA-DRL approach successfully eliminated voltage violations, reduced power losses through optimized reactive power dispatch, and decreased transformer tap changes and capacitor operations, thereby extending equipment lifespan. These results demonstrate the scalability and practicality of adaptive reward-based MA-DRL for addressing modern grid management challenges.
more »
« less
Efficient Operations of Micro-Grids with Meshed Topology and Under Uncertainty through Exact Satisfaction of AC-PF, Droop Control and Tap-Changer Constraints
Micro-grids’ operations offer local reliability; in the event of faults or low voltage/frequency events on the utility side, micro-grids can disconnect from the main grid and operate autonomously while providing a continued supply of power to local customers. With the ever-increasing penetration of renewable generation, however, operations of micro-grids become increasingly complicated because of the associated fluctuations of voltages. As a result, transformer taps are adjusted frequently, thereby leading to fast degradation of expensive tap-changer transformers. In the islanding mode, the difficulties also come from the drop in voltage and frequency upon disconnecting from the main grid. To appropriately model the above, non-linear AC power flow constraints are necessary. Computationally, the discrete nature of tap-changer operations and the stochasticity caused by renewables add two layers of difficulty on top of a complicated AC-OPF problem. To resolve the above computational difficulties, the main principles of the recently developed “l1-proximal” Surrogate Lagrangian Relaxation are extended. Testing results based on the nine-bus system demonstrate the efficiency of the method to obtain the exact feasible solutions for micro-grid operations, thereby avoiding approximations inherent to existing methods; in particular, fast convergence of the method to feasible solutions is demonstrated. It is also demonstrated that through the optimization, the number of tap changes is drastically reduced, and the method is capable of efficiently handling networks with meshed topologies.
more »
« less
- PAR ID:
- 10342722
- Date Published:
- Journal Name:
- Energies
- Volume:
- 15
- Issue:
- 10
- ISSN:
- 1996-1073
- Page Range / eLocation ID:
- 3662
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The growing integration of Distributed Energy Resources (DERs) into modern power grids, managed via DER Management Systems (DERMS), has introduced significant cybersecurity challenges. Communication vulnerabilities in DERMS architecture create opportunities for adversarial attacks, such as Man-in-the-Middle (MITM), that intercept and manipulate critical data flows, disrupting grid operations and compromising reliability. This paper investigates MITM attacks targeting three critical nodes in DER-DERMS communication: (i) DER gateway routers, (ii) inter-router pathways, and (iii) DERMS gateway routers. To analyze these vulnerabilities, we developed CPS-DERMS, a multi-co-simulation hardware-in-the-loop testbed integrating grid-side simulations with OPAL-RT, large-scale cyber simulations using NS-3, and an indige-nous Python-based DERMS platform. The study demonstrates innovative strategies for executing MITM attacks, including maintaining payload integrity and bypassing detection mechanisms, while highlighting the critical vulnerabilities within DER-DERMS communication systems. We considered a real utility's DER-integrated distribution grid as a case study. Experimental results reveal that MITM attacks can cause overvoltages, reactive power imbalances, and increased tap-changer operations, significantly degrading power quality and grid reliability.more » « less
-
This paper presents an algorithm for restoring AC power flow feasibility from solutions to simplified optimal power flow (OPF) problems, including convex relaxations, power flow approximations, and machine learning (ML) models. The proposed algorithm employs a state estimation-based post-processing technique in which voltage phasors, power injections, and line flows from solutions to relaxed, approximated, or ML-based OPF problems are treated similarly to noisy measurements in a state estimation algorithm. The algorithm leverages information from various quantities to obtain feasible voltage phasors and power injections that satisfy the AC power flow equations. Weight and bias parameters are computed offline using an adaptive stochastic gradient descent method. By automatically learning the trustworthiness of various outputs from simplified OPF problems, these parameters inform the online computations of the state estimation-based algorithm to both recover feasible solutions and characterize the performance of power flow approximations, relaxations, and ML models. Furthermore, the proposed algorithm can simultaneously utilize combined solutions from different relaxations, approximations, and ML models to enhance performance. Case studies demonstrate the effectiveness and scalability of the proposed algorithm, with solutions that are both AC power flow feasible and much closer to the true AC OPF solutions than alternative methods, often by several orders of magnitude in the squared two-norm loss function.more » « less
-
This paper makes use of electric vehicles (EVs) that are simultaneously connected to the Photovoltaic Cells (PV) and the power grid. In micro-grids, batteries of the electric vehicles (EVs) used as a source of power to feed the power grid in the peak demands of electricity. EVs can help regulation of the power grid by storing excess solar energy and returning it to the grid during high demand hours. This paper proposes a new architecture of micro-grids by using a rooftop solar system, Battery Electric Vehicles (BEVs), grid connected inverters, a boost converter, a bidirectional half-bridge converter, output filter, including L, LC, or LCL, and transformers. The main parts of this micro-grid are illustrated and modeled, as well as a simulation of their operation. In addition, simulation results explore the charging and discharging scenarios of the BEVs.more » « less
-
Microgrids must be able to restore voltage and frequency to their reference values during transient events; inverters are used as part of a microgrid's hierarchical control for maintaining power quality. Reviewed methods either do not allow for intuitive trade-off tuning between the objectives of synchronous state restoration, local reference tracking, and disturbance rejection, or do not consider all of these objectives. In this paper, we address all of these objectives for voltage restoration in droop-controlled inverter-based islanded micro-grids. By using distributed model predictive control (DMPC) in series with an unscented Kalman Filter (UKF), we design a secondary voltage controller to restore the voltage to the reference in finite time. The DMPC solves a reference tracking problem while rejecting reactive power disturbances in a noisy system. The method we present accounts for non-zero mean disturbances by design of a random-walk estimator. We validate the method's ability to restore the voltage in finite time via modeling a multi-node microgrid in Simulink.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.3390/en15103662
