Search for: All records

This paper describes how domain knowledge of power system operators can be integrated into reinforcement learning (RL) frameworks to effectively learn agents that control the grid's topology to prevent thermal cascading. Typical RL-based topology controllers fail to perform well due to the large search/optimization space. Here, we propose an actor-critic-based agent to address the problem's combinatorial nature and train the agent using the RL environment developed by RTE, the French TSO. To address the challenge of the large optimization space, a curriculum-based approach with reward tuning is incorporated into the training procedure by modifying the environment using network physics for enhanced agent learning. Further, a parallel training approach on multiple scenarios is employed to avoid biasing the agent to a few scenarios and make it robust to the natural variability in grid operations. Without these modifications to the training procedure, the RL agent failed for most test scenarios, illustrating the importance of properly integrating domain knowledge of physical systems for real-world RL learning. The agent was tested by RTE for the 2019 learning to run the power network challenge and was awarded the 2nd place in accuracy and 1st place in speed. The developed code is open-sourced for public use. Analysis of a simple system proves the enhancement in training RL-agents using the curriculum. 

The Phasor measurement unit (PMU) measurements are mandatory to monitor the power system’s voltage stability margin in an online manner. Monitoring is key to the secure operation of the grid. Traditionally, online monitoring of voltage stability using synchrophasors required a centralized communication architecture, which leads to the high investment cost and cyber-security concerns. The increasing importance of cyber-security and low investment costs have recently led to the development of distributed algorithms for online monitoring of the grid that are inherently less prone to malicious attacks. In this work, we proposed a novel distributed non-iterative voltage stability index (VSI) by recasting the power flow equations as circles. The processors embedded at each bus in the smart grid with the help of PMUs and communication of voltage phasors between neighboring buses perform simultaneous online computations of VSI. The distributed nature of the index enables the real-time identification of the critical bus of the system with minimal communication infrastructure. The effectiveness of the proposed distributed index is demonstrated on IEEE test systems and contrasted with existing methods to show the benefits of the proposed method in speed, interpretability, identification of outage location, and low sensitivity to noisy measurements.