More Like this

1. Decentralized control of distributed energy resources in radial distribution systems

   https://doi.org/10.1109/SmartGridComm.2016.7778777  

   Lin, Weixuan; Bitar, Eilyan (November 2016, 2016 IEEE International Conference on Smart Grid Communications (SmartGridComm))

   We consider the decentralized control of radial distribution systems with controllable photovoltaic inverters and storage devices. For such systems, we consider the problem of designing controllers that minimize the expected cost of meeting demand, while respecting distribution system and resource constraints. Employing a linear approximation of the branch flow model, we formulate this problem as the design of a decentralized disturbance-feedback controller that minimizes the expected value of a convex quadratic cost function, subject to convex quadratic constraints on the state and input. As such problems are, in general, computationally intractable, we derive an inner approximation to this decentralized control problem, which enables the efficient computation of an affine control policy via the solution of a conic program. As affine policies are, in general, suboptimal for the systems considered, we provide an efficient method to bound their suboptimality via the solution of another conic program. A case study of a 12 kV radial distribution feeder demonstrates that decentralized affine controllers can perform close to optimal. 

   more » « less
2. Bridging Transient and Steady-State Performance in Voltage Control: A Reinforcement Learning Approach With Safe Gradient Flow

   https://doi.org/10.1109/LCSYS.2023.3289435  

   Feng, Jie; Cui, Wenqi; Cortés, Jorge; Shi, Yuanyuan (June 2023, IEEE Control Systems Letters)

   Deep reinforcement learning approaches are becoming appealing for the design of nonlinear controllers for voltage control problems, but the lack of stability guarantees hinders their real-world deployment. This letter constructs a decentralized RL-based controller for inverter-based real-time voltage control in distribution systems. It features two components: a transient control policy and a steady-state performance optimizer. The transient policy is parameterized as a neural network, and the steady-state optimizer represents the gradient of the long-term operating cost function. The two parts are synthesized through a safe gradient flow framework, which prevents the violation of reactive power capacity constraints. We prove that if the output of the transient controller is bounded and monotonically decreasing with respect to its input, then the closed-loop system is asymptotically stable and converges to the optimal steady-state solution. We demonstrate the effectiveness of our method by conducting experiments with IEEE 13-bus and 123-bus distribution system test feeders. 

   more » « less
3. Decentralized Stochastic Control of Distributed Energy Resources

   https://doi.org/10.1109/TPWRS.2017.2700472  

   Lin, Weixuan; Bitar, Eilyan (May 2017, IEEE Transactions on Power Systems)

   We consider the decentralized control of radial distribution systems with controllable photovoltaic inverters and energy storage resources. For such systems, we investigate the problem of designing fully decentralized controllers that minimize the expected cost of balancing demand, while guaranteeing the satisfaction of individual resource and distribution system voltage constraints. Employing a linear approximation of the branch flow model, we formulate this problem as the design of a decentralized disturbance-feedback controller that minimizes the expected value of a convex quadratic cost function, subject to robust convex quadratic constraints on the system state and input. As such problems are, in general, computationally intractable, we derive a tractable inner approximation to this decentralized control problem, which enables the efficient computation of an affine control policy via the solution of a finite-dimensional conic program. As affine policies are, in general, suboptimal for the family of systems considered, we provide an efficient method to bound their suboptimality via the optimal solution of another finite-dimensional conic program. A case study of a 12 kV radial distribution system demonstrates that decentralized affine controllers can perform close to optimal. 

   more » « less
4. Data-Driven Hierarchical Predictive Learning in Unknown Environments

   https://doi.org/10.1109/CASE48305.2020.9216872  

   Vallon, Charlott; Borrelli, Francesco (January 2020, IEEE International Conference on Automation Science and Engineering CASE)

   We propose a hierarchical learning architecture for predictive control in unknown environments. We consider a constrained nonlinear dynamical system and assume the availability of state-input trajectories solving control tasks in different environments. A parameterized environment model generates state constraints specific to each task, which are satisfied by the stored trajectories. Our goal is to find a feasible trajectory for a new task in an unknown environment. From stored data, we learn strategies in the form of target sets in a reduced-order state space. These strategies are applied to the new task in real-time using a local forecast of the new environment, and the resulting output is used as a terminal region by a low-level receding horizon controller. We show how to i) design the target sets from past data and then ii) incorporate them into a model predictive control scheme with shifting horizon that ensures safety of the closed-loop system when performing the new task. We prove the feasibility of the resulting control policy, and verify the proposed method in a robotic path planning application. 

   more » « less
5. Robust Reinforcement Learning for Risk-Sensitive Linear Quadratic Gaussian Control

   https://doi.org/10.1109/TAC.2024.3397928  

   Cui, Leilei; Başar, Tamer; Jiang, Zhong-Ping (January 2024, IEEE Transactions on Automatic Control)

   This paper proposes a novel robust reinforcement learning framework for discrete-time linear systems with model mismatch that may arise from the sim-to-real gap. A key strategy is to invoke advanced techniques from control theory. Using the formulation of the classical risk-sensitive linear quadratic Gaussian control, a dual-loop policy optimization algorithm is proposed to generate a robust optimal controller. The dual-loop policy optimization algorithm is shown to be globally and uniformly convergent, and robust against disturbances during the learning process. This robustness property is called small-disturbance input-to-state stability and guarantees that the proposed policy optimization algorithm converges to a small neighborhood of the optimal controller as long as the disturbance at each learning step is relatively small. In addition, when the system dynamics is unknown, a novel model-free off-policy policy optimization algorithm is proposed. Finally, numerical examples are provided to illustrate the proposed algorithm. 

   more » « less