More Like this

1. Simultaneous Allocation and Control of Distributed Energy Resources via Kullback-Leibler-Quadratic Optimal Control

   https://doi.org/10.23919/ACC45564.2020.9147402  

   Cammardella, Neil; Busic, Ana; Meyn, Sean (July 2020, American Control Conference)

   There is enormous flexibility potential in the power consumption of the majority of electric loads. This flexibility can be harnessed to obtain services for managing the grid: with carefully designed decision rules in place, power consumption for the population of loads can be ramped up and down, just like charging and discharging a battery, without any significant impact to consumers' needs. The concept is called Demand Dispatch, and the grid resource obtained from this design virtual energy storage (VES). In order to deploy VES, a balancing authority is faced with two challenges: 1. how to design local decision rules for each load given the target aggregate power consumption (distributed control problem), and 2. how to coordinate a portfolio of resources to maintain grid balance, given a forecast of net-load (resource allocation problem).Rather than separating resource allocation and distributed control, in this paper the two problems are solved simultaneously using a single convex program. The joint optimization model is cast as a finite-horizon optimal control problem in a mean-field setting, based on the new KLQ optimal control approach proposed recently by the authors.The simplicity of the proposed control architecture is remarkable: With a large portfolio of heterogeneous flexible resources, including loads such as residential water heaters, commercial water heaters, irrigation, and utility-scale batteries, the control architecture leads to a single scalar control signal broadcast to every resource in the domain of the balancing authority. Keywords: Smart grids, demand dispatch, distributed control, controlled Markov chains. 

   more » « less
2. Kullback-Leibler-Quadratic Optimal Control of Flexible Power Demand

   https://doi.org/10.1109/CDC40024.2019.9029512  

   Cammardella, Neil; Busic, Ana; Ji, Yuting; Meyn, Sean (December 2019, IEEE Conference Decision and Control)

   null (Ed.)

   A new stochastic control methodology is introduced for distributed control, motivated by the goal of creating virtual energy storage from flexible electric loads, i.e. Demand Dispatch. In recent work, the authors have introduced Kullback- Leibler-Quadratic (KLQ) optimal control as a stochastic control methodology for Markovian models. This paper develops KLQ theory and demonstrates its applicability to demand dispatch. In one formulation of the design, the grid balancing authority simply broadcasts the desired tracking signal, and the hetero-geneous population of loads ramps power consumption up and down to accurately track the signal. Analysis of the Lagrangian dual of the KLQ optimization problem leads to a menu of solution options, and expressions of the gradient and Hessian suitable for Monte-Carlo-based optimization. Numerical results illustrate these theoretical results. 

   more » « less
3. Virtual energy storage through decentralized load control with quality of service bounds

   https://doi.org/10.23919/ACC.2017.7963040  

   Brooks, Jonathan; Barooah, Prabir (May 2017, American Control Conference)

   null (Ed.)

   We propose a decentralized algorithm to help reduce demand-supply imbalance in a power grid by varying the demand from loads, just like charging and discharging a battery. The algorithm ensures strict bounds on the consumers' quality of service (QoS) by constraining the bandwidth of demand variation. A model-predictive-control formulation is adopted to compute local decisions at the loads. The algorithm is decentralized in the sense that loads do not communicate with one another. Instead, loads coordinate using local measurements of the grid frequency, which provide information about global demand-supply imbalance. It is envisioned that consumers will be recruited through long-term contracts, aided by the QoS guarantees provided by the proposed scheme. Simulation results show that loads are able to reduce frequency deviations while maintaining QoS constraints and that the performance of the algorithm scales well with the number of loads. Closed-loop stability is established under some assumptions. 

   more » « less
4. Balancing the Power Grid with Cheap Assets

   https://doi.org/10.1109/CDC49753.2023.10383996  

   Meyn, Sean; Lu, Fan; Mathias, Joel (December 2023, IEEE)

   We have all heard that there is growing need to secure resources to obtain supply-demand balance in a power grid facing increasing volatility from renewable sources of energy. There are mandates for utility scale battery systems in regions all over the world, and there is a growing science of “demand dispatch” to obtain virtual energy storage from flexible electric loads such as water heaters, air conditioning, and pumps for irrigation. The question addressed in this tutorial is how to manage a large number of assets for balancing the grid. The focus is on variants of the economic dispatch problem, which may be regarded as the “feed-forward” component in an overall control architecture. 1) The resource allocation problem is identical to a finite horizon optimal control problem with degenerate cost—so called “cheap control”. This implies a form of state space collapse, whose form is identified: the marginal cost for each load class evolves in a two-dimensional subspace, spanned by a scalar co-state process and its derivative. 2) The implication to distributed control is remarkable. Once the co-state process is synthesized, this common signal may be broadcast to each asset for optimal control. However, the optimal solution is extremely fragile, in a sense made clear through results from numerical studies. 3) Several remedies are proposed to address fragility. One is described through “robust training” in a particular Q-learning architecture (one approach to reinforcement learning). In numerical studies it is found that specialized training leads to more robust control solutions. 

   more » « less
5. Kullback–Leibler-Quadratic Optimal Control

   https://doi.org/10.1137/21M1433654  

   Cammardella, Neil; Bušić, Ana; Meyn, Sean P. (October 2023, SIAM Journal on Control and Optimization)

   Editor-in-Chief: George Yin (Ed.)

   This paper presents approaches to mean-field control, motivated by distributed control of multi-agent systems. Control solutions are based on a convex optimization problem, whose domain is a convex set of probability mass functions (pmfs). The main contributions follow: 1. Kullback-Leibler-Quadratic (KLQ) optimal control is a special case, in which the objective function is composed of a control cost in the form of Kullback-Leibler divergence between a candidate pmf and the nominal, plus a quadratic cost on the sequence of marginals. Theory in this paper extends prior work on deterministic control systems, establishing that the optimal solution is an exponential tilting of the nominal pmf. Transform techniques are introduced to reduce complexity of the KLQ solution, motivated by the need to consider time horizons that are much longer than the inter-sampling times required for reliable control. 2. Infinite-horizon KLQ leads to a state feedback control solution with attractive properties. It can be expressed as either state feedback, in which the state is the sequence of marginal pmfs, or an open loop solution is obtained that is more easily computed. 3. Numerical experiments are surveyed in an application of distributed control of residential loads to provide grid services, similar to utility-scale battery storage. The results show that KLQ optimal control enables the aggregate power consumption of a collection of flexible loads to track a time-varying reference signal, while simultaneously ensuring each individual load satisfies its own quality of service constraints. 

   more » « less