Many engineering systems must be operated at an "optimal" steady-state that minimizes operational costs. For example, the generators that supply power to the electrical grid are scheduled according to the solution of an optimization problem which minimizes the total production cost of the generated power [1]. This same theme of guiding system variables to optimizers emerges in other areas, such as network congestion management [2], [3], chemical processing [4], [5], and wind turbine power capture [6, Section 2.7], [7].
Traditionally, the optimal steady-state set-points are computed offline in advance, and then controllers are used in real time to track the set-points. However, this two-step design method is inefficient if the set-points must be updated repeatedly and often. For instance, the increased use of renewable energy sources causes rapid fluctuations in power networks, which decreases the effectiveness of the separated approach due to the rapidly-changing optimal steady-state [8]. Furthermore, the two-step method is infeasible if unmeasurable disturbances change the optimal set-points. To continuously keep operational costs to a minimum, such systems should instead employ a controller that continuously solves the optimization problem and guides the system to an optimizer despite disturbances; we will call the problem of designing such a controller the optimal steady-state (OSS) control problem.
The prevalence of the OSS control problem in applications has motivated much work on its solution for general system classes. In the extremum-seeking control approach to OSS control, a harmonic signal is used to perturb an uncertain system, and the gradient of a cost function is then inferred by filtering system measurements; a control signal is applied to drive the gradient to zero [4], [9], [10]. Jokić, Lazar, and van den Bosch propose using the Karush-Kuhn-Tucker conditions for optimality as the basis of a nonlinear feedback controller that guides the outputs of a system to an optimizer [11], [12]. Nelson and Mallada consider an optimization problem over system states and apply gradient feedback with a proportionalintegral (PI) controller; if the full system state cannot be directly measured, a Luenberger observer is employed [13].
Much of the literature on OSS control problems focuses on the optimization of either the steady-state input or the steady-state output of the system. The optimal power flow problem, for example, concerns the minimization of the total cost of power produced by dispatchable units that serve as inputs [14]- [16]. Other work has focused on optimization problems over the output only; the cost of applying the input necessary to achieve the desired output is not considered [4], [11], [12], [17]- [19].
These input-only or output-only optimization designs are formulated under the assumption that the resulting optimizer is in fact consistent with the dynamic system operating in equilibrium. As such, the key obstacle to relaxing this assumption -and to extending the formulations to steadystate input-output optimization -is that an arbitrary inputoutput optimizer need not be consistent with dynamic equilibrium. Controllers proposed currently in the literature for optimization over both the input and output only apply to systems with special structure: [16], [20] assume the plant can be interpreted as a primal-dual gradient algorithm, while Hatanaka and colleagues consider the specific case of temperature dynamics in a building [21]. An OSS controller design for the general case is still lacking.
The main contribution of this paper is an OSS controller design for any stabilizable linear time-invariant (LTI) system where the steady-state objective function depends on both the input and output. We ensure the optimizer is consistent with the equilibrium conditions for the LTI system by including the equilibrium equations as explicit equality constraints of the optimization problem. Our key insight is that, rather than enforcing these equality constraints with dual variables in the controller, one may enforce that the gradient of the objective function lie (at equilibrium) in a subspace defined by the LTI system matrices. This eliminates the need for additional dynamic controller states, resulting in a lower-dimensional controller design. We offer two strategies to ensure closed-loop stability: search for appropriate proportional-integral gains using a linear matrix inequality stability criterion, or design a dynamic stabilizer using H ∞ synthesis methods.
The n × n identity matrix is I n , 0 is a matrix of zeros of appropriate dimension, and 0 n is the n-vector of all zeros. The vector norm • is always assumed to be the Euclidean norm • 2 . For symmetric matrices A and B, the inequality A B means A -B is positive definite, while A B means A -B is positive semidefinite. If h : R n → R is differentiable, ∇h : R n → R n denotes its gradient. We say h : R n → R is strongly convex with parameter κ > 0 if (∇h(x) -∇h(y))
T (x -y) ≥ κ x -y 2 for all x, y ∈ R n .
The gradient ∇h is (globally) Lipschitz continuous with parameter L > 0 if ∇h(x) -∇h(y) ≤ L x -y for all x, y ∈ R n .
We consider the LTI system
as the plant whose input and output we wish to optimize in steady-state. We omit the direct throughput term Du for simplicity of presentation. The control input is u ∈ R m , the state is x ∈ R n , and the output is y ∈ R p . We assume that m, p ≤ n. The plant is subject to an unknown but constant disturbance d ∈ R n . We denote equilibrium values of the state, input, and output by x, ū, and ȳ respectively, which satisfy Ax + B ū + d = 0 n and ȳ = C x.
The desired steady-state operating point (ȳ, ū) of the system (1) is determined by the optimization problem
The cost function g : R p ×R m → R is a differentiable convex function of the equilibrium output ȳ and the equilibrium control input ū. The feasible region is the set of forced equilibrium points for the plant dynamics. Including additional equality and inequality constraints in (2) is a relatively straightforward extension of the results that follow, and will be detailed in a forthcoming extended manuscript. Consider a nonlinear feedback controller for the plant (1) with state x c ∈ R nc of the form
The precise problem we wish to solve is as follows.
Problem 2.1 (Optimal Steady-State Control Problem): Design a feedback controller of the form (3) processing measurements y(t) and producing control signal u(t) such that for any constant disturbance vector d ∈ R n the following hold for the closed-loop system (1) and ( 3):
(i) the closed-loop system has a globally asymptotically stable equilibrium point; (ii) for any initial condition (x(0),
) is an optimizer of (2).
As the control system operates in real time, the disturbance d may change, and each change in its value changes the optimal operating point of (2) by modifying the feasible set. Problem 2.1 specifies that the controller must maintain stability and automatically guide the plant to the new optimizer.
We make a number of assumptions that are necessary for the OSS control problem to be feasible.
Assumption 2.1 (Stabilizable Plant): The matrix pair (A, B) is stabilizable.
By the Popov-Belevitch-Hautus test, (A, B) is stabilizable if and only if rank A -λI n B = n for all complex numbers λ with nonnegative real part [22,Section 14.3]. The OSS control problem is infeasible if the plant fails to be stabilizable, as a lack of internal stability precludes reaching steady-state from an arbitrary initial condition.
Remark 2.1 (Nonempty Feasible Set): Assumption 2.1 implies the feasible set of ( 2) is non-empty. Take λ = 0 in the definition of stabilizability to obtain
The OSS control problem is infeasible if the optimization problem has no solution, since the desired steady-state operating point does not exist.
With these necessary assumptions in place, we move on to controller design. The OSS control problem is composed of two sub-problems. The first sub-problem is to design a controller that establishes the optimizers of (2) as the only equilibrium points of the closed-loop system. The second sub-problem is to ensure closed-loop stability. In Section III we present a controller that satisfies the equilibrium criterion, then in Section IV we discuss stabilization.
We will first examine the full-state measurement case when y = x, and then show how to extend the results to the measurement case when y = Cx. To distinguish the former from the latter, we denote the cost function in the optimization problem by f (x, ū) instead of g(ȳ, ū). The desired steadystate operating point is determined by
We begin by discussing the optimality conditions associated with the problem (4).
For a convex optimization problem, the Karush-Kuhn-Tucker (KKT) conditions are a set of necessary and sufficient conditions for a point to be a global optimizer, provided strong duality holds. For a convex optimization problem with affine constraints only -like problems ( 2) and ( 4) -strong duality holds as long as the feasible set is nonempty and the domain of the objective function is open [23, Section 5.2.3], both of which follow from our assumptions.
The KKT conditions for (4) state: (x , u ) is a global optimizer if and only if there exists a λ ∈ R n such that
The first condition (5a) is simply feasibility of (x , u ). The second condition (5b) is the gradient condition, with λ the vector of dual variables associated with the constraints of the optimization problem.
We can achieve steady-state optimality by enforcing the KKT conditions (5) in equilibrium using a feedback controller. Previous designs in the literature enforce constraints using controller states that act as dual variables; consider [11,Equation (4a)], or [20,Equation (8f)]. However, if the closedloop system is internally stable, the plant dynamics already enforce the constraint (5a) in steady-state; this suggests that dynamic dual variables are unnecessary. We can dispense with dual variables entirely and simplify the design procedure with an alternative interpretation of the KKT conditions.
The gradient condition (5b) is equivalent to the statement ∇f (x , u ) ∈ range A B
The condition (6b) can be interpreted geometrically as saying that the gradient ∇f (x , u ) is orthogonal to the subspace of first-order feasible variations at an optimal point; see [25,Section 3.1]. This alternative formulation of the KKT conditions does not explicitly require dual variables, and we can enforce (6b) in equilibrium using a well-chosen matrix in the controller, as we describe next.
To enforce (6b) in equilibrium, the controller must drive the component of ∇f in the subspace null A B to zero. The core idea of our control strategy is to make this component of ∇f the error signal input to a servo-compensator [26] with integral action; here we will simply use a proportionalintegral controller.
Since rank A B = n by Assumption 2.1, it follows from the rank-nullity theorem that dim null A B = m. Let {v 1 , v 2 , . . . , v m } be a basis of null A B and concatenate these vectors to form the matrix Q ∈ R m×(n+m) as We take e := -Q T ∇f as the error signal. We can interpret Q T as extracting the component of ∇f in the subspace null A B , which must be driven to zero to achieve optimality.
The equations of the system in feedback with the controller in the full-state measurement case are
A block diagram for the closed-loop system is shown in Figure 1. Equation (8b) is the error term which the controller brings to zero using proportional-integral control. The integrator is given by (8c). The input equation (8d) contains proportional and integral gains K P , K I ∈ R m×m , where K I is assumed to be invertible. We assume going forward that the closed-loop system is well-posed, meaning that for any (x, η), the equations (8b) and (8d) can always be solved for a unique value of u.
We now show the closed-loop system achieves optimality in steady-state. For a constant disturbance d, the equilibrium points (x, η) of the closed-loop system (8) are the solutions of
Define the set of equilibrium state-input pairs as
and let the set of optimizers for the optimization problem be
Theorem 3.1 (Equilibria and Optimizers):
The sets E and χ of ( 10) and ( 11) satisfy E = χ.
Proof: First note that (9a) and (9b) are equivalent to the KKT conditions (6). The former is the feasibility condition (6a), and the latter implies (6b) since
We show that χ ⊂ E. The set of optimizers χ is non-empty by Assumption 2.2, so there exists a (x , u ) ∈ χ. Since the KKT conditions are necessary for optimality, (x , u ) satisfy (9a) and (9b). Therefore, (x, η) = (x , K -1 I u ) is a solution of (9). It follows that χ ⊂ E, which also implies E is nonempty. Conversely, we now show E ⊂ χ. If (x, ū) ∈ E, then x and ū satisfy (9a) and (9b). The KKT conditions are sufficient for optimality, thus (x, ū) must be a global minimizer of (4). Therefore E ⊂ χ.
Before moving on to the measurement case, let us make two comments regarding the matrix Q.
First, the choice of Q is not unique; the only required property is (7). While a detailed study is outside the present scope, the choice of Q clearly affects the performance of the closed-loop system, and the sparsity patterns of different choices can provide flexibility for distributed implementations of OSS controllers. The construction of Q relies on knowledge of the plant matrices A and B; this raises the question of what will happen if A and B are not known accurately. Suppose the actual plant matrices are à and B. The controller (8) still guides the system to the optimizer of
and if the closed-loop system remains internally stable. We defer a more complete robustness analysis to a future paper.
Second, consider the advantage of enforcing the KKT conditions using the matrix Q over enforcing the KKT conditions using dual variables as employed in [11] or [20] for example. The latter approach requires additional controller states representing the dual variables. Our alternative strategy omits such additional states, resulting in a lower-order controller.
We now return to the original input-output optimization problem (2) with measurements y = Cx and frame the measurement case as an instance of the full-state measurement case by defining f (x, u) := g(Cx, u). It is straightforward to prove from the definition of a convex function that f is convex whenever g is convex. We then apply the controller (8) with
by the multidimensional chain rule. The closed-loop system becomes
where
Theorem 3.1 holds for (12), and under additional assumptions on the plant and the objective function g we can further show uniqueness of the closed-loop equilibrium point.
Proposition 3.2 (Unique Equilibrium Point): Suppose the objective function g is strictly convex and (C, A) is detectable. Then for each d ∈ R n the closed-loop system (12) has a unique equilibrium point.
Proof: If g is strictly convex, then for every optimizer (x , y , u ) of ( 2 is unique. Therefore, the set χ of ( 11) is a singleton. Since χ = E by Theorem 3.1, we conclude ( 12) has a unique equilibrium point.
We have so far shown that our controller solves the first subproblem of the OSS control problem; namely, the equilibrium points of the closed-loop system are optimal. We turn our attention to the second sub-problem, that of stabilizing the closed-loop system.
The proportional and integral gain matrices K P and K I may be tuned "by hand," using a stability criterion to determine whether the arbitrarily chosen gains result in a stable system; alternatively, we might consider employing a synthesis procedure to design a stabilizing controller. The former is simpler, and we can continue to use the PI structure we have been discussing up to this point. The latter is more complex, requiring a dynamic stabilizer, but systematizes gain tuning. We consider each strategy in turn.
If we wish to select the gain matrices K P and K I heuristically, we require a method of assessing whether the resulting closed-loop system is stable. The only nonlinearity in the closed-loop system (12) is the memoryless operator ∇g. If the assumptions of Proposition 3.2 hold, and a unique equilibrium point exists, then we can put the closed-loop system in exactly the form of the absolute stability problem [27, Section 7.1] and apply the corresponding analysis tools. We therefore assume (C, A) is detectable and g is strictly convex in this section.
We centre the equations ( 12) about the unique (optimal) equilibrium point (x , η ) by defining the new state variables x := x -x and η := η -η . We further define y := Cx and u := K I η . Straightforward calculations show that this change-of-variables results in the feedback interconnection of the memoryless operator Φ(ỹ, ũ) := ∇g(ỹ + y , ũ + u ) -∇g(y , u ) and the LTI system H, which collects all the linear elements of (12). The system H has the realization
(y, u) -Fig. 2: The feedback interconnection of Figure 1 in the standard form of the absolute stability problem. The linear system H collects all the linear elements and the nonlinearity Φ is a memoryless operator satisfying (incremental) sector bounds.
The operator Φ is a static nonlinearity in the sector [0, ∞) owing to the convexity of g; if, furthermore, g is strongly convex with parameter κ and ∇g is Lipschitz continuous with parameter L, then Φ is in the sector [κ, L]. 1 The feedback interconnection of H and Φ, depicted in Figure 2, is therefore precisely in the form of the absolute stability problem. We present a stability criterion in the form of a linear matrix inequality (LMI). For background on LMIs, see [28]. The following proposition is equivalent to [27, Theorem 7.1], but has the advantage of being easy to verify with a program like CVX or YALMIP. Proposition 4.1 (OSS Stability Criterion): Suppose g is strongly convex with parameter κ > 0 and ∇g is Lipschitz continuous with parameter L > 0. Let
where ⊗ denotes the Kronecker product. The unique equilibrium point of ( 12) is globally asymptotically stable if there exists a symmetric matrix P = P T ∈ R (n+m)×(n+m) and a real number α ≥ 0 such that
Proof: Apply [29, Corollary 6] to the system under consideration with the circle criterion multiplier [29,Class 12]. Remark 4.1 (Form of Matrix M ): The term in parentheses in the definition of M is a product of three matrices. The second matrix comes from the definition of a memoryless operator with a lower sector bound of κI p+m and an upper sector bound of LI p+m . The first and third matrices are the congruence transformation from [30, Section 6.1] necessary to make the feedback interconnection positive instead of negative, as required by [29,Corollary 6].
If the closed-loop system satisfies the LMI of Proposition 4.1, the OSS control problem is solved. Relaxing the strong convexity and Lipschitz continuity assumptions appears to be feasible but will necessitate a more sophisticated stability analysis, which we do not pursue here. We will demonstrate by simulation in Section V that the Lipschitz assumption is not necessary for stability.
Using the stability criterion of Proposition 4.1, one can attempt gain tuning by performing a search over the gain matrices K P and K I for selections that result in feasibility of the LMI. For instance, set K P = k P I m and K I = k I I m with scalar parameters k P ∈ R and k I ∈ R and search over a grid for a stabilizing combination (k P , k I ) ∈ R 2 . The drawback to this strategy is its heuristic nature.
Should the "hand-tuned" method of the previous section fail, we can attempt a more general synthesis procedure at the cost of a more complex controller structure. We replace the PI gain equation (12e) with a dynamic stabilizer, so that the input u is generated by
in which σ := (y, η, e). The design variables are the matrices A s , B s , C s , and D s , which we select to enforce that a stability LMI like the one of Proposition 4.1 is satisfied.
The most straightforward synthesis method is to perform a loop transformation to move the nonlinearity Φ from the sector [κ, L] to the sector [-1, 1] and apply standard tools from the H ∞ synthesis problem to minimize the L 2 gain γ of the augmented linear system. The resulting closed-loop system is stable by the small-gain theorem if γ < 1. For more on loop transformations, see [27,Chapters 6,7]. For details on the H ∞ synthesis problem, see [31,Chapter 3].
We illustrate the tools of Sections III and IV through three simple examples. The first example shows the use of Proposition 4.1 to verify stability using hand-tuned PI gains. The second example shows a simulation that suggests the Lipschitz bound on ∇g is not necessary for stability. The third example shows that a dynamic controller serves to stabilize the closed-loop system when the PI controller fails.
We take as the plant matrices
One may verify that (A, B) is stabilizable and (C, A) is detectable. We make the columns of Q an orthonormal basis of null A B , resulting in Q T = 0.1661 0.2491 -0.6644 0.1661 0.6644 .
We let R be as defined in (13).
We suppose we are interested in stabilizing the system for any objective function that is strongly convex with parameter κ = 1/9 and whose gradient is Lipschitz continuous with parameter L = 1. We were able to verify the corresponding optimizer is globally asymptotically stable using Proposition 4.1 for the 100 gain combinations (K P , K I ) ∈ {0.2, 0.4, . . . , 2} 2 .
Although Proposition 4.1 supposes ∇g is globally Lipschitz continuous, simulations show that the closed-loop system may be stable even if this assumption does not hold. Consider the objective function
The function g is strongly convex with parameter κ = 1/9, but ∇g is not globally Lipschitz. We suppose the disturbance input d(t) is -1 3 1 2 T for 0 ≤ t < 5, 2 -3 0 0 T for 5 ≤ t < 10, and 1 0 0 -1 T for t ≥ 10. We set K I = 5 and K P = 10. Simulating the closedloop system in MATLAB using the ode15i implicit ODE solver yields Figure 3. The controller tracks the optimizer asymptotically for each value of the disturbance input, despite ∇g failing the global Lipschitz condition. The matrix M in Proposition 4.1 can be modified to attempt stability verification in the case when g is strongly convex but ∇g is not globally Lipschitz. Specifically, one may set
However, this modified LMI was infeasible for any gain value we tried. This suggests that in general, more sophisticated stability criteria are required when the objective function does not have a globally Lipschitz gradient. Note ∇g of this example is still locally Lipschitz; that is, ∇g has a Lipschitz parameter when its domain is restricted to a bounded subset of R p+m . If one could establish boundedness of trajectories in advance, then semi-global stability results should follow.
As mentioned in Section IV, an alternative to hand-tuning the PI controller and verifying stability using Proposition 4.1 is to use a fully dynamic LTI controller and employ H ∞ synthesis methods to select the controller matrices. We suppose we are interested in OSS control for the unstable
and objective functions that are strongly convex with parameter κ = 1 and have Lipschitz continuous gradients with parameter L = 2. Once again (A, B) is stabilizable and (C, A) is detectable. The matrix A has eigenvalues {-2, -2 + 2i, -2 -2i, 1}. We let the disturbance d(t) be the same as in Section V-B. We attempted stability verification using Proposition 4.1 for the 49 gain combinations P , K I ) ∈ {10 -3 , 10 -2 , . . . , 10 2 , 10 3 } 2 . The LMI solver failed in each case, and simulations further suggest the PI controller is incapable of stabilizing the closed-loop system. By contrast, we were able to synthesize a functioning dynamic controller whose behaviour is demonstrated in Figure 4 for the objective function g(y, u) = y 2 1 + 1 2 y 2 2 + 1 2 u 2 . While the dynamic controller is stabilizing, the closed-loop performance is poor. The response exhibits large overshoot and slow convergence to the optimizer. A more sophisticated synthesis procedure using the methods described in [32] could design for stability and performance simultaneously, and is one focus of future work.
We have presented a controller that solves the optimal steady-state control problem for linear time-invariant systems. Our controller is able to optimize over both the input and output by including the equilibrium equations as constraints of the optimization problem, but does not require additional dual variable controller states to be associated with these constraints. The potential application areas of optimal steadystate controllers are numerous -including power systems and chemical processing plants -as these controllers minimize the operating cost of any engineering system whose optimal set-point changes over time.
Future work will present a more general definition of the OSS control problem, with an exploration of the necessary and sufficient conditions on the controller. This generalization will include: adding equality and inequality constraints to the optimization problem to represent engineering constraints such as actuator limits or tie-line flow contracts in multiarea power systems; extending the approach to time-varying disturbances; explicitly considering robustness to parametric uncertainty; and outlining an architecture for OSS controllers that divides the controller structure into several sub-systems with well-defined roles.
Sector-boundedness is a property of the input-output behaviour of a static nonlinearity; see [27, Definition 6.2] for details.