Automatica 38 (2002) 3}20 The explicit linear quadratic regulator for constrained systems夽 Alberto Bemporad *, Manfred Morari, Vivek Dua, Efstratios N. Pistikopoulos Dip. Ingegneria dell'Informazione, Universita% di Siena, Via Roma 56, 53100 Siena, Italy Automatic Control Laboratory, ETH Zentrum, ETL I 26, 8092 Zurich, Switzerland Centre for Process Systems Engineering, Imperial College, London SW7 2BY, UK Received 24 September 1999; revised 9 October 2000; received in "nal form 16 June 2001 We present a technique to compute the explicit state-feedback solution to both the xnite and inxnite horizon linear quadratic optimal control problem subject to state and input constraints. We show that this closed form solution is piecewise linear and continuous. As a practical consequence of the result, constrained linear quadratic regulation becomes attractive also for systems with high sampling rates, as on-line quadratic programming solvers are no more required for the implementation. Abstract For discrete-time linear time invariant systems with constraints on inputs and states, we develop an algorithm to determine explicitly, the state feedback control law which minimizes a quadratic performance criterion. We show that the control law is piece-wise linear and continuous for both the "nite horizon problem (model predictive control) and the usual in"nite time measure (constrained linear quadratic regulation). Thus, the on-line control computation reduces to the simple evaluation of an explicitly de"ned piecewise linear function. By computing the inherent underlying controller structure, we also solve the equivalent of the Hamilton}Jacobi}Bellman equation for discrete-time linear constrained systems. Control based on on-line optimization has long been recognized as a superior alternative for constrained systems. The technique proposed in this paper is attractive for a wide range of practical problems where the computational complexity of on-line optimization is prohibitive. It also provides an insight into the structure underlying optimization-based controllers. 2001 Elsevier Science Ltd. All rights reserved. Keywords: Piecewise linear controllers; Linear quadratic regulators; Constraints; Predictive control 1. Introduction As we extend the class of system descriptions beyond the class of linear systems, linear systems with constraints are probably the most important class in practice and the most studied. It is well accepted that for these systems, in general, stability and good performance can only be 夽 This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor Per-Olof Gutman under the direction of Editor Tamer Basar. * Corresponding author. Dip. Ingegneria dell'Informazione, Universita% di Siena, Via Roma 56, 53100 Siena, Italy. E-mail addresses: [email protected] (A. Bemporad), [email protected] (M. Morari), [email protected] (V. Dua), [email protected] (E. N. Pistikopoulos). achieved with a non-linear control law. The most popular approaches for designing non-linear controllers for linear systems with constraints fall into two categories: anti-windup and model predictive control. Anti-windup schemes assume that a well functioning linear controller is available for small excursions from the nominal operating point. This controller is augmented by the anti-windup scheme in a somewhat ad hoc fashion, to take care of situations when constraints are met. Kothare et al. (1994) reviewed numerous apparently di!erent anti-windup schemes and showed that they di!er only in their choice of two static matrix parameters. The least conservative stability test for these schemes can be formulated in terms of a linear matrix inequality (LMI) (Kothare & Morari, 1999). The systematic and automatic synthesis of anti-windup schemes which guarantee closed-loop stability and achieve some kind of optimal 0005-1098/02/$ - see front matter 2001 Elsevier Science Ltd. All rights reserved. PII: S 0 0 0 5 - 1 0 9 8 ( 0 1 ) 0 0 1 7 4 - 1 4 A. Bemporad et al. / Automatica 38 (2002) 3}20 performance, has remained largely elusive, though some promising steps were achieved recently (Mulder, Kothare, & Morari, 1999; Teel & Kapoor, 1997). Despite these drawbacks, anti-windup schemes are widely used in practice because in most SISO situations, they are simple to design and work adequately. Model predictive control (MPC) has become the accepted standard for complex constrained multivariable control problems in the process industries. Here, at each sampling time, starting at the current state, an open-loop optimal control problem is solved over a "nite horizon. At the next time step, the computation is repeated starting from the new state and over a shifted horizon, leading to a moving horizon policy. The solution relies on a linear dynamic model, respects all input and output constraints, and optimizes a quadratic performance index. Thus, as much as a quadratic performance index together with various constraints can be used to express true performance objectives, the performance of MPC is excellent. Over the last decade, a solid theoretical foundation for MPC has emerged so that in real life, large-scale MIMO applications controllers with non-conservative stability guarantees can be designed routinely and with ease (Qin & Badgwell, 1997). The big drawback of the MPC is the relatively formidable on-line computational e!ort, which limits its applicability to relatively slow and/or small problems. In this paper, we show how to move all the computations necessary for the implementation of MPC o!line, while preserving all its other characteristics. This should largely increase the range of applicability of MPC to problems where anti-windup schemes and other ad hoc techniques dominated up to now. Moreover, such an explicit form of the controller provides additional insight for better understanding of the control policy of MPC. We also show how to solve the equivalent of the Hamilton}Jacobi}Bellman equation for discrete-time linear constrained systems. Rather than gridding the state space in some ad hoc fashion, we discover the inherent underlying controller structure and provide its most e$cient parameterization. The paper is organized as follows. The basics of MPC are reviewed "rst to derive the quadratic program which needs to be solved to determine the optimal control action. We proceed to show that the form of this quadratic program is maintained for various practical extensions of the basic setup, for example, trajectory following, suppression of disturbances, time-varying constraints and also the output feedback problem. As the coe$cients of the linear term in the cost function and the right-hand side of the constraints depend linearly on the current state, the quadratic program can be viewed as a multiparametric quadratic program (mp-QP). We analyze the properties of mp-QP, develop an e$cient algorithm to solve it, and show that the optimal solution is a piecewise a$ne function of the state (con"rming other investigations on the form of MPC laws (Chiou & Za"riou, 1994; Tan, 1991; Za"riou, 1990; Johansen, Petersen, & Slupphaug, 2000). The problem of synthesizing stabilizing feedback controllers for linear discrete-time systems, subject to input and state constraints, was also addressed in Gutman (1986) and Gutman and Cwikel (1986). The authors obtain a piecewise linear feedback law de"ned over a partition of the set of states X into simplicial cones, by computing a feasible input sequence for each vertex via linear programming (this technique was later extended in Blanchini, 1994). Our approach provides a piecewise a$ne control law which not only ensures feasibility and stability, but is also optimal with respect to LQR performance. The paper concludes with a series of examples which illustrate the di!erent features of the method. Other material related to the contents of this paper can be found at http://control.ethz.ch/ &bemporad/ explicit. 2. Model predictive control Consider the problem of regulating to the origin the discrete-time linear time invariant system x(t#1)"Ax(t)#Bu(t), (1) y(t)"Cx(t), while ful"lling the constraints y )y(t))y , u )u(t))u (2) at all time instants t*0. In (1)}(2), x(t)31L, u(t)31K, and y(t)31N are the state, input, and output vectors, respectively, y )y (u )u ) are p(m)-dimensional vectors, and the pair (A, B) is stabilizable. Model predictive control (MPC) solves such a constrained regulation problem in the following way. Assume that a full measurement of the state x(t) is available at the current time t. Then, the optimization problem ,W \ J(;, x(t))"x W Px W # [x min R>, R R>, R R>IR I 3OSR 2SR>,S\ Qx #u Ru ] , R>IR R>I R>I The results of this paper hold also for the more general mixed constraints D x(t)#D u(t))d. A. Bemporad et al. / Automatica 38 (2002) 3}20 s.t. y )y )y , k"1,2, N , R>IR u )u )u , k"0,1,2, N , R>I x "x(t), RR x "Ax #Bu , k*0, R>I>R R>IR R>I y "Cx , k*0, R>IR R>IR u "Kx , N )k(N , R>I R>IR S W (3) is solved at each time t, where x denotes the predicted R>IR state vector at time t#k, obtained by applying the input sequence u ,2, u to model (1) starting from the R R>I\ state x(t). In (3), we assume that Q"Q_0, R"RY0, P_0, (Q, A) detectable (for instance, Q"CC with (C, A) detectable), K is some feedback gain, N , N , N W S are the output, input, and constraint horizons, respectively, with N )N and N )N !1 (the results of this S W W paper also apply for N *N ). W One possibility is to choose K"0 (Rawlings & Muske, 1993), and P as the solution of the Lyapunov equation P"APA#Q. (4) The choice is only meaningful for open-loop stable systems, as it implies that after N time steps, the control S is turned o!. Alternatively, one can set K"K (Chmielewski */ & Manousiouthakis, 1996; Scokaert & Rawlings, 1998), where K and P are, the solutions of the unconstrained */ in"nite horizon LQR problem with weights Q, R K "!(R#BPB)\BPA, */ (5a) P"(A#BK )P(A#BK )#K RK #Q. */ */ */ */ (5b) This choice of K implies that, after N time steps, the S control is switched to the unconstrained LQR. With P obtained from (5), J(;, x(t)) measures the settling cost of the system from the present time t to in"nity under this control assumption. In addition, if one guarantees that the predicted input and output evolutions do not violate the constraints also at later time steps t#k, k"N #1,2,#R, this strategy is indeed the optimal S in"nite horizon constrained LQ policy. This point will be addressed in Section 3. The MPC control law is based on the following idea: At time t, compute the optimal solution ;H(t)" uH,2, uH S to problem (3), apply R R>, \ u(t)"uH R 5 The two main issues regarding this policy are feasibility of the optimization problem (3) and stability of the resulting closed-loop system. When N (R, there is no guarantee that the optimization problem (3) will remain feasible at all future time steps t, as the system might enter `blind alleysa where no solution to problem (3) exists. On the other hand, setting N "R leads to an optimization problem with an in"nite number of constraints, which is impossible to handle. If the set of feasible state#input vectors is bounded and contains the origin in its interior, by using arguments from maximal output admissible set theory (Gilbert & Tin Tan, 1991), for the case K"0 Bemporad (1998) showed that a "nite constraint horizon, N , can be chosen without loss of guarantees of constraint ful"llment, and a similar argument can be repeated for the case K"K . Similar */ results about the choice of the smallest N ensuring feasibility of the MPC problem (3) at all time instants were also proved in Gutman and Cwikel (1986) and have been extended recently in Kerrigan and Maciejowski (2000). The stability of MPC feedback loops was investigated by numerous researchers. Stability is, in general, a complex function of the various tuning parameters N , N , N , P, Q, and R. For applications, it is most S W useful to impose some conditions on N , N and P, so W that stability is guaranteed for all Q_0, RY0. Then, Q and R can be freely chosen as tuning parameters to a!ect performance. Sometimes, the optimization problem (3) is augmented with a so-called `stability constrainta (see Bemporad & Morari (1999) for a survey of di!erent constraints proposed in the literature). This additional constraint imposed over the prediction horizon, explicitly forces the state vector either to shrink in some norm or to reach an invariant set at the end of the prediction horizon. Most approaches for proving stability follow in spirit the arguments of Keerthi and Gilbert (1988) who establish the fact that under some conditions, the value function <(t)"J(;H(t), t) attained at the minimizer ;H(t) is a Lyapunov function for the system. Below, we recall a simple stability result based on such a Lyapunov argument (see also Bemporad, Chisci, & Mosca, 1994). Theorem 1. Let N "R, K"0 or K"K , and W */ N (R be suzciently large for guaranteeing existence of feasible input sequences at each time step. Then, the MPC law (3)}(6) asymptotically stabilizes system (1) while (6) as input to system (1), and repeat the optimization (3) at time t#1, based on the new state x(t#1). Such a control strategy is also referred to as moving or receding horizon. As discussed above, several techniques exist to compute constraint horizons N which guarantee feasibility, see Bemporad (1998), Gilbert and Tin Tan (1991), Gutman and Cwikel (1986) and Kerrigan and Maciejowski (2000). 6 A. Bemporad et al. / Automatica 38 (2002) 3}20 enforcing the fulxllment of the constraints (2) from all initial states x(0) such that (3) is feasible at t"0. Theorem 1 ensures stability, provided that the optimization problem (3) is feasible at time t"0. The problem of determining the set of initial conditions x(0) for which (3) is feasible has been addressed in Gutman and Cwikel (1986), and more recently in Kerrigan and Maciejowski (2000). (Acevedo & Pistikopoulos, 1999; Dua & Pistikopoulos, 1999, 2000; Fiacco, 1983; Gal, 1995). In Section 4, we will describe an algorithm to solve mp-QP problems. To the authors' knowledge, no algorithm for solving mp-QP problems has appeared in the literature. Once the multiparametric problem (7) has been solved o! line, i.e. the solution ;H";H(x(t)) of (7) has been found, the model R predictive controller (3) is available explicitly, as the optimal input u(t) consists simply of the "rst m components of ;H(x(t)) 2.1. MPC computation u(t)"[I 0 2 0];H(x(t)). By substituting x "AIx(t)#I\AHBu , R>IR H R>I\\H Eq. (3) can be rewritten as 1 1 ;H;#x(t)F;, <(x(t))" x(t)>x(t)#min 2 2 3 s.t. G;)=#Ex(t) , (7) where the column vector ;O[u ,2, u S ]31Q, R R>, \ sOmN , is the optimization vector, H"HY0, and S H, F, >, G, =, E are easily obtained from Q, R, and (3) (as only the optimizer ; is needed, the term involving > is usually removed from (7)). The optimization problem (7) is a quadratic program (QP). Since the problem depends on the current state x(t), the implementation of MPC requires the on-line solution of a QP at each time step. Although e$cient QP solvers based on active-set methods and interior point methods are available, computing the input u(t) demands signi"cant on-line computation e!ort. For this reason, the application of MPC has been limited to `slowa and/or `smalla processes. In this paper, we propose a new approach to implement MPC, where the computation e!ort is moved o!-line. The MPC formulation described in Section 2 provides the control action u(t) as a function of x(t) implicitly de"ned by (7). By treating x(t) as a vector of parameters, our goal is to solve (7) o!-line, with respect to all the values of x(t) of interest, and make this dependence explicit. The operations research community has addressed parameter variations in mathematical programs at two levels: sensitivity analysis, which characterizes the change of the solution with respect to small perturbations of the parameters, and parametric programming, where the characterization of the solution for a full range of parameter values is sought. In the jargon of operations research, programs which depend only on one scalar parameter are referred to as parametric programs, while problems depending on a vector of parameters as multi-parametric programs. According to this terminology, (7) is a multiparametric quadratic program (mp-QP). Most of the literature deals with parametric problems, but some authors have addressed the multi-parametric case (8) In Section 4, we will show that the solution ;H(x) of the mp-QP problem is a continuous and piecewise a$ne function of x. Clearly, because of (8), the same properties are inherited by the controller. Section 3 is devoted to investigate the regulation problem over an in"nite prediction horizon N "#R, W leading to solve explicitly the so-called constrained linear quadratic regulation (C-LQR) problem. 3. Piecewise linear solution to the constrained linear quadratic regulation problem In their pioneering work, Sznaier and Damborg (1987), showed that "nite horizon optimization (3), (6), with P satisfying (5), also provides the solution to the in"nitehorizon linear quadratic regulation problem with constraints < (x(0))" min x(t)Qx(t)#u(t)Ru(t) , SS2 R s.t. y )Cx(t))y , u )u(t))u , x(0)"x , x(t#1)"Ax(t)#Bu(t), t*0 (9) with Q, R, A, and B as in (3). The equivalence holds for a certain set of initial conditions, which depends on the length of the "nite horizon. This idea has been reconsidered later by Chmielewski and Manousiouthakis (1996) independently by Scokaert and Rawlings (1998), and recently also in Chisci and Zappa (1999). In Scokaert and Rawlings (1998), the authors extend the idea of We refer to (3) as xnite horizon also in the case N "R. In fact, W such a problem can be transformed into a "nite-horizon problem by choosing P as in (4) or (5). Therefore `"nitenessa should refer to the input and constraint horizons, as these are indeed the parameters which a!ect the complexity of the optimization problem (3). A. Bemporad et al. / Automatica 38 (2002) 3}20 Sznaier and Damborg (1987) by showing that the controller is stabilizing, and that the C-LQR problem is solved by a "nite-dimensional QP problem. Unfortunately, the dimension of the QP depends on the initial state x(0), and no upper-bound on the horizon (and therefore on the QP size) is given. On the other hand, Chmielewski and Manousiouthakis (1996) describe an algorithm which provides a semi-global upper-bound. Namely, for any given compact set of initial conditions, their algorithm provides the horizons N "N such that S the "nite horizon controller (3), (6) solves the in"nite horizon problem (9). We brie#y recall the approach given by Chmielewski and Manousiouthakis (1996) and propose some modi"cations, which will allow us to compute the closed-form of the constrained linear quadratic controller (9) u(t)"f (x(t)), ∀t*0 (10) for a compact set of initial conditions (0), and show that it is piecewise a$ne. Assume that (i) QY0, (ii) the sets , U of feasible states and inputs, respectively, are bounded (e.g. u , u , y , y (R, and C"I) and contain the origin in their interior, and (iii) that for all the initial states x(0)3(0), there exists an input sequence driving the system to the origin under constraints. We also assume that ,U are polytopes, as this is the case at hand in our paper, to simplify the exposition. Denote by P,, problem (3) with NON "N , N "R, K"K (or, equivaS W */ lently, N "N, and P solving the Riccati equation (5b)) W and by P , problem (9). Note that in view of Bellman's principle of optimality (Bellman, 1961), (10) can be reformulated by substituting x ,u for x(t), u(t), R>IR R>I respectively, as predicted and actual trajectories coincide. Chmielewski and Manousiouthakis (1996) prove that, for a given x(0), there exists a "nite N such that P, and P are equivalent, and that the associated controller is exponentially stabilizing. The same result was proved in Scokaert and Rawlings (1998). In addition, by exploiting the convexity and continuity of the value function < with respect to the initial condition x(0), Chmielewski and Manousiouthakis (1996) provide the tools necessary to compute an upper-bound on N for every given set (0) of initial conditions. Such an upper-bound is computed by modifying the algorithm in Chmielewski and Manousiouthakis (1996), as follows. (1) Let XM "x3: K x3U the set of states such that */ the unconstrained LQ gain K is feasible. As ,U */ are polytopes, XM is a polytope, de"ned by a set of linear inequalities of the form AM Gx)bM G, i"1,2, n , where AM ,bM ,n depend on the de"nition of ,U. 7 (2) Compute Z "x31L: xPx)c, where the largest A c such that Z is an invariant subset of XM , is deterA mined analytically (Bemporad, 1998) as bM G c" min . GP\(A A M M G) 2 G L (11) (3) Let q"c (P\Q), where denotes the min imum eigenvalue. (4) Let (0) be a compact set of initial conditions. Without loss of generality, assume that (0) is a polytope, and let x l be the set of its vertices. For each G G x compute < (x ). This is done by computing the G G "nite horizon value function <(x ) for increasing G values of N until x 3Z (this computation is R>I, A equivalent to the one described in Scokaert and Rawlings (1998), where a ball B is used instead of Z ). P A < (x ), which is an upper (5) Let ;"max G2l G bound to < (x) on (0), as < and (0) are convex (in Chmielewski & Manousiouthakis, 1996), a di!erent approach is proposed where ; depends on x ). (6) Choose N as the minimum integer such that N*(;!c)/q. In Section 4, we will show that P, has a continuous and piecewise a$ne solution. Therefore, on a compact set of initial conditions (0), the solution (10) to the CLQR problem P is also piecewise a$ne. Note that if . then the solution is also global because ,U are bounded. 4. Multi-parametric quadratic programming In this section, we investigate multi-parametric quadratic programs (mp-QP) of form (7). We want to derive an algorithm to express the solution ;H(x) and the minimum value <(x)"J(;H(x)) as an explicit function of the parameters x, and to characterize the analytical properties of these functions. In particular, we will prove that the solution ;H(x) is a continuous piecewise a$ne function of x, in the following sense. De5nition 1. A function z(x) : XC1Q, where X-1L is a polyhedral set, is piecewise a$ne if it is possible to partition X into convex polyhedral regions, CR , and G z(x)"HGx#kG, ∀x3CR . G Piecewise quadraticity is de"ned analogously by letting z(x) be a quadratic function x=Gx#HGx#kG. 4.1. Fundamentals of the algorithm Before proceeding further, it is useful to de"ne zO;#H\Fx(t), (12) 8 A. Bemporad et al. / Automatica 38 (2002) 3}20 z31Q, and to transform (7) by completing squares to obtain the equivalent problem 1 < (x)"min zHz X 2 X s.t. Gz)=#Sx(t), (13) where SOE#GH\F, and < (x)"<(x)!x X (>!FH\F)x. In the transformed problem, the parameter vector x appears only on the rhs of the constraints. In order to start solving the mp-QP problem, we need an initial vector x inside the polyhedral set X"x: ¹x)Z of parameters over which we want to solve the problem, such that the QP problem (13) is feasible for x"x . A good choice for x is the center of the largest ball contained in X for which a feasible z exists, determined by solving the LP max VXC s.t. , ¹Gx#¹G)ZG, (14) Gz!Sx)= (in particular, x will be the Chebychev center of X when the QP problem (13) is feasible for such an x ). If )0, then the QP problem (13) is infeasible for all x in the interior of X. Otherwise, we "x x"x and solve the QP problem (13), in order to obtain the corresponding optimal solution z . Such a solution is unique, because HY0, and therefore uniquely determines a set of active constraints GI z "SI x #= I out of the constraints in (13). We can then prove the following result: Theorem 2. Let HY0. Consider a combination of active constraints GI , SI , = I , and assume that the rows of GI are linearly independent. Let CR be the set of all vectors x, for which such a combination is active at the optimum (CR is referred to as critical region). Then, the optimal z and the associated vector of Lagrange multipliers are uniquely dexned azne functions of x over CR . Proof. The "rst-order Karush}Kuhn}Tucker (KKT) optimality conditions (Bazaraa et al., 1993) for the mp-QP are given by Hz#G"0, 31O, (15a) (GGz!=G!SGx)"0, i"1,2, q, G (15b) *0, (15c) Gz)=#Sx, (15d) where the superscript i denotes the ith row. We solve (15a) for z, z"!H\G (16) and substitute the result into (15b) to obtain the complementary slackness condition (!GH\G! =!Sx)"0. Let x and I denote the Lagrange multipliers corresponding to inactive and active constraints, respectively. For inactive constraints, x "0. For active constraints, !GI H\GI I != I !SI x"0, and therefore, I "!(GI H\GI )\(= I #SI x), (17) where GI ,= I ,SI correspond to the set of active constraints, and (GI H\GI )\ exists because the rows of GI are linearly independent. Thus, is an a$ne function of x. We can substitute I from (17) into (16) to obtain z"H\GI (GI H\GI )\(= I #SI x) (18) and note that z is also an a$ne function of x. 䊐 A similar result was obtained by Za"riou (Chiou & Za"riou, 1994; Za"riou, 1990) based on the optimality conditions for QP problems reported in Fletcher (1981). Although his formulation is for "nite impulse response models realized in a particular space form, where the state includes past inputs, his arguments can be adapted directly to the state-space formulation (3)}(6). However, his result does not make the piecewise linear dependence of u on x explicit, as the domains over which the di!erent linear laws are de"ned are not characterized. We show next, that such domains are indeed polyhedral regions of the state space. Theorem 2 characterizes the solution only locally in the neighborhood of a speci"c x , as it does not provide the construction of the set CR where this characteriza tion remains valid. On the other hand, this region can be characterized immediately. The variable z from (16) must satisfy the constraints in (13): GH\GI (GI H\GI )\(= I #SI x))=#Sx (19) and by (15c), the Lagrange multipliers in (17) must remain non-negative: !(GI H\GI )\(= I #SI x)*0, (20) as we vary x. After removing the redundant inequalities from (19) and (20), we obtain a compact representation of CR . Obviously, CR is a polyhedron in the x-space, and represents the largest set of x3X such that the combination of active constraints at the minimizer remains unchanged. Once the critical region CR has been de"ned, the rest of the space CR"X!CR has to be ex plored and new critical regions generated. An e!ective approach for partitioning the rest of the space was proposed in Dua and Pistikopoulos (2000). The following theorem justi"es such a procedure to characterize the rest of the region CR. A. Bemporad et al. / Automatica 38 (2002) 3}20 Theorem 3. Let >-1L be a polyhedron, and CR O x3>: Ax)b a polyhedral subset of >, CR O. Also let AGx'bG R " x3>: , G AHx)bH, ∀j(i i"1,2, m, where m"dim(b), and let CROK R . Then G G (i) CR CR ">, (ii) CR R ", R R ", ∀i G G H Oj, i.e. CR , R ,2, R is a partition of >. K Proof. (i) We want to prove that given an x3>, x either belongs to CR or to R for some i. If x3CR , we are G done. Otherwise, there exists an index i such that AGx'bG. Let iH"min i: AGx'bG. Then, x3R H , beGWK G cause AGHx'bGH and AHx)bH, ∀j(iH, by de"nition of iH. (ii) Let x3CR . Then xi such that AGx'bG, which implies that x,R , ∀i)m. Let x3R and take i'j. Since G G x3R , by de"nition of R (i'j) AHx)bH, which implies G G that x,R . 䊐 H Example 4.1. In order to exemplify the procedure proposed in Theorem 3 for partitioning the set of parameters X, consider the case when only two parameters x and x are present. As shown in Fig. 1(a), X is de"ned by the inequalities x\)x )x>, x\)x )x>, and CR by the inequalities C1)0,2, C5)0 where C1,2, C5 are a$ne functions of x. The procedure consists of considering, one by one, the inequalities which de"ne CR . Considering, for example, the inequality C1)0, the "rst set of the rest of the region 9 CROXCR is given by R "C1*0, x *x\, x\)x )x>, which is obtained by reversing the sign of the inequality C1)0 and removing redundant constraints (see Fig. 1(b)). Thus, by considering the rest of the inequalities, the complete rest of the region is CR" R , where R ,2, R are graphically reG G ported in Fig. 1(d). Note that the partition strategy suggested by Theorem 3 can be also applied also when X is unbounded. Theorem 3 provides a way of partitioning the nonconvex set, XCR , into polyhedral subsets R . For each G R , a new vector x is determined by solving the LP (14), G G and, correspondingly, an optimum z , a set of active G constraints GI G, SI G, = I G, and a critical region CR . Theorem G 3 is then applied to partition R CR into polyhedral G G subsets, and the algorithm proceeds iteratively. The complexity of the algorithm will be fully discussed in Section 4.3. Note that Theorem 3 introduces cuts in the x-space which might split critical regions into subsets. Therefore, after the whole x-space has been covered, those polyhedral regions CRG are determined where the function z(x) is the same. If their union is a convex set, it is computed to permit a more compact description of the solution (Bemporad, Fukuda, & Torrisi, 2001). Alternatively, in Borrelli, Bemporad, and Morari (2000), the authors propose not to intersect (19)}(20) with the partition generated by Theorem 3, and simply use Theorem 3 to guide the exploration. As a result, some critical regions may appear more than once. Duplicates can be easily eliminated by recognizing regions where the combination of active constraints is the same. In the sequel, we will denote by N , the "nal number of polyhedral cells de"n ing the mp-QP solution (i.e., after the union of neighboring cells or removal of duplicates, respectively). 4.1.1. Degeneracy So far, we have assumed that the rows of GI are linearly independent. It can happen, however, that by solving the QP (13), one determines a set of active constraints for which this assumption is violated. For instance, this happens when more than s constraints are active at the optimizer z 31Q, i.e., in a case of primal degeneracy. In this case, the vector of Lagrange multipliers might not be uniquely de"ned, as the dual problem of (13) is not strictly convex (instead, dual degeneracy and non-uniqueness of z cannot occur, as HY0). Let GI 31l"Q, and let r"rank GI , r(l. In order to characterize such a degenerate situation, consider the QR decomposition GI "[0 ]Q of GI , and rewrite the active constraints in the form Fig. 1. Partition of CROXCR ; (b) partition of CR step 1; (c) partition of CR Step 2; and (d) "nal partition of CR. R z"= #S x, (21a) 0"= #S x, (21b) 10 A. Bemporad et al. / Automatica 38 (2002) 3}20 where [1 ]"Q\SI , [5 ]"Q\= I . If S is non-zero, 1 5 because of the equalities (21b), CR is a lower-dimen sional region, which, in general, corresponds to a common boundary between two or more full-dimensional regions. Therefore, it is not worth to explore this combination GI , SI , = I . On the other hand, if both = and S are zero, the KKT conditions do not lead directly to (19) and (20), but only to a polyhedron expressed in the (, x) space. In this case, a full-dimensional critical region can be obtained by a projection algorithm (Fukuda, 1997), which, however, is computationally expensive (the case = O0, S "0 is not possible, since the LP (14) was feasible). In this paper, we suggest a simpler way to handle such a degenerate situation, which consists of collecting r constraints arbitrarily chosen, and proceed with the new reduced set, therefore avoiding the computation of projections. Due to the recursive nature of the algorithm, the remaining other possible subsets of combinations of constraints leading to full-dimensional critical regions will automatically be explored later. 4.2. Continuity and convexity properties Continuity of the value function < (x) and the solution X z(x), can be shown as simple corollaries of the linearity result of Theorem 2. This fact, together with the convexity of the set of feasible parameters X -X (i.e. the set of parameters x3X such that a feasible solution z(x) exists to the optimization problem (13)), and of the value function < (x), is proved in next Theorem. X Theorem 4. Consider the multi-parametric quadratic program (13) and let HY0, X convex. Then the set of feasible parameters X -X is convex, the optimizer z(x) : X C1Q is continuous and piecewise azne, and the optimal solution < (x) : X C1 is continuous, convex and piecewise X quadratic. Proof. We "rst prove convexity of X and < (x). Take X generic x , x 3X , and let < (x ), < (x ) and z , z the X X corresponding optimal values and minimizers. Let 3[0,1], and de"ne z Oz #(1!)z , x Ox # ? ? (1!)x . By feasibility, z , z satisfy the constraints Gz )=#Sx , Gz )=#Sx . These inequalities can be linearly combined to obtain Gz )=#Sx , and ? ? therefore, z is feasible for the optimization problem (13), ? In general, the set of feasible parameters x can be a lower-dimensional subset of X (Filippi, 1997). However, when in the MPC formulation (3), u (u , y (y , the mp-QP problem has always a full-dimensional solution in the x-space, as the critical region corresponding to the unconstrained solution contains a full-dimensional ball around the origin. where x(t)"x . This proves that z(x ) exists, and there? ? fore, convexity of X " CR . In particular, X G G is connected. Moreover, by optimality of < (x ), < (x )) X ? X ? z Hz , and hence < (x )![z Hz #(1!)z Hz ]) ? ? X ? z Hz ![z Hz #(1!)z Hz ]"[z Hz # ? ? (1!)z Hz #2(1!)z Hz !z Hz !(1!)z Hz ]"!(1!)(z !z )H(z !z ))0, i.e. < X (x #(1!)x ))< (x )#(1!)< (x ), X X ∀x , x 3X, ∀3[0,1], which proves the convexity of < (x) on X . Within the closed polyhedral regions CR in X G X , the solution z(x) is a$ne (18). The boundary between two regions belongs to both closed regions. Since HY0, the optimum is unique, and hence, the solution must be continuous across the boundary. The fact that < (x) X is continuous and piecewise quadratic, follows trivially. 䊐 In order to prove the convexity of the value function <(x) of the MPC problems (3) and (7), we need the following lemma. Lemma 1. Let J(;, x)";H;#xF;#x>x, and let >F F _0. Then <(x)Omin J(x, ;) subject to H 3 G;)=#Ex is a convex function of x. Proof. By Theorem 4, the value function < (x) of the X optimization problem < (x)Omin zHz subject to X X Gz)=#Sx is a convex function of x. Let zH(x), ;H(x) be the optimizers of < (x) and <(x), respectively, where X zH(x)";H(x)#H\Fx. Then, <(x)";H(x)H;H(x)# xF;H(x)#x>x"[zH(x)!H\Fx]H[zH(x)!H\ Fx]#xF[zH(x)!H\Fx]#x>x"< (x)!xFH\ X F Fx#x>x"< (x)#x(>!FH\F)x. As > F H X _0, its Schur's complement >!FH\F_0, and therefore, <(x) is a convex function, being the sum of convex functions. 䊐 Corollary 1. The value function <(x) dexned by the optimization problem (3), (7) is continuous and piecewise quadratic. Proof. By (3), J(;, x)*0, ∀x, ;, being the sum of nonY F negative terms. Therefore, J(;, x)"[V] [V]*0 S F H S for all [V], and the proof easily follows by Lemma 1. 䊐 S A simple consequence of Corollary 1 is that the Lyapunov function used to prove Theorem 1 is continuous, convex, and piecewise quadratic. Finally, we can establish the analytical properties of the controller (3), (6) through the following corollary of Theorem 4. A. Bemporad et al. / Automatica 38 (2002) 3}20 Corollary 2. The control law u(t)"f (x(t)), f : 1LC1K, dexned by the optimization problem (3) and (6) is continuous and piece-wise azne 1.5 (22) 1.6 1.7 1.8 where the polyhedral sets HGx)kG, i"1,2, N ) N are a partition of the given set of states X. 1.9 end f (x)"FGx#gG if HGx)kG, i"1,2, N Proof. By (12), ;(x)"z(x)!H\Fx is a linear function of x in each region CR "x : HGx)KG, i"1,2, N . G By (8), u"f (x) is a combination of linear functions, and therefore, is linear on CR . Also, u is a combination of G continuous functions, and therefore, is continuous. 䊐 Multiparametric quadratic programming problems can also be addressed by employing the principles of parametric non-linear programming, exploiting the Basic Sensitivity Theorem (Fiacco, 1976, 1983) (a direct consequence of the KKT conditions and the Implicit Function Theorem). In this paper, we opted for a more direct approach, which exploits the linearity of the constraints and the fact that the function to be minimized is quadratic. We remark that the command actions provided by the (explicit) feedback control law (22) and the (implicit) feedback control law (3)}(6) are exactly equal. Therefore, the control designer is allowed to tune the controller by using standard MPC tools (i.e., based on on-line QP solvers) and software simulation packages, and "nally run Algorithm 1 to obtain the explicit solution (22) to e$ciently implement the controller. 4.3. Ow-line algorithm for mp-QP and explicit MPC Based on the above discussion and results, the main steps of the o!-line mp-QP solver are outlined in the following algorithm. Algorithm 1 1 Let X-1L be the set of parameters (states); 2 execute partition(X); 3 for all regions where z(x) is the same and whose union is a convex set, compute such a union as described by Bemporad, Fukuda, and Torrisi (2001); 4 end. procedure partition(>) 1.1 let x 3> and the solution to the LP (14); 1.2 if )0 then exit; (no full dimensional CR is in >) 1.3 For x"x , compute the optimal solution (z , ) of the QP (13); 1.4 Determine the set of active constraints when z"z , x"x , and build GI , = I , SI ; 11 If r"rank GI is less than the number l of rows of GI , take a subset of r linearly independent rows, and rede"ne GI , = I , SI accordingly; Determine I (x), z(x) from (17) and (18); Characterize the CR from (19) and (20); De"ne and partition the rest of the region as in Theorem 3; For each new sub-region R , partition(R ); G G procedure. The algorithm explores the set X of parameters recursively: Partition the rest of the region as in Theorem 3 into polyhedral sets R , use the same method to partiG tion each set R further, and so on. This can be representG ed as a search tree, with a maximum depth equal to the number of combinations of active constraints (see Section 4.4 below). The algorithm solves the mp-QP problem by partitioning the given parameter set X into N convex poly hedral regions. For the characterization of the MPC controller, in step 3 the union of regions is computed where the "rst N components of the solution z(x) are the S same, by using the algorithm developed by Bemporad, Fukuda, and Torrisi (2001). This reduces the total number of regions in the partition for the MPC controller from N to N . 4.4. Complexity analysis The number N of regions in the mp-QP solution depends on the dimension n of the state, and on the number of degrees of freedom s"mN and constraints S q in the optimization problem (13). As the number of combinations of l constraints out of a set of q is (Ol)"q!/(q!l)!l!, the number of possible combinations of active constraints at the solution of a QP is at most Ol (Ol)"2O. This number represents an upper bound on the number of di!erent linear feedback gains which describe the controller. In practice, far fewer combinations are usually generated as x spans X. Furthermore, the gains for the future input moves u ,2, u S are R> R>, \ not relevant for the control law. Thus, several di!erent combinations of active constraints may lead to the same "rst m components uH(x) of the solution. On the other R hand, the number N of regions of the piecewise a$ne solution is, in general, larger than the number of feedback gains, because non-convex critical regions are split into several convex sets. For instance, the example reported in Fig. 6(d) involves 13 feedback gains, distributed over 57 regions of the state space. A worst-case estimate of N can be computed from the way Algorithm 1 generates critical regions. The "rst critical region CR is de"ned by the constraints (x)*0 (q constraints) and Gz(x))=#Sx (q constraints). If the strict complementary slackness condition holds, only 12 A. Bemporad et al. / Automatica 38 (2002) 3}20 q constraints can be active, and hence, every CR is de"ned by q constraints. From Theorem 3, CR consists of q convex polyhedra R , de"ned by at most q inG equalities. For each R , a new CR is determined which G G consists of 2q inequalities (the additional q inequalities come from the condition CR -R ), and therefore, the G G corresponding CR partition includes 2q sets de"ned by 2q inequalities. As mentioned above, this way of generating regions can be associated with a search tree. By induction, it is easy to prove that at the tree level k#1, there are k!mI regions de"ned by (k#1)q constraints. As observed earlier, each CR is the largest set corresponding to a certain combination of active constraints. Therefore, the search tree has a maximum depth of 2O, as at each level, there is one admissible combination less. In conclusion, the number of regions in the solution to the mp-QP problem is N )O\k!qI, each one de"ned by I at most q2O linear inequalities. Note that the above analysis is largely overestimating the complexity, as it does not take into account: (i) the elimination of redundant constraints when a CR is generated, and (ii) that empty sets are not partitioned further. Table 1 O!-line computation time to solve the mp-QP problem and, in parentheses, number of regions N in the MPC controller (N "num S ber of control moves, n"number of states) N S n"2 n"3 n"4 n"5 2 3 4 0.44 s (7) 1.15 s (13) 2.31 s (21) 0.49 s (7) 2.08 s (15) 5.87 s (29) 0.55 s (7) 1.75 s (15) 3.68 s (29) 1.43 s (7) 5.06 s (17) 15.93 s (37) problems on random SISO plants subject to input constraints. In the comparison, we vary the number of free moves N and the number of states of the open-loop S system. Computation times were evaluated by running Algorithm 1 in Matlab 5.3 on a Pentium III-650 MHz machine. No attempts were made to optimize the e$ciency of the algorithm and its implementation. 4.7. On-line computation time 4.5. Dependence on n, m, p, N , N S Let q Orank S, q )q. For n'q the number of polyQ Q Q hedral regions N remains constant. To see this, consider the linear transformation x "Sx, x 31O. Clearly, x and x de"ne the same set of active constraints, and therefore the number of partitions in the x - and x-space are the same. Therefore, the number of partitions, N , of the x-space de"ning the optimal controller is insensitive to the dimension n of the state x for all n*q , i.e. to the Q number of parameters involved in the mp-QP. In particular, the additional parameters that we will introduce in Section 6 to extend MPC to reference tracking, disturbance rejection, soft constraints, variable constraints, and output feedback, do not a!ect signi"cantly, the number of polyhedral regions N (i.e., the complexity of the mp QP), and hence, the number N of regions in the MPC controller (22). The number q of constraints increases with N and, in the case of input constraints, with N . For instance, S q"2s"2mN for control problems with input conS straints only. From the analysis above, the larger N , N , m, p, the larger q, and therefore N . Note that S many control problems involve input constraints only, and typically horizons N "2,3 or blocking of control S moves are adopted, which reduces the number of constraints q. 4.6. Ow-line computation time In Table 1, we report the computation time and the number of regions obtained by solving a few test MPC The simplest way to implement the piecewise a$ne feedback law (22) is to store the polyhedral cells HGx)kG, perform an on-line linear search through them to locate the one which contains x(t), then lookup the corresponding feedback gain (FG, gG). This procedure can be easily parallelized (while for a QP solver, the parallelization is less obvious). However, more e$cient on-line implementation techniques which avoid the storage and the evaluation of the polyhedral cells are currently under development. 5. State-feedback solution to constrained linear quadratic control For t"0, the explicit solution to (3) provides the optimal input pro"le u(0),2, u(N !1) as a function of W the state x(0). The equivalent state-feedback form u( j)"f (x( j)), j"0,2, N !1 can be obtained by solvH W ing N mp-QPs. In fact, clearly, f (x)"Kx for all S H j"N ,2, N !1, where f : X C1K, and X O1L for S W H H H j"N #1,2, N , X Ox: y )C(A#BK)Fx)y , W H u )K(A#BK)Fx)u , h"0,2, N !j for j" N ,2, N . For 0)j)N !1, f (x) is obtained by solvS S H ing the mp-QP problem F (x)O H min J(;, x)"x(N )Px(N ) W W 3OS H2S,S \ ,W \ # [x(k)Qx(k)#u(k)Ru(k)] IH A. Bemporad et al. / Automatica 38 (2002) 3}20 s.t. y )y(k))y , k"j,2, N , u )u(k))u , k"j,2, N , x( j)"x, x(k#1)"Ax(k)#Bu(k), k*j, 13 sider the MPC problem (23) y(k)"Cx(k), k*j, u(k)"Kx(k), N )k(N S W and setting f (x)"[I 0 2 0]F (x) (note that, compared H H to (3), for j"0, (23) includes the extra constraint y )y(0))y . However, this may only restrict the set of x(0) for which (23) is feasible, but does not change the control function f (x), as u(0) does not a!ect y(0)). Similar to the unconstrained "nite-time LQ problem, where the state-feedback solution is linear timevarying, the explicit state-feedback solution to (3) is the time-varying piecewise a$ne function f : 1KC1L, H j"0,2, N !1. Note that while in the unconstrained W case dynamic programming nicely provides the statefeedback solution through Riccati iterations, because of the constraints here, dynamic programming would lead to solving a sequence of multiparametric piecewise quadratic programs, instead of the mp-QPs (23). The in"nite horizon-constrained linear quadratic regulator can also be obtained in state-feedback form by choosing N "N "N "N, where N is de"ned acS W cording to the results of Section 3. 6. Reference tracking, disturbances, and other extensions The basic formulation (3) can be extended naturally to situations where the control task is more demanding. As long as the control task can be expressed as an mp-QP, a piecewise a$ne controller results, which can be easily designed and implemented. In this section, we will mention only a few extensions to illustrate the potential. To our knowledge, these types of problems are di$cult to formulate from the point of view of anti-windup or other techniques not related to MPC. 6.1. Reference tracking The controller can be extended to provide o!set-free tracking of asymptotically constant reference signals. Future values of the reference trajectory can be taken into account explicitly, by the controller, so that the control action is optimal for the future trajectory in the presence of constraints. Let the goal be to have the output vector y(t) track r(t), where r(t)31N is the reference signal. To this aim, con- ,W \ min [y !r(t)]Q[y !r(t)] R>IR R>IR 3OBSR 2BSR>,S\ I #u Ru R>IR R>IR s.t. y )y )y , k"1,2, N , R>IR u )u )u , k"0,1,2, N , R>I u )u )u , k"0,1,2, N !1, R>I S x "Ax #Bu , k*0, R>I>R R>IR R>I y "Cx , k*0, R>IR R>IR u "u #u , k*0, R>I R>I\ R>I u "0, k*N . R>I S (24) Note that the u-formulation (24) introduces m new states in the predictive model, namely, the last input u(t!1) (this corresponds to adding an integrator in the control loop). Just like the regulation problem (3), we can transform the tracking problem (24) into the form 1 min ;H;#[x(t) u(t!1) r(t)]F; 2 3 x(t) s.t. G;)=#E u(t!1) , (25) r(t) where r(t) lies in a given (possibly unbounded) polyhedral set. Thus, the same mp-QP algorithm can be used to obtain an explicit piecewise a$ne solution u(t)"F(x(t), u(t!1), r(t)). In case the reference r(t) is known in advance, one can replace r(t) by r(t#k) in (24) and similarly get a piecewise a$ne anticipative controller u(t)"F(x(t), u(t!1), r(t),2, r(t#N !1)). W 6.2. Disturbances We distinguish between measured and unmeasured disturbances. Measured disturbances v(t) can be included in the prediction model x "Ax #Bu #<v(t#kt), R>I>R R>IR R>I (26) where v(t#kt) is the prediction of the disturbance at time t#k based on the measured value v(t). Usually, v(t#kt) is a linear function of v(t), for instance v(t#kt),v(t) where it is assumed that the disturbance is constant over the prediction horizon. Then v(t) appears as a vector of additional parameters in the mp-QP, and the piecewise a$ne control law becomes u(t)"F(x(t), v(t)). Alternatively, as for reference tracking, when v(t) is known in advance one can replace v(t#kt) by v(t#k) in (26) and get an anticipative controller u(t)"F(x(t), u(t!1), v(t),2, v(t#N !1)). W 14 A. Bemporad et al. / Automatica 38 (2002) 3}20 Usually unmeasured disturbances are modeled as the output of a linear system driven by a white Gaussian noise. The state vector x(t) of the linear prediction model (1) is augmented by the state x (t) of such a linear disturL bance model, and the mp-QP provides a control law of the form u(t)"F(x(t), x (t)) within a certain range of L states of the plant and of the disturbance model. Clearly, x (t) is estimated on line from output measurements by L a linear observer. 6.3. Soft constraints Fig. 2. Example 7.1: (a) closed-loop MPC; (b) state-space partition and closed-loop MPC trajectories. State and output constraints can lead to feasibility problems. For example, a disturbance may push the output outside the feasible region where no allowed control input may exist which brings the output back inside at the next time step. Therefore, in practice, the output constraints (2) are relaxed or softened (Zheng & Morari, 1995) as y !M)y(t))y #M, where M31N is a constant vector (MG*0 is related to the `concerna for the violation of the ith output constraint), and the term is added to the objective to penalize constraint violations ( is a suitably large scalar). The variable plays the role of an independent optimization variable in the mpQP and is adjoined to z. The solution u(t)"F(x(t)) is again a piecewise a$ne controller, which aims at keeping the states in the constrained region without ever running into feasibility problems. 6.4. Variable constraints The bounds y , y ,u ,u , u , u may change depending on the operating conditions, or in the case of a stuck actuator, the constraints become u "u "0. This possibility can again be built into the control law. The bounds can be treated as parameters in the mp-QP and added to the vector x. The control law will have the form u(t)"F(x(t), y , y ,u ,u , u , u ). 7. Examples 7.1. A simple SISO system Consider the second order system 0.7326 !0.0861 0.1722 y(t)"[0 1.4142]x(t). To this aim, we design an MPC controller based on the optimization problem x Px # [x x # 0.01u ] R>R R>R R>IR R>IR R>I I !2)u )2, k"0,1, (29) R>I x "x(t) RR min SR SR> s.t. where P solves the Lyapunov equation P"APA#Q (in this example, Q"I, R"0.01, N "N "2, N "1). W S Note that this choice of P corresponds to setting u "0 R>I for k*2 and minimize x x #0.01u . R>IR R>IR R>I I 0.9909 x(t)# 0.0609 0.0064 u(t). (27) (30) The MPC controller (29) is globally asymptotically stabilizing. In fact, it is easy to show that the value function is a Lyapunov function of the system. The closed-loop response from the initial condition x(0)"[1 1] is shown in Fig. 2(a). The mp-QP problem associated with the MPC law has the form (7) with sample the dynamics with ¹"0.1 s, and obtain the state-space representation (28) !2)u(t))2. H" 2 y(t)" u(t), s#3s#2 x(t#1)" The task is to regulate the system to the origin while ful"lling the input constraint 1.5064 0.4838 0.4838 1.5258 , F" 9.6652 5.2115 7.0732 !7.0879 , Let ;H"[uH, uH] be the optimal solution at time t. Then R ;"[uH,0] is feasible at time t#1. The cost associated with ; is J(t#1, ;)"J(t, ;H)!x(t)x(t)!0.01u(t)*J(t#1, ;H ), which R R> implies that J(t, ;H) is a converging sequence. Therefore, R x(t)x(t)#0.01u(t))J(t, ;H)!J(t#1, ;H )P0, which shows stabR R> ility of the system. A. Bemporad et al. / Automatica 38 (2002) 3}20 saturated controller, and regions C6 and C9 are transition regions between the unconstrained and the saturated controller. Note that the mp-QP solver provides three di!erent regions C2,C3,C4, although in all of them, u"uH"2. The reason for this is that the R second component of the optimal solution, uH , is di!erR> ent, in that uH "[!3.4155 4.6452]x(t)!0.6341 in reR> gion C2, uH "2 in region C3, and uH "!2 in R> R> region C4. Moreover, note that regions C2 and C4 are joined, as their union is a convex set, but the same cannot 1 0 !1 0 G" 0 1 0 !1 2 , =" 2 2 0 0 , E" 2 0 0 . 0 0 0 0 CFFFFFFFFFFFFFFFFFFFFFDFFFFFFFFFFFFFFFFFFFFFE The solution was computed by Algorithm 1 in 0.66 s (15 regions examined), and the corresponding polyhedral partition of the state-space into N "9 polyhedral cells is depicted in Fig. 2(b). The MPC law is [!5.9220 !6.8883]x 5.9220 6.8883 x) !1.5379 6.8291 1.5379 2.0000 , 2.0000 (Region C2, C4) !3.4155 4.6452 if 0.1044 0.1259 if 2.0000 2.0000 !6.8291 (Region C1) 2.0000 2.0000 !5.9220 !6.8883 if 15 2.6341 0.1215 x) !0.0353 , 0.0922 !0.0267 0.0679 !0.0924 0.1259 0.0922 !0.0524 x) , !0.0519 (Region C3) !0.1259 !0.0922 !2.0000 if u" !0.0679 0.0924 !0.0519 x) , !0.0524 (Region C5) [!6.4159 !4.6953]x#0.6423 if !0.0275 0.1220 6.4159 4.6953 (Region C6) !2.0000 [!6.4159 !4.6953]x!0.6423 !6.4159 !4.6953 3.4155 !4.6452 1.3577 x) !0.0357 , 2.6423 2.6341 if !0.1044 !0.1215 x) !0.0353 , !0.1259 !0.0922 (Region C7, C8) 6.4159 if 0.0275 4.6953 1.3577 !0.1220 x) !0.0357 !6.4159 !4.6953 (Region C9) and consists of N "7 regions. Region C1 corres ponds to the unconstrained linear controller, regions C2, C3, C4 and C5, C7, C8 correspond to the !0.0267 2.6423 be done with region C3, as their union would not be a convex set, and therefore cannot be expressed as one set of linear inequalities. 16 A. Bemporad et al. / Automatica 38 (2002) 3}20 Fig. 3. Example 7.1. Additional constraint x *!0.5: (a) closedR>IR loop MPC; (b) state-space partition and closed-loop MPC trajectories. The same example is repeated with the additional state constraint x *x , R>IR !0.5 x O , !0.5 k"1. The closed-loop behavior from the initial condition x(0)"[1 1] is depicted in Fig. 3(a). The MPC controller was computed in 0.99 s (22 regions examined). The polyhedral partition of the state space corresponding to the modi"ed MPC controller is depicted in Fig. 3(b). The partition provided by the mp-QP algorithm consists now of N "11 regions (as regions C1, C2 and C3, C9 can be joined, the MPC controller consists of N "9 regions). Note that there are feasible states smaller than x , and vice versa, infeasible states x*x . This is not surprising. For instance, the initial state x(0)"[!0.6,0] is feasible for the MPC controller (which checks state constraints at time t#k, k"1), because there exists a feasible input such that x(1) is within the limits. On the contrary, for x(0)"[!0.47,!0.47] no feasible input is able to produce a feasible x(1). Moreover, the union of the regions depicted in Fig. 3(b) should not be confused with the region of attraction of the MPC closed-loop. For instance, by starting at x(0)"[46.0829,!7.0175] (for which a feasible solution exists), the MPC controller runs into infeasibility after t"9 time steps. 7.2. Reference tracking for a MIMO system Consider the plant (31) which was studied by Mulder, Kothare, and Morari (1999) and Zheng, Kothare and Morari (1994) and by other authors as an example for anti-windup control synthesis. The input u(t) is subject to the saturation constraints i"1, 2. Mulder et al. (1999) use a decoupler and two identical PI controllers 1 K(s)" 1# 100s 2 2.5 1.5 2 . For the set-point change r"[0.63,0.79] they show that very large oscillations result during the transient when the output of the PI controller saturates. We sample the dynamics (31) with ¹"2 s, and design an MPC law (24) with N "20, N "1, N "0, W S Q"I, R"0.1I. The closed-loop behavior starting from zero initial conditions is depicted in Fig. 4(a). It is similar to the result reported by Mulder et al. (1999), where an anti-windup scheme is used on top of the linear controller K(s). The mp-QP problem associated with the MPC law has the form (7) with two optimization variables (two inputs over the one-step control horizon), and six parameters (two states of the original system, two states to memorize the last input u(t!¹ ), and two reference signals), Q with H" 0.7578 !0.9699 !0.9699 1.2428 , F" 4 !5 10 u(t), y(t)" 100s#1 !3 4 !1)u (t))1, G Fig. 4. MIMO example: (a) closed-loop MPC: output y(t) (left), input u(t) (right); (b) state-space partition obtained by setting u"[0 0] and r"[0.63 0.79]. 0.1111 !0.1422 !0.0711 0.0911 0.7577 !0.9699 !0.9699 1.2426 !0.1010 0.1262 0.0757 !0.1010 , G" 1 0 !1 0 0 1 0 !1 , =" 1 1 1 1 , A. Bemporad et al. / Automatica 38 (2002) 3}20 0 0 !1 E" 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 !1 0 0 . The solution was computed in 1.15 s (13 regions examined) with Algorithm 1, and the explicit MPC controller is de"ned over N "9 regions. A section of the x-space of the piecewise a$ne solution obtained by setting u" and r" is depicted in Fig. 4(b). The solution can be interpreted as follows. Region C1 corresponds to the unconstrained optimal controller. Regions C4,C5,C6,C9 correspond to the saturation of both inputs. For instance in region C4 the optimal input variation u(t)"!u(t!1). In regions C2,C3,C7,C8 only one component of the input vector saturates. Note that the component which does not saturate depends linearly on the state x(t), past input u(t!1), and the reference r(t), but with di!erent gains than the unconstrained controller of region C1. Thus, the optimal piecewise a$ne controller is not just the simple saturated version of a linear controller. In summary, for this example, the mp-QP solver determines: (i) a two-degree of freedom optimal controller for this MIMO system (31); and (ii) a set-point dependent optimal anti-windup scheme, a nontrivial task as the regions in the (x, r, u)-space where the controller should be switched must be determined. We stress the fact that optimality refers precisely to the design performance requirement (24). 7.3. Inxnite horizon LQR for the double integrator Consider the double integrator 1 y(t)" u(t) s Fig. 5. Double integrator example (N "2, N "7, computation S time 0.77 s): (a) closed-loop MPC; (b) polyhedral partition of the statespace and closed-loop MPC trajectories. This task is addressed by using the MPC algorithm (3) where N "2, N "2, W S 1 0 Q" , 0 0 R"0.01, and K, P solve the Riccati equation (5). For a certain set of initial conditions (0), this choice of P corresponds to setting u "K x " R>I */ R>IR [!0.81662 !1.7499]x and minimizes R>IR 1 y y # u . (35) R>IR R>IR 10 R>I I On (0), the MPC controller coincides with the constrained linear quadratic regulator, and is therefore stabilizing. Its domain of attraction is, however, larger than (0). We test the closed-loop behavior from the initial condition x(0)"[10!5], which is depicted in Fig. 5(a). The mp-QP problem associated with the MPC law has form (7) with H" and its equivalent discrete-time state-space representation x(t#1)" 1 1 0 1 x(t)# 0 1 (32) obtained by setting yK (t)+(y (t#¹)!y (t))/¹, y (t)+ (y(t#¹)!y(t))/¹, ¹"1 s. We want to regulate the system to the origin while minimizing the quadratic performance measure 1 y(t)y(t)# u(t) 10 R subject to the input constraint (33) (34) 1.6730 0.7207 0.7207 0.4118 , F" 1 0 !1 0 2.5703 1.0570 0 1 0 !1 1 , =" 1 1 1 0.9248 0.3363 G" u(t), y(t)"[1 0]x(t), !1)u(t))1. 17 , 0 0 , E" 0 0 0 0 0 0 and was computed in 0.77 s (16 regions examined). The corresponding polyhedral partition of the state-space is depicted in Fig. 5(b). The same example was solved by increasing the number of degrees of freedom N . The corresponding partiS tions, computation times, and number of regions are reported in Fig. 6. Note that by increasing the number of free control moves N , the control law only changes far S away from the origin, the more in the periphery the larger N . This must be expected from the results of Section 3, S as the set where the MPC law approximates the 18 A. Bemporad et al. / Automatica 38 (2002) 3}20 Fig. 6. Partition of the state space for the MPC controller: N "numS ber of control degrees of freedom, N "number of polyhedral re gions in the controller, o!-line computation time: (a) N "3, S N "15, CPU time: 2.63 s; (b) N "4, N "25, CPU time: 560 s; S (c) N "5, N "39, CPU time: 9.01 s; (d) N "6, N "57, CPU S S time: 16.48 s. constrained in"nite-horizon linear quadratic regulation (C-LQR) problem gets larger when N increases S (Chmielewski & Manousiouthakis, 1996; Scokaert & Rawlings, 1998). By the same arguments, the exact piecewise a$ne solution of the C-LQR problem can be obtained for any set of initial conditions by choosing the "nite horizon as outlined in Section 3. Preliminary ideas about the shape of the constrained linear quadratic regulator for the double integrator were presented in Tan (1991), and are in full agreement with our results. By extrapolating the plots in Fig. 6, where the band of unsaturated control actions is partitioned in 2N !1 sets, one may conjecture that as N PR, the S S number of regions of the state-space partition tends to in"nity as well. Note that although the band of unsaturated control may shrink asymptotically as xPR, it cannot disappear. In fact, such a gap is needed to ensure the continuity of the controller proved in Corollary 2. The fact that more degrees of freedom are needed as the state gets larger in order to preserve LQR optimality is also observed by Scokaert and Rawlings (1998), where for the state-space realization of the double integrator chosen by the authors the state x"[20,20] requires N "33 degrees of freedom. S 8. Conclusions We showed that the linear quadratic optimal controller for constrained systems is piece-wise a$ne and we provided an e$cient algorithm to determine its parameters. The controller inherits all the stability and performance properties of model predictive control (MPC) but can be implemented without any involved on-line computations. The new technique is not intended to replace MPC, especially not in some of the larger applications (systems with more than 50 inputs and 150 outputs have been reported from industry). It is expected to enlarge its scope of applicability to situations which cannot be covered satisfactorily with anti-windup schemes or where the on-line computations required for MPC are prohibitive for technical or cost reasons, such as those arising in the automotive and aerospace industries. The decision between on-line and o!-line computations must be related also to a tradeo! between CPU (for computing QP) and memory (for storing the explicit solution). Moreover, the explicit form of the MPC controller allows to better understand the control action, and to analyze its performance and stability properties (Bemporad, Torrisi, & Morari, 2000). Current research is devoted to develop on-line implementation techniques which do not require the storage of the polyhedral cells (Borrelli, Baotic, Bemporad, & Morari, 2001), and to develop suboptimal methods that allow one to trade o! between performance loss and controller complexity (Bemporad & Filippi, 2001). All the results in this paper can be extended easily to 1-norm and R-norm objective functions instead of the 2-norm employed in here (Bemporad, Borrelli, & Morari, 2000). The resulting multiparametric linear program can be solved in a similar manner as suggested by Borrelli et al. (2000) or by Gal (1995). For MPC of hybrid systems, an extension involving multiparametric mixed-integer linear programming is also possible (Bemporad, Borrelli, & Morari, 2000; Dua & Pistikopoulos, 2000). Finally, we note that the semi-global stabilization problem for discrete-time constrained systems with multiple poles on the unit circle which has received much attention since the early paper by Sontag (1984) can be addressed in a completely general manner in the proposed framework. Acknowledgements This research was supported by the Swiss National Science Foundation. We wish to thank Nikolaos A. Bozinis for his help with the initial implementation of the algorithm, and Francesco Borrelli and Ali H. Sayed for suggesting improvements to the original manuscript. References Acevedo, J., & Pistikopoulos, E. N. (1999). An algorithm for multiparametric mixed-integer linear programming problem. Operations Research Letters, 24, 139}148. A. Bemporad et al. / Automatica 38 (2002) 3}20 Bazaraa, M. S., Sherali, H. D., & Shetty, C. M. (1993). Nonlinear programming*theory and algorithms (2nd ed.). New York: Wiley. Bellman, R. (1961). Adaptive control processes*a guided tour. Princeton, NJ: Princeton University Press. Bemporad, A. (1998). A predictive controller with arti"cial Lyapunov function for linear systems with input/state constraints. Automatica, 34(10), 1255}1260. Bemporad, A. (1998). Reducing conservativeness in predictive control of constrained systems with disturbances. Proceedings of the 37th IEEE Conference on Decision and Control, Tampa, FL (pp. 1384}1391). Bemporad, A., Borrelli, F., & Morari, M. (2000). Explicit solution of LP-based model predictive control. Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, Australia, December. Bemporad, A., Borrelli, F., & Morari, M. (2000). Piecewise linear optimal controllers for hybrid systems. Proceedings of the American Control Conference, Chicago, IL. Bemporad, A., Chisci, L., & Mosca, E. (1994). On the stabilizing property of SIORHC.. Automatica, 30(12), 2013}2015. Bemporad, A., & Filippi, C. (2001), Suboptimal explicit MPC via approximate multiparametric quadratic programming. Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, Florida. Bemporad, A., Fukuda, K., & Torrisi, F. D. (2001). Convexity recognition of the union of polyhedra. Computational Geometry, 18, 141}154. Bemporad, A., & Morari, M. (1999). Robust model predictive control: A survey. In A. Garulli, A. Tesi, & A. Vicino (Eds.), Robustness in identixcation and control, Lecture Notes in Control and Information Sciences Vol. 245 (pp. 207}226). Berlin: Springer. Bemporad, A., Torrisi, F. D., & Morari, M. (2000). Performance analysis of piecewise linear systems and model predictive control systems. Proceedings of the 39th IEEE Conference on Decision and Control, Sydney, Australia, December. Blanchini, F. (1994). Ultimate boundedness control for uncertain discrete-time systems via set-induced Lyapunov functions. IEEE Transactions on Automatic Control, 39(2), 428}433. Borrelli, F., Baotic, M., Bemporad, A., & Morari, M. (2001). E$cient on-line computation of closed-form constrained optimal control laws. Proceedings of the 40th IEEE Conference on Decision and Control, Orlando, Florida. Borrelli, F., Bemporad, A., & Morari, M. (2000). A geometric algorithm for multi-parametric linear programming. Technical Report AUT00-06, Automatic Control Laboratory, ETH Zurich, Switzerland, February 2000. Chiou, H. W., & Za"riou, E. (1994). Frequency domain design of robustly stable constrained model predictive controllers. Proceedings of the American Control Conference, Vol. 3 (pp. 2852}2856). Chisci, L., & Zappa, G. (1999). Fast algorithm for a constrained in"nite horizon LQ problem. International Journal of Control, 72(11), 1020}1026. Chmielewski, D., & Manousiouthakis, V. (1996). On constrained in"nite-time linear quadratic optimal control. Systems and Control Letters, 29(3), 121}130. Dua, V., & Pistikopoulos, E. N. (1999). Algorithms for the solution of multiparametric mixed-integer nonlinear optimization problems. Industrial Engineering Chemistry Research, 38(10), 3976}3987. Dua, V., & Pistikopoulos, E. N. (2000). An algorithm for the solution of multiparametric mixed integer linear programming problems. Annals of Operations Research, 99, 123}139. Fiacco, A. V. (1976). Sensitivity analysis for nonlinear programming using penalty methods. Mathematical Programming, 10(3), 287}311. Fiacco, A. V. (1983). Introduction to sensitivity and stability analysis in nonlinear programming. London, UK: Academic Press. Filippi, C. (1997). On the geometry of optimal partition sets in multiparametric linear programming. Technical report 12, Department of Pure and Applied Mathematics, University of Padova, Italy, June. 19 Fletcher, R. (1981). Practical methods of optimization; Vol. 2: Constrained optimization. New York: Wiley. Fukuda, K. (1997). cdd/cdd# reference manual, (0.61 (cdd) 0.75 (cdd#), ed.). Zurich, Switzerland: Institute for Operations Research. Gal, T. (1995). Postoptimal analyses, parametric programming, and related topics (2nd ed.). Berlin: de Gruyter. Gilbert, E. G., & Tin Tan, K. (1991). Linear systems with state and control constraints: The theory and applications of maximal output admissible sets. IEEE Transactions on Automatic Control, 36(9), 1008}1020. Gutman, P. O. (1986). A linear programming regulator applied to hydroelectric reservoir level control. Automatica, 22(5), 533}541. Gutman, P. O., & Cwikel, M. (1986). Admissible sets and feedback control for discrete-time linear dynamical systems with bounded control and states. IEEE Transactions on Automatic Control, AC31(4), 373}376. Johansen, T. A., Petersen, I., & Slupphaug, O. (2000). On explicit suboptimal LQR with state and input constraints. Proceedings of the 39th IEEE Conference on Decision and Control (pp. 662}667). Sydney, Australia, December. Keerthi, S. S., & Gilbert, E. G. (1988). Optimal in"nite-horizon feedback control laws for a general class of constrained discrete-time systems: stability and moving-horizon approximations. Journal of Optical Theory and Applications, 57, 265}293. Kerrigan, E. C., & Maciejowski, J. M. (2000). Invariant sets for constrained nonlinear discrete-time systems with application to feasibility in model predictive control. Proceedings of the 39th IEEE Conference on Decision and Control. Kothare, M. V., Campo, P. J., Morari, M., & Nett, C. N. (1994). A uni"ed framework for the study of anti-windup designs. Automatica, 30(12), 1869}1883. Kothare, M. V., & Morari, M. (1999). Multiplier theory for stability analysis of anti-windup control systems. Automatica, 35, 917}928. Mulder, E. F., Kothare, M. V., & Morari, M. (1999). Multivariable anti-windup controller synthesis using iterative linear matrix inequalities. Proceedings of the European Control Conference. Qin, S. J., & Badgwell, T. A. (1997). An overview of industrial model predictive control technology. In Chemical process control*V, Vol. 93, No. 316 (pp. 232}256). AIChe Symposium Series*American Institute of Chemical Engineers. Rawlings, J. B., & Muske, K. R. (1993). The stability of constrained receding-horizon control. IEEE Transactions on Automatic Control, 38, 1512}1516. Scokaert, P. O. M., & Rawlings, J. B. (1998). Constrained linear quadratic regulation. IEEE Transactions on Automatic Control, 43(8), 1163}1169. Sontag, E. D. (1984). An algebraic approach to bounded controllability of linear systems. International Journal of Control, 39(1), 181}188. Sznaier, M., & Damborg, M. J. (1987) Suboptimal control of linear systems with state and control inequality constraints. Proceedings of the 26th IEEE Conference on Decision and Control, Vol. 1 (pp. 761}762). Tan, K. T. (1991). Maximal output admisible sets and the nonlinear control of linear discrete-time systems with state and control constraints. Ph.D. thesis, University of Michigan. Teel, A., & Kapoor, N. (1997). The L anti-windup problem: its de"nition and solution. European Control Conference, Brussels, Belgium. Za"riou, E. (1990). Robust model predictive control of processes with hard constraints. Computers and Chemical Engineering, 14(4/5), 359}371. Zheng, A., Kothare, M. V., & Morari, M. (1994). Anti-windup design for internal model control. International Journal of Control, 60(5), 1015}1024. Zheng, A., & Morari, M. (1995). Stability of model predictive control with mixed constraints. IEEE Transactions on Automatic Control, 40, 1818}1823. 20 A. Bemporad et al. / Automatica 38 (2002) 3}20 of the AIChE and was elected to the National Academy of Engineering (U.S.). Professor Morari has held appointments with Exxon R&E and ICI and has consulted internationally for a number of major corporations. Alberto Bemporad received the master degree in Electrical Engineering in 1993 and the Ph.D. in Control Engineering in 1997 from the University of Florence, Italy. He spent the academic year 1996/97 at the Center for Robotics and Automation, Dept. Systems Science & Mathematics, Washington University, St. Louis, as a visiting researcher. In 1997-1999, he held a postdoctoral position at the Automatic Control Lab, ETH, Zurich, Switzerland, where he is currently a$liated as a senior researcher. Since 1999, he is assistant professor at the University of Siena, Italy. He received the IEEE Center and South Italy section `G. Barzilaia and the AEI (Italian Electrical Association) `R. Mariania awards. He has published papers in the area of hybrid systems, model predictive control, computational geometry, and robotics. He is involved in the development of the Model Predictive Control Toolbox for Matlab. Since 2001, he is an Associate Editor of the IEEE Transactions on Automatic Control. Vivek Dua is a Research Associate at the Centre for Process Systems Engineering, Imperial College. He obtained B.E.(Honours) in Chemical Engineering from Panjab University, Chandigarh, India in 1993 and M.Tech. in Chemical Engineering from the Indian Institute of Technology, Kanpur in 1995. He joined Kinetics Technology India Ltd. as a Process Engineer in 1995 and then Imperial College in 1996 where he obtained PhD in Chemical Engineering in 2000. His research interests are in the areas of mathematical programming and its application in process systems engineering. Manfred Morari was appointed head of the Automatic Control Laboratory at the Swiss Federal Institute of Technology (ETH) in Zurich, in 1994. Before that he was the McCollum-Corcoran Professor and Executive O$cer for Control and Dynamical Systems at the California Institute of Technology. He obtained the diploma from ETH Zurich and the Ph.D. from the University of Minnesota. His interests are in hybrid systems and the control of biomedical systems. In recognition of his research he received numerous awards, among them the Eckman Award of the AACC, the Colburn Award and the Professional Progress Award Stratos Pistikopoulos is a Professor in the Department of Chemical Engineering at Imperial College. He obtained a Diploma in Chemical Engineering from the Aristotle University of Thessaloniki, Greece in 1984 and his PhD in Chemical Engineering from Carnegie Mellon University, USA in 1988. His research interests include the development of theory, algorithms and computational tools for continuous and integer parametric programming. He has authored or co-authored over 150 research publications in the area of optimization and process systems engineering applications.
* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project