Terminal design and MPC stability

A terminal set provides somewhere the controller can continue safely. A terminal cost provides an accounting rule for the cost of that continuation. When the two are compatible, the finite optimization value decreases after every executed action, even though the next optimizer may discard the old plan and choose a different one.

The central argument of this chapter is a comparison with a feasible shifted plan. It proves recursive feasibility first and convergence second. We then construct terminal ingredients for the cart, derive a small explicit MPC law, and explain why tracking a nonzero reference requires a different objective from regulating to zero.

Grounded in AA203 lecture 12, including the preceding lecture's deferred feasibility-lemma proof. The full sequence covers feasibility, Lyapunov stability, terminal design, explicit MPC, and input-increment tracking. The local quadratic bounds, boundedness counterexample, cart ellipsoid, and scalar explicit controller are teaching completions and original examples. The official slides add the backward induction through truncated feasible sets, the general Lyapunov theorem and the labeled assumptions of the stability theorem, maximal positively invariant terminal sets and the Riccati equation, the constrained-LQR problem and the piecewise-affine form of its explicit solution, the design recipe with the horizon alternative to a terminal constraint, and the full input-increment tracking problem; where their notation differs the text translates it, and where a statement needs further hypotheses the text supplies them.

The truncated feasible set explains the first feasibility lemma 3:20

Retain the nominal discrete model \(x^+=Ax+Bu\), admissible sets \(X,U\), horizon \(N\geq1\), and terminal set \(X_f\subseteq X\). Let \(X_0\) contain states with a feasible \(N\)-step sequence, and let \(X_1\) contain states with a feasible \(N-1\)-step sequence ending in \(X_f\). For \(N=1\), take \(X_1=X_f\). The one-step predecessor is \(\operatorname{Pre}(S)=\{x:\exists u\in U,\ Ax+Bu\in S\}\).

The definitions imply \(X_0=X\cap\operatorname{Pre}(X_1)\): a full feasible plan consists of an admissible first transition followed by a feasible truncated plan. If \(X_1\) is control invariant, every state in it belongs to \(X\) and has a successor in \(X_1\). Consequently

Start in \(x\in X_0\) and execute the first input of any feasible plan. Its successor \(x^+\) belongs to \(X_1\), since the remainder of that plan is a feasible \(N-1\)-step sequence. The inclusion then gives \(x^+\in X_0\). Induction proves feasibility at every subsequent time, provided each executed first input comes from a feasible plan and the actual successor agrees with the nominal model.

Optimality was not used. Recursive feasibility is fundamentally a statement about available continuation, not about the quality of the selected cost. The lemma is nevertheless awkward as a design recipe because \(X_1\) is an induced set, not an independent parameter we directly choose.

An invariant terminal set makes every feasible plan extendable 11:34

Suppose \(X_f\) is control invariant: for every \(z\in X_f\), some admissible terminal action \(\kappa_f(z)\) satisfies \(Az+B\kappa_f(z)\in X_f\). The lecture proves recursive feasibility by propagating invariance backward through truncated feasible sets. The following equivalent shifted-sequence proof makes the implementation connection explicit.

Let a feasible plan from \(x\) have controls \(u_0,\ldots,u_{N-1}\) and predicted states \(x_0=x,x_1,\ldots,x_N\), with \(x_N\in X_f\). Execute \(u_0\). At the next state \(x_1\), consider the length-\(N\) candidate

The first \(N-1\) transitions reproduce the previously feasible suffix. The last action is admissible and returns the successor to \(X_f\). Since \(X_f\subseteq X\), the old terminal state also satisfies its new role as an intermediate state. Every constraint in the next optimization is therefore satisfied. The optimizer may improve this candidate, but a feasible point already exists.

The backward route reaches the same conclusion through the lemma of the previous section. Index the truncated feasible sets by the stage at which they apply: \(X_N=X_f\), and for \(j=N-1,\ldots,0\) let \(X_j\) contain the states \(x\in X\) from which some admissible \((N-j)\)-step input sequence keeps every intermediate state in \(X\) and ends in \(X_f\). Equivalently, \(X_j=X\cap\operatorname{Pre}(X_{j+1})\). This indexing agrees with the earlier definitions of \(X_0\) and \(X_1\). The subscript counts remaining steps and labels a set, not a time; a predicted state \(x_N\) can belong to \(X_{N-1}\) without any conflict between the two indices.

The first step concerns the one-step set \(X_{N-1}=\{x\in X:\exists u\in U,\ Ax+Bu\in X_f\}\). Take any \(z\in X_f\). Control invariance supplies \(u\in U\) with \(Az+Bu\in X_f\), and the inclusion \(X_f\subseteq X\) supplies \(z\in X\). These are exactly the two membership conditions of \(X_{N-1}\), so \(X_f\subseteq X_{N-1}\). This inclusion makes \(X_{N-1}\) control invariant: from \(x\in X_{N-1}\) an admissible input reaches \(X_f\), and \(X_f\) lies inside \(X_{N-1}\). The hypothesis \(X_f\subseteq X\) carries real weight here. Without it, a terminal state outside \(X\) could not serve as the initial state of a feasible one-step problem.

The same argument repeats with \(X_{j+1}\) in the role of \(X_f\). If \(X_{j+1}\) is control invariant, then, because it is contained in \(X\), \(X_{j+1}\subseteq X\cap\operatorname{Pre}(X_{j+1})=X_j\). Every state of \(X_j\) then has an admissible successor in \(X_{j+1}\subseteq X_j\), so \(X_j\) is control invariant as well. Induction from \(j=N-1\) down to \(j=0\) yields the nested chain

in which every set is control invariant. Control invariance of \(X_1\) is the hypothesis of the lemma, which then gives persistent feasibility. The chain also has a direct reading. With an invariant terminal set, adding a step to the horizon never removes a feasible initial state, because the extra step can be appended at the end, where an invariance-preserving action keeps the state in \(X_f\). For the integrator example later in this section, with \(|u|\leq0.5\), \(X_f=[-r_f,r_f]\), and no state constraints, \(X_j=[-(r_f+(N-j)/2),\,r_f+(N-j)/2]\), and the chain is visible as a sequence of widening intervals.

This proof identifies its own failure boundaries. An unmodeled disturbance makes the measured successor differ from \(x_1\), so the old suffix need not be feasible. A time-varying obstacle invalidates the reuse of yesterday's constraints. A numerical solver returning a constraint-violating action breaks the premise. A nominal terminal invariant set does not, by itself, repair these changes.

The singleton \(X_f=\{0\}\) works when zero input is admissible and the origin is an equilibrium. But forcing an exact zero in \(N\) steps can make the feasible region smaller and increase effort. For an integrator with \(|u|\leq0.5\), terminal equality gives the \(N\)-step feasible interval \([-N/2,N/2]\), before state constraints. A terminal interval of radius \(r_f\) extends it to radius \(r_f+N/2\). A larger suitable terminal region can accommodate more initial states, although it must remain compatible with the terminal control and cost.

The terminal set is an artificial constraint. It expresses no physical requirement of the task; it is introduced solely because it yields a sufficient condition for persistent feasibility. That origin explains why a large terminal set is preferred. Adding the constraint \(x_N\in X_f\) can only remove candidate plans, so for a fixed objective the feasible region can only shrink and the optimal value can only rise. Conversely, replacing \(X_f\) by a larger set \(X_f'\supseteq X_f\) that is still control invariant and contained in \(X\) keeps every previously feasible plan feasible, so \(X_0\) cannot shrink and, with the same terminal cost, \(V_N\), the optimal value defined in the decrease section, cannot increase. The integrator numbers quantify the feasible-region effect: radius \(N/2\) for \(X_f=\{0\}\) against \(r_f+N/2\) for an interval. Any enlargement must be accompanied by a terminal cost that remains valid on the larger set, which the terminal-design section makes concrete.

A decreasing value becomes a stability argument only with bounds 23:10

Persistent feasibility does not make the closed loop converge to the desired equilibrium. An original scalar example isolates the gap. Take \(x^+=x+u\) with \(|u|\leq0.5\), \(|x|\leq2\), horizon \(N=1\), terminal set \(X_f=[-1,1]\), stage cost \(x^2+u^2\), and no terminal cost. The terminal set is control invariant, since \(u=0\) holds every point of it in place, so the feasibility theorem applies. But the current state cost \(x^2\) is fixed before the input is chosen, so for \(|x|\leq1\) the optimizer minimizes \(u^2\) subject to \(|x+u|\leq1\) and returns \(u=0\). Every point of \([-1,1]\) is then an equilibrium of the closed loop, and a state starting at \(x=1\) stays there forever. One of the most widely used constructions therefore assigns separate ingredients to the two properties: a control invariant terminal set for feasibility and a terminal cost \(p_f\) for stability. With \(p_f=0\), the terminal inequality of the decrease section reduces to \(x^2+u^2\leq0\), which fails at every nonzero state of \(X_f\). The tool that turns a suitable terminal cost into a convergence proof is Lyapunov stability theory.

For a discrete autonomous system \(x^+=f_{\rm cl}(x)\), an equilibrium satisfies \(f_{\rm cl}(0)=0\). Lyapunov stability means sufficiently small initial deviations remain small for all future times. Attraction means trajectories converge to the equilibrium. Asymptotic stability combines the two. A function called a Lyapunov function must establish these properties through positivity, appropriate local bounds, and decrease along actual closed-loop transitions.

A convenient quantitative version uses positive constants \(a,b,c\) and a function \(V\) satisfying, near the origin,

The lower bound prevents a large state from hiding in a tiny value. The upper bound says a small state starts with a small value. Decrease ensures the trajectory cannot leave the corresponding value sublevel set. Summing the decrease inequality gives \(c\sum_{t=0}^{M-1}\|x(t)\|^2\leq V(x(0))\), so the state norm tends to zero if feasibility and the inequality continue indefinitely.

The lecture introduces this reasoning with a dissipating-energy analogy. An optimization value need not be physical energy; its usefulness comes from the proved inequalities. Merely observing that a cost decreases on a plotted simulation does not establish decrease for every feasible initial condition.

The quadratic version is a special case of a theorem that needs no quadratic bounds. Let \(\Omega\subset\mathbb R^n\) be closed, bounded, and positively invariant for \(x^+=f_{\rm cl}(x)\), with \(0\in\Omega\). Let \(V:\mathbb R^n\to\mathbb R\) be continuous at the origin, with \(V(0)=0\) and \(V(x)\gt0\) for \(x\in\Omega\setminus\{0\}\). Suppose a continuous function \(\alpha\), with \(\alpha(0)=0\) and \(\alpha(x)\gt0\) for \(x\neq0\), bounds the decrease from below:

Then the origin is asymptotically stable in \(\Omega\): it is Lyapunov stable, and every trajectory starting in \(\Omega\) converges to it. For stability, fix \(\varepsilon\gt0\) and let \(m_\varepsilon\gt0\) be the minimum of \(\alpha\) over the compact set \(\{x\in\Omega:\|x\|\geq\varepsilon\}\); if that set is empty there is nothing to prove. Continuity of \(V\) at the origin gives \(\delta\gt0\) with \(V(x)\lt m_\varepsilon\) whenever \(\|x\|\lt\delta\). Along a trajectory that starts in \(\Omega\) within \(\delta\) of the origin, \(V\) never increases, so \(V(x(t))\lt m_\varepsilon\) for all \(t\). If some state had \(\|x(t)\|\geq\varepsilon\), then \(V(x(t))\geq V(x(t))-V(x(t+1))\geq\alpha(x(t))\geq m_\varepsilon\), using \(V\geq0\) on \(\Omega\), which is a contradiction. For attraction, summing the decrease gives \(\sum_{t=0}^{M-1}\alpha(x(t))\leq V(x(0))\) for every \(M\), so \(\alpha(x(t))\to0\), and the positive minimum \(m_\varepsilon\) allows \(\|x(t)\|\geq\varepsilon\) only finitely often.

The margin \(\alpha\) cannot be dropped in favor of the bare strict inequality \(V(f_{\rm cl}(x))\lt V(x)\) on \(\Omega\setminus\{0\}\) while \(V\) is assumed continuous only at the origin. The successive decreases can then shrink to zero away from the origin, so \(V\) settles at a positive limit and the state stalls; Exercise 7 constructs such a system. Strict decrease suffices when \(V\) and \(f_{\rm cl}\) are continuous on \(\Omega\), but MPC does not need that route. The decrease proof delivers the margin directly, \(V_N(x^+)-V_N(x)\leq-c(x,u_0^\star)\leq-\lambda_{\min}(Q)\|x\|^2\), so \(\alpha(x)=\lambda_{\min}(Q)\|x\|^2\) with the notation introduced there. This margin also removes the need for compactness, since on any set its values at points with \(\|x\|\geq\varepsilon\) are at least \(\lambda_{\min}(Q)\varepsilon^2\). The set \(\Omega\) can therefore be taken to be the positively invariant feasible region \(X_0\), whether or not \(X_0\) is bounded. The candidate Lyapunov function is the optimal value \(V_N\), and the next section verifies its hypotheses.

The terminal inequality cancels the added tail cost 32:41

Take stage cost \(c(x,u)=x^\mathsf TQx+u^\mathsf TRu\), with \(Q\succ0\), \(R\succ0\), and terminal cost \(p_f(x)=x^\mathsf TPx\), \(P\succ0\). Suppose the origin lies in the interiors of the closed admissible and terminal sets, optimization minima exist, and for every \(z\in X_f\) a feasible terminal action satisfies

This says the drop in the terminal cost can pay for the terminal controller's next stage cost. The same action must satisfy both this inequality and terminal invariance. Using one action to prove invariance and an unrelated infeasible action to prove decrease is insufficient.

Let \(V_N(x)\) be the optimal \(N\)-step value, and denote an optimal sequence and states by \(u_0^\star,\ldots,u_{N-1}^\star\) and \(x_0^\star=x,\ldots,x_N^\star\). After applying \(u_0^\star\), the shifted feasible candidate has cost

The new optimizer can do no worse than this candidate: \(V_N(x_1^\star)\leq J_{\rm shift}\). Subtract the previous value. Every common intermediate stage cancels, leaving

The terminal inequality removes the three tail terms. Notice why the proof compares costs rather than asserting the shifted sequence is optimal: reoptimization generally changes the plan. An inequality is enough because every improvement strengthens the decrease bound.

At zero, the all-zero sequence is feasible and has zero cost, so \(V_N(0)=0\) and the optimizer applies zero input. For nonzero \(x\), the first state penalty gives \(V_N(x)\geq\lambda_{\min}(Q)\|x\|^2>0\). Repeated decrease proves

In particular, \(\sum_t\|x(t)\|^2\) is finite and \(x(t)\to0\). To establish stability as well as attraction, use the terminal controller repeatedly for a state initially in \(X_f\). Its stage costs telescope under the terminal inequality, so its entire \(N\)-step candidate cost is at most \(p_f(x)\). Thus \(V_N(x)\leq p_f(x)\leq\lambda_{\max}(P)\|x\|^2\) locally. This supplies the upper bound without leaving a continuity claim unexplained.

The proof works on the initially feasible region \(X_0\), subject to the stated nominal and optimization assumptions. The region need not be bounded solely because \(X_f\) is bounded. For example, if \(A=0\), \(B=1\), \(X=\mathbb R\), \(U=[-1,1]\), and \(X_f=[-1,1]\), every initial state reaches \(X_f\) in one step using \(u=0\). The feasible region is all of \(\mathbb R\). The lecture slide's boundedness assertion therefore requires extra hypotheses. The quadratic lower bound instead makes each finite value sublevel set bounded, which is sufficient for the argument here.

The Rawlings–Mayne–Diehl textbook develops terminal assumptions and alternative MPC stability constructions. The conditions used here are sufficient, not necessary. A practical controller can work without an explicit terminal set, but dropping that condition means this particular shifted-plan proof no longer establishes its behavior.

The official course slides state the theorem with labeled assumptions, and the correspondence with this section is direct. A0 is \(Q=Q^\mathsf T\succ0\), \(R=R^\mathsf T\succ0\), \(P\succ0\). A1 requires \(X\), \(X_f\), and \(U\) to be closed and to contain the origin in their interiors. A2 requires \(X_f\subseteq X\) to be control invariant and bounded. A3 is the terminal inequality, written as a minimum over the admissible actions \(u\in U\) with \(Az+Bu\in X_f\). The conclusion is asymptotic stability of the origin with domain of attraction \(X_0\). The slides write \(J_0^\star\) for \(V_N\), \(p(\cdot)\) for \(p_f\), and \(\bar v\) for the appended action \(\kappa_f(x_N^\star)\), and they mark the plan computed at time zero with a superscript \([0]\) where this chapter uses a star. Two details deserve comment. A3 is written as a minimum; under A0-A2 that minimum is attained, since \(R\succ0\) makes the objective coercive in \(u\) and the admissible set is closed and, by control invariance, nonempty, so the minimum form and the existence form used here coincide. The boundedness of \(X_f\) in A2 is not used by the proof, and it does not by itself make \(X_0\) bounded, as the counterexample above shows. Borrelli, Bemporad, and Morari, Predictive Control for Linear and Hybrid Systems (Cambridge University Press, 2017), treat the theorem in this setting, including continuity of the optimal value function; here the local bound \(V_N\leq p_f\) supplies the continuity at the origin that the Lyapunov theorem requires.

The control invariance of \(X_f\) can be traded for a condition on the horizon. Suppose the terminal ingredients are as above, with \(X_f\) a sublevel set \(\Omega_\rho=\{x:p_f(x)\leq\rho\}\), \(\rho\gt0\), on which \(\kappa_f\) is admissible, keeps the state in \(\Omega_\rho\subseteq X\), and satisfies the terminal inequality. Remove the constraint \(x_N\in\Omega_\rho\) but keep \(p_f\), and write \(\widetilde V_N\) for the resulting optimal value, with optimal predicted states \(x_k^\star\). If some \(x_k^\star\) lies in \(\Omega_\rho\), the terminal controller run from \(x_k^\star\) gives a feasible tail whose cost, by the telescoping argument above, is at most \(p_f(x_k^\star)\leq\rho\). By the principle of optimality the optimal tail costs no more, and its terminal cost \(p_f(x_N^\star)\) is only one of its nonnegative terms, so \(x_N^\star\in\Omega_\rho\).

Contrapositively, if \(x_N^\star\notin\Omega_\rho\), then none of \(x_0^\star,\ldots,x_{N-1}^\star\) lies in \(\Omega_\rho\). Outside \(\Omega_\rho\), \(\|x\|^2\gt\rho/\lambda_{\max}(P)\), so each stage cost exceeds \(\eta=\lambda_{\min}(Q)\rho/\lambda_{\max}(P)\) and \(\widetilde V_N(x)\gt N\eta\). Hence \(\widetilde V_N(x)\leq N\eta\) forces \(x_N^\star\in\Omega_\rho\): the omitted constraint is satisfied without being imposed, the two problems share their optimizer at \(x\), and the shifted-plan argument applies verbatim. The resulting decrease keeps the next state in the same sublevel set, so \(\{x:\widetilde V_N(x)\leq N\eta\}\) is positively invariant and the stability conclusion holds on it. The threshold \(N\eta\) grows linearly with the horizon. If \(x\) can reach \(\Omega_\rho\) feasibly in \(N_0\) steps, appending terminal actions to an optimal \(N_0\)-step plan shows \(\widetilde V_N(x)\leq V_{N_0}(x)\) for every \(N\geq N_0\), where \(V_{N_0}\) is the value with the terminal constraint; any \(N\geq\max\{N_0,V_{N_0}(x)/\eta\}\) therefore certifies that state. The guarantee covers a sublevel set that depends on \(N\), not all of \(X_0\), and without either the terminal constraint or a horizon condition of this kind the shifted-plan proof certifies nothing.

Lyapunov and Riccati equations construct compatible terminal costs 58:05

First suppose \(A\) is Schur stable, meaning every eigenvalue has magnitude below one, and zero input is admissible. Choose a positively invariant terminal set for \(x^+=Ax\) inside \(X\). The matrix

exists, is positive definite when \(Q\succ0\), and satisfies \(P=Q+A^\mathsf TPA\). With terminal action zero, the terminal decrease expression is exactly zero. The terminal cost equals the accumulated state cost of letting the stable unforced dynamics continue. The MPC law is still free to choose a better action.

The largest terminal set this construction allows is the maximal positively invariant set of the unforced system inside \(X\),

It is positively invariant, because the unforced trajectory from \(Ax\) is the tail of the trajectory from \(x\). It contains every positively invariant subset of \(X\), because each point of such a subset has an unforced trajectory that never leaves \(X\). Zero input is admissible, so \(O_\infty\) is also control invariant for \(x^+=Ax+Bu\), and zero is a terminal action that satisfies invariance and the Lyapunov-equation decrease together. The infinite intersection reduces to a finite one under mild hypotheses. Suppose \(X\) contains the ball of radius \(r_{\rm in}\gt0\) around the origin and lies inside the ball of radius \(r_{\rm out}\). Schur stability gives \(\|A^k\|\to0\), so some \(\bar k\geq1\) satisfies \(\|A^k\|\leq r_{\rm in}/r_{\rm out}\) for all \(k\geq\bar k\). Every \(x\in X\) then has \(\|A^kx\|\leq r_{\rm in}\), hence \(A^kx\in X\), for \(k\geq\bar k\), and

For polyhedral \(X\) this is a polyhedron with finitely many inequalities. The bound \(\bar k\) is usually conservative, and a sharper stopping rule follows from the same invariance idea. Write \(O_t=\bigcap_{k=0}^{t}\{x:A^kx\in X\}\). If the constraint \(A^{t+1}x\in X\) is implied by those defining \(O_t\), then for \(x\in O_t\) the successor \(Ax\) satisfies \(A^k(Ax)\in X\) for \(k=0,\ldots,t\), so \(O_t\) is positively invariant and therefore equals \(O_\infty\). Exercise 8 applies the rule to a two-state example in which \(O_\infty\) is strictly smaller than \(X\).

For an unstable but stabilizable pair \((A,B)\), use a stabilizing infinite-horizon LQR controller \(u=Fx\). With \(Q\succ0\), \(R\succ0\), and the usual stabilizing Riccati solution \(P\), define

The sign is included in \(F\). Completing the LQR square gives the identity

Multiplying on both sides by \(x\) proves terminal decrease with equality. The remaining task is geometric: choose \(X_f\) so that \(x\in X_f\) implies \(x\in X\), \(Fx\in U\), and \((A+BF)x\in X_f\). The uncontrolled plant being unstable does not guarantee a stabilizing Riccati solution; stabilizability is the needed qualification. With merely semidefinite state cost, detectability enters the standard infinite-horizon existence theory as well.

The matrix \(P\) is the stabilizing solution of the discrete-time algebraic Riccati equation

and \(x^\mathsf TPx\) is the unconstrained infinite-horizon LQR value from \(x\). The course slides write \(P_\infty\) and \(F_\infty\) for this \(P\) and \(F\). The closed-loop identity is the same equation after the terminal action has been substituted. With \(S=R+B^\mathsf TPB\) and \(L=B^\mathsf TPA\), so that \(F=-S^{-1}L\), expansion gives \((A+BF)^\mathsf TP(A+BF)+F^\mathsf TRF=A^\mathsf TPA+L^\mathsf TF+F^\mathsf TL+F^\mathsf TSF\). The two cross terms each equal \(-L^\mathsf TS^{-1}L\) and the last term equals \(+L^\mathsf TS^{-1}L\), so the sum is \(A^\mathsf TPA-L^\mathsf TS^{-1}L\). Subtracting \(P\) and adding \(Q\) yields the left side of the Riccati equation, which is zero. Inserting \(u=Fz\) into the terminal inequality therefore produces equality, which is the algebraic reason the LQR value serves as a terminal cost.

The lecture proposes the maximal positively invariant set for this constrained closed loop. A smaller set can be easier to construct and remains sufficient. For instance an ellipsoid \(X_f=\{x:x^\mathsf TPx\leq\rho\}\), \(\rho>0\), is invariant under \(A+BF\) because the Riccati identity makes its quadratic value decrease. We only need choose \(\rho\) small enough to fit the state and input constraints.

Both constructions are workable choices, not optimal ones. Telescoping the terminal inequality along repeated terminal actions from \(z\in X_f\) gives \(p_f(z)\geq\sum_{j=0}^{\infty}c(z_j,\kappa_f(z_j))\), where \(z_0=z\) and \(z_{j+1}=Az_j+B\kappa_f(z_j)\); the right side is the cost of one admissible infinite input sequence. That cost is at least the unconstrained LQR value \(z^\mathsf TP_{\rm ric}z\), where \(P_{\rm ric}\) denotes the stabilizing Riccati solution. A quadratic terminal cost \(z^\mathsf TPz\) that satisfies the terminal inequality on a neighborhood of the origin therefore has \(P\succeq P_{\rm ric}\). Two hinges carry this step. The inequality gives \(p_f(z_{j+1})\leq p_f(z_j)\), so a small sublevel set of \(p_f\) inside that neighborhood keeps the iterates where the inequality holds; and the resulting bound \(z^\mathsf TPz\geq z^\mathsf TP_{\rm ric}z\) near the origin extends to every \(z\) by scaling, because both sides are quadratic. The Riccati choice is the smallest valid quadratic terminal cost, and near the origin, where no constraint is active, MPC with it reproduces the LQR law, because the finite-horizon Riccati recursion started from \(P_{\rm ric}\) stays at \(P_{\rm ric}\). The Lyapunov choice of the stable case instead charges the cost of applying zero input forever. For the data of Exercise 3, with an input added through \(B=1\) and weighted by \(R=1\), it gives \(P_{\rm lyap}=25/3\approx8.3333\), whereas \(P_{\rm ric}\approx3.4977\), smaller by a factor of about \(2.38\).

The terminal sets are not the largest possible either. Each is positively invariant under one fixed linear law, whereas feasibility needs only some admissible action at each point. In the scalar system \(x^+=x+u\) with \(|u|\leq0.5\), \(|x|\leq2\), and \(Q=R=1\), the LQR law is \(u=-Kx\) with \(K\approx0.618034\), and its constrained maximal positively invariant set is \([-0.5/K,0.5/K]\approx[-0.809017,0.809017]\), limited by the input bound. The maximal control invariant set is all of \([-2,2]\), since zero input holds any state in place. That larger set also admits a valid terminal cost. With \(p_f(x)=3x^2\) and the clipped action \(\kappa_f(x)=-0.75x\) for \(|x|\leq2/3\) and \(\kappa_f(x)=-0.5\operatorname{sign}(x)\) beyond, the terminal expression equals \(-1.25x^2\) on the inner interval and \(x^2-3|x|+1\) outside it; the latter is nonpositive for \(0.382\leq|x|\leq2.618\), which covers \(2/3\lt|x|\leq2\). With horizon one, the feasible region grows from \([-1.309017,1.309017]\) to all of \([-2,2]\). The price is local. Near the origin the horizon-one law becomes \(u=-0.75x\), whose infinite-horizon cost \(x^2(1+0.5625)/(1-0.0625)\approx1.666667x^2\) exceeds the LQR value \(1.618034x^2\) by about \(3.0\) percent. Terminal ingredients thus trade feasible region against local optimality, and the two recipes are convenient points in that trade rather than its solution.

A cart terminal ellipsoid can be checked by support functions

Use the exact sampled cart with interval one, \(A=\begin{bmatrix}1&1\\0&1\end{bmatrix}\), \(B=(1/2,1)^\mathsf T\), and \(Q=I\), \(R=1\). Let the state limits be \(|p|\leq2\), \(|v|\leq1\), and acceleration limit \(|u|\leq0.5\). The state coordinate \(p\) here is position, distinct from terminal cost \(p_f\). Numerical Riccati solution gives

For any row vector \(a^\mathsf T\), the maximum of \(|a^\mathsf Tx|\) over \(x^\mathsf TPx\leq\rho\) is \(\sqrt{\rho\,a^\mathsf TP^{-1}a}\). To prove this, write \(y=P^{1/2}x\) and apply Cauchy–Schwarz to \(a^\mathsf TP^{-1/2}y\); equality is attained with \(y\) aligned to \(P^{-1/2}a\).

Apply the formula to the two coordinate rows and to \(F\). A sufficient radius is

The input constraint is the active limiter here. This is an analytically justified ellipsoid with numerically computed coefficients, not a claim to have found the maximal invariant polytope. The executable check verifies the Riccati identity to floating-point tolerance and checks the support bounds. Sampling boundary points is included only as a diagnostic; the support-function calculation supplies the all-points argument.

This construction connects the course's recurring cart to a complete terminal design: the Riccati matrix supplies both the value and a convenient invariant geometry. It also exposes conservatism. An ellipsoid may fit inside a larger admissible polytope and therefore shrink the finite-horizon feasible region more than necessary.

Explicit MPC stores the optimizer's active-set regions 1:06:36

The problem solved at each sampling instant is the constrained linear-quadratic regulator over the horizon,

Its optimal value is the function \(V_N\) of the stability proof. Because the optimizer depends only on the measured state \(x\), the map from \(x\) to the first optimal input can, in important cases, be computed before the controller runs, so that no optimization remains online. That precomputation is explicit MPC.

With linear dynamics, quadratic cost, and polyhedral constraints, eliminating predicted states gives a parameterized convex quadratic program in the input sequence. Write its objective as \(\tfrac12\mathbf u^\mathsf TH\mathbf u+x^\mathsf TG\mathbf u+\text{constant}(x)\), with \(H\succ0\), and inequalities \(M\mathbf u\leq b+Ex\). For a fixed nondegenerate active constraint set \(\mathcal A\), stationarity and active equalities give

Solving this linear system makes the optimizer affine in the parameter \(x\). The inactive inequalities and nonnegative active multipliers define the region where that solution remains valid. Those conditions are linear in \(x\), hence define a polyhedron. A collection of such regions produces the explicit policy. Degenerate active sets require careful treatment, but a strictly convex parametric quadratic program has a unique optimizer, and the standard constrained-LQR result is a continuous piecewise-affine law on its feasible domain.

The data of this quadratic program follow from eliminating the predicted states. Stack \(\mathbf x=(x_1,\ldots,x_N)\) and \(\mathbf u=(u_0,\ldots,u_{N-1})\). The dynamics give \(x_i=A^ix+\sum_{j=1}^{i}A^{i-j}Bu_{j-1}\), that is, \(\mathbf x=\Phi x+\Gamma\mathbf u\), where \(\Phi\) stacks \(A,A^2,\ldots,A^N\) and \(\Gamma\) is block lower triangular with block \(A^{i-j}B\) in block row \(i\) and block column \(j\leq i\). With \(\bar Q=\operatorname{diag}(Q,\ldots,Q,P)\), containing \(N-1\) copies of \(Q\), and \(\bar R=\operatorname{diag}(R,\ldots,R)\), containing \(N\) copies of \(R\), the objective equals \(x^\mathsf TQx+(\Phi x+\Gamma\mathbf u)^\mathsf T\bar Q(\Phi x+\Gamma\mathbf u)+\mathbf u^\mathsf T\bar R\mathbf u\). Matching terms gives

The Hessian \(H\) is positive definite because \(\bar R\) is. Each linear inequality describing \(U\) constrains one block of \(\mathbf u\). Each inequality \(a^\mathsf Tx_i\leq\beta\) describing \(X\) (for \(1\leq i\leq N-1\)) or \(X_f\) (for \(i=N\)) becomes \(a^\mathsf T\Gamma_i\mathbf u\leq\beta-a^\mathsf T\Phi_ix\), where \(\Gamma_i\) and \(\Phi_i\) are the \(i\)th block rows. Collecting these rows gives \(M\mathbf u\leq b+Ex\). The constraint \(x_0=x\in X\) contains no decision variable; it restricts the parameter only. For the horizon-one scalar example below, \(\Phi=\Gamma=1\), \(\bar Q=P\), and \(\bar R=1\), so \(H=2(1+P)\) and \(G=2P\), and the unconstrained minimizer \(-H^{-1}G^\mathsf Tx=-Px/(1+P)\) is the law \(-Kx\) derived there.

Indexing the regions by \(j\) gives the explicit law in the form

where each region \(\mathcal R_j\) corresponds to one active set, \(\Theta_j\) and \(\theta_j\) form the first block of its affine optimizer, and \(n_r\) is the number of regions. The course slides write \(F_k^j\), \(g_k^j\), \(H_k^j\), \(K_k^j\), and \(N_k^r\); the letters used here avoid collisions with the terminal gain \(F\), the Hessian \(H\), and the scalar gain \(K\). The slide index \(k\) reflects that the finite-horizon constrained LQR problem has a piecewise-affine optimal feedback law at every predicted step; a receding-horizon controller applies only the stage-zero law. The offset \(\theta_j\) has a definite source. If \(M_\mathcal A\) has full row rank, solving the active-set system above gives

The constant part of \(\mathbf u\) is \(H^{-1}M_\mathcal A^\mathsf T(M_\mathcal AH^{-1}M_\mathcal A^\mathsf T)^{-1}b_\mathcal A\). It is nonzero whenever an active constraint has a nonzero right-hand side, as an actuator limit does, and it vanishes in the region with an empty active set, where the law is linear. The region itself is where the active set stays optimal: the conditions \(\lambda_\mathcal A(x)\geq0\) and the inactive inequalities evaluated at \(\mathbf u(x)\) are linear in \(x\), and they stack into \(D_jx\leq d_j\).

The regions partition the feasible set \(X_0\), not the whole admissible state set \(X\). In the scalar example below, the states with \(1\lt|x|\leq2\) satisfy the state constraint but admit no feasible plan, so the law is undefined there. The number of regions is at most the number of possible active sets, \(2^{n_c}\) for \(n_c\) inequality rows in \(M\mathbf u\leq b+Ex\). Even with two states, the partition can consist of many cells of irregular shape rather than a few parallel strips, and storage and point location scale with \(n_r\).

This connects directly to the primary paper by Bemporad, Morari, Dua, and Pistikopoulos. The offline work computes regions and affine gains; online work locates the current state and evaluates the corresponding gain. A large number of regions can make storage and point location expensive, so explicit computation is not automatically faster than a modern online solver.

An original scalar example makes the regions visible. Let \(x^+=x+u\), \(|u|\leq0.5\), \(|x|\leq2\), \(X_f=[-0.5,0.5]\), horizon one, and objective \(x^2+u^2+P(x+u)^2\), where \(P=(1+\sqrt5)/2\). The unconstrained minimizer is \(u=-Kx\), \(K=P/(1+P)\approx0.618034\). Feasibility requires \(|x|\leq1\). On that domain the exact policy is

The terminal constraint is satisfied throughout these regions: in the central interval the successor is \((1-K)x\), and in the positive saturated interval it is \(x-0.5\in[0.309017,0.5]\). Negative states are symmetric. At \(x=1\), the action is \(-0.5\), the next state is \(0.5\), and the value drops from \(1.65450850\) to \(0.40450850\), exactly paying the first stage cost \(1.25\). Unlike the previous chapter's feasible but stalled unstable plant, this controller enters the terminal feedback region and converges.

The multiplier test reproduces the switch point. For \(x\gt0\) the candidate active constraint is the actuator bound written as \(-u\leq0.5\), so \(M_\mathcal A=-1\), \(b_\mathcal A=0.5\), and \(E_\mathcal A=0\). With \(H=2(1+P)\) and \(G=2P\), the formulas above give \(\lambda_\mathcal A=2Px-(1+P)\) and \(u=-0.5\), a region with \(\Theta_j=0\) and \(\theta_j=-0.5\). The multiplier is nonnegative exactly when \(x\geq(1+P)/(2P)=0.5/K\approx0.809017\), the boundary found by clipping. The scalar law therefore has \(n_r=3\) full-dimensional regions, although its four inequality rows in \(u\), two actuator bounds and two terminal bounds, would permit up to \(2^4=16\) active sets.

A design recipe fixes the horizon, terminal set, weights, and terminal cost 1:10:10

The theorems of this chapter combine into a design procedure for squared two-norm costs. Choose the horizon \(N\) and a control invariant terminal set \(X_f\subseteq X\). Choose \(Q\succ0\) and \(R\succ0\) for the desired closed-loop behavior. Then choose \(P\) so that the terminal inequality holds on \(X_f\): the Lyapunov equation when \(A\) is Schur stable, the Riccati equation otherwise, with the corresponding invariant sets of the terminal-design section. For any stabilizable pair \((A,B)\) and closed constraint sets that contain the origin in their interiors, the resulting controller satisfies the constraints and is asymptotically stable on its feasible region \(X_0\), provided each optimization is solved exactly and the model is exact. Output constraints are included, because a constraint \(Cx\in Y\) on an output \(y=Cx\) is the state constraint \(x\in\{x:Cx\in Y\}\).

This uniformity is what distinguishes the method. Among the tools developed earlier in the course, unconstrained LQR ignores the constraints, and dynamic programming on a grid scales exponentially with the state dimension. The broader statement that no technique other than MPC currently delivers constraint satisfaction with a stability guarantee for general large multivariable linear systems with input and output constraints is an assessment of the state of the field rather than a theorem. What this chapter proves is the guarantee itself, under the hypotheses just listed.

The order of the steps hides two couplings. First, the terminal ingredients depend on the weights. In the unstable case the gain \(F\), and with it the invariant set built from \(A+BF\) and the input constraint \(Fx\in U\), changes whenever \(Q\) or \(R\) changes, so retuning the weights requires recomputing both \(P\) and \(X_f\). The weights are free only in the sense that every choice with \(Q\succ0\) and \(R\succ0\) admits compatible terminal ingredients. Second, \(X_f\) should be as large as a valid terminal cost permits, because a larger terminal set on which the same terminal cost remains valid cannot shrink \(X_0\) or raise \(V_N\), as shown in the terminal-invariance section. A larger set can require a larger \(P\), and then \(V_N\) can rise: in the scalar comparison of the terminal-design section, the horizon-one value near the origin is \(1.618034x^2\) with the Riccati weight and \(1.75x^2\) with \(P=3\). The horizon enters through the nested chain \(X_f\subseteq X_{N-1}\subseteq\cdots\subseteq X_0\): lengthening the horizon never shrinks the feasible region, while the number of decision variables in the quadratic program grows linearly in \(N\).

The set computations are polyhedral geometry: maximal positively invariant sets, control invariant sets, and explicit-MPC partitions. The Multi-Parametric Toolbox 3 (MPT3), an open-source MATLAB toolbox for parametric optimization, computational geometry, and model predictive control, provides invariant-set computation and explicit MPC synthesis for these tasks.

In practice, a good terminal cost without the terminal constraint is sometimes sufficient, but the guarantee then changes. By the horizon argument of the decrease section, on the sublevel set \(\{x:\widetilde V_N(x)\leq N\eta\}\) the omitted constraint is satisfied automatically and the stability proof applies. For the scalar explicit example, the terminal set \([-0.5,0.5]\) is the sublevel set of \(p_f(x)=Px^2\) at level \(\rho=P/4\), with \(\lambda_{\min}(Q)=1\) and \(\lambda_{\max}(P)=P\), so \(\eta=0.25\). Since \(V_1(1)=1.65450850\) with the terminal constraint, every horizon \(N\geq7\) certifies the state \(x=1\) without imposing the constraint. For the cart ellipsoid, \(\lambda_{\max}(P)\approx3.600743\) and \(\eta\approx0.169821\). Outside such a sublevel set, dropping the constraint removes the certificate, although the controller may still behave well.

Input increments remove a steady-input penalty but do not solve every tracking problem 1:11:08

The final discussion addresses a reference that requires nonzero steady control. Penalizing \(u^\mathsf TRu\) favors zero input, potentially competing with perfect tracking. It need not cause oscillation in every system; a steady tracking bias is already enough to show the mismatch.

For example, consider \(x^+=0.5x+u\) and desired constant state one. Exact steady tracking requires \(u_s=0.5\). Minimizing the steady criterion \((x-1)^2+u^2\) subject to \(u=0.5x\) gives \(x=0.8\), \(u=0.4\). The cost itself prefers a nonzero error. Penalizing \((u-u_s)^2\), when an appropriate feasible reference pair is known, aligns the cost with exact tracking.

The lecture instead introduces \(\Delta u_k=u_k-u_{k-1}\). A tracking objective can penalize output error \(y_k-r_k\) and input change, where \(y_k=Cx_k\), \(r_k\) is the reference output, and \(C\) is the output matrix. A constant nonzero equilibrium input has \(\Delta u=0\), so it incurs no rate penalty.

Written out, the receding-horizon problem at time \(t\) is

with \(\|v\|_W^2=v^\mathsf TWv\). The weights \(Q_y\succeq0\) and \(R_\Delta\succ0\) act on output errors and input increments rather than on states and absolute inputs; the course slides call them \(Q\) and \(R\) and write \(\delta u\) for \(\Delta u\). Only the first increment is used. The applied input is \(u(t)=u(t-1)+\Delta u_0^\star\), and \(u(t)\) becomes the stored \(u_{-1}\) of the next problem, so the controller carries one input of memory and acts as a discrete-time integrator of the optimized increments. The \(k=0\) output term involves only the measured state and does not affect the minimizer. The terminal constraint needs care. A terminal set designed for regulation to the origin would force each plan to end near zero, in conflict with a nonzero reference. For a constant reference \(r\), \(X_f\) must instead be a terminal set for the steady pair \((x_s,u_s)\) with \(x_s=Ax_s+Bu_s\) and \(Cx_s=r\), which is the compatibility requirement discussed next.

This formulation has a precise augmented-state interpretation. Let \(z_k=(x_k,u_{k-1})\). With \(\Delta u_k\) as the new control,

The optimization must still enforce the absolute input constraint on \(u_k=u_{k-1}+\Delta u_k\). The previous input is part of the known initial condition. Terminal conditions and stability analysis must be compatible with this augmented tracking problem; one cannot blindly reuse the zero-regulation theorem with a different objective. Input increments alone also do not guarantee offset-free tracking under arbitrary unknown disturbances or an infeasible reference. State and disturbance estimation, reference feasibility, and terminal design remain relevant.

Executable checks: terminal identities and explicit regions

import numpy as np
from scipy.linalg import solve_discrete_are
from scipy.optimize import minimize_scalar

A = np.array([[1., 1.], [0., 1.]])
B = np.array([[.5], [1.]])
Q, R = np.eye(2), np.ones((1, 1))
P = solve_discrete_are(A, B, Q, R)
F = -np.linalg.solve(R+B.T@P@B, B.T@P@A)
Acl = A+B@F
residual = Acl.T@P@Acl-P+Q+F.T@R@F
assert np.max(np.abs(residual)) < 1e-11
Pinv = np.linalg.inv(P)
rho = min(4/Pinv[0, 0], 1/Pinv[1, 1],
          .25/(F@Pinv@F.T)[0, 0])
assert abs(rho-.611480067553223) < 1e-10
rows = np.vstack([np.eye(2), F])
bounds = np.array([2., 1., .5])
support = np.sqrt(rho*np.diag(rows@Pinv@rows.T))
assert np.all(support <= bounds+1e-12)

p = (1+np.sqrt(5))/2
k = p/(1+p)
def policy(x):
    if abs(x) > 1+1e-12:
        raise ValueError("outside one-step feasible region")
    return np.clip(-k*x, -.5, .5)
def value(x):
    u = policy(x)
    return x*x+u*u+p*(x+u)**2

for x in np.linspace(-1, 1, 81):
    lo, hi = max(-.5, -.5-x), min(.5, .5-x)
    objective = lambda u: x*x+u*u+p*(x+u)**2
    if hi-lo > 1e-12:
        result = minimize_scalar(objective, bounds=(lo, hi),
                                 method="bounded",
                                 options={"xatol": 1e-12})
        candidates = [lo, hi, result.x]
    else:
        candidates = [lo]
    independent = min(objective(u) for u in candidates)
    assert abs(value(x)-independent) < 1e-10
    u = policy(x)
    assert value(x+u)-value(x)+x*x+u*u < 1e-10
assert abs(value(1)-1.654508497187474) < 1e-12
assert abs(value(.5)-.404508497187474) < 1e-12
print("Cart terminal radius:", rho)
print("Explicit MPC switch:", .5/k)
print("Value drop at x=1:", value(.5)-value(1))

A second check covers the terminal-ingredient comparisons, the invariant-set example, and the horizon argument. It verifies the terminal inequality for \(p_f(x)=3x^2\) with the clipped action on 801 grid points of \([-2,2]\), a diagnostic sample of the closed-form argument in the terminal-design section. It confirms \(P_{\rm ric}\approx3.4977\lt25/3\) for the data of Exercise 3. For Exercise 8 it solves linear programs that maximize each row of \(A^2x\) over the three-constraint candidate set and finds the largest value \(1\), so the stopping rule applies. It then builds the condensed matrices \(\Phi\), \(\Gamma\), \(\bar Q\), \(H\), and \(G\) for the scalar example with horizon seven, keeps the actuator bounds, omits the terminal constraint, and solves the bounded quadratic program from \(x=1\) with a quasi-Newton method. The state constraints are checked after the solve; since the relaxed optimizer satisfies them, it is also optimal with them imposed. The computed value equals \(V_1(1)=1.65450850\) to the printed digits, below the threshold \(7\eta=1.75\), and the predicted terminal state, about \(0.00155\), lies in \([-0.5,0.5]\) although nothing enforced it. Finally it recomputes the cart constant \(\eta\).

import numpy as np
from scipy.linalg import solve_discrete_are
from scipy.optimize import linprog, minimize

p = (1+np.sqrt(5))/2
for x in np.linspace(-2, 2, 801):
    u = np.clip(-.75*x, -.5, .5)
    assert abs(x+u) <= 2
    assert -3*x*x+x*x+u*u+3*(x+u)**2 <= 1e-12
p_ric = solve_discrete_are(np.eye(1)*.8, np.eye(1), 3*np.eye(1),
                           np.eye(1))[0, 0]
assert abs(p_ric-3.4977052142105913) < 1e-9
assert p_ric < 25/3 and p < 3

A = np.array([[.5, 1.], [0., .5]])
O1 = np.vstack([np.eye(2), A])
assert np.max(np.abs(O1@np.ones(2))) > 1
worst = 0.
for row in A@A:
    for sign in (1., -1.):
        lp = linprog(-sign*row, A_ub=np.vstack([O1, -O1]),
                     b_ub=np.ones(8), bounds=[(None, None)]*2)
        worst = max(worst, -lp.fun)
assert worst <= 1+1e-9

n_h = 7
Phi = np.ones((n_h, 1))
Gam = np.tril(np.ones((n_h, n_h)))
Qbar = np.diag([1.]*(n_h-1)+[p])
H = 2*(Gam.T@Qbar@Gam+np.eye(n_h))
G = 2*Phi.T@Qbar@Gam
const = 1+(Phi.T@Qbar@Phi)[0, 0]
res = minimize(lambda u: .5*u@H@u+G[0]@u+const, np.zeros(n_h),
               jac=lambda u: H@u+G[0], bounds=[(-.5, .5)]*n_h,
               method="L-BFGS-B", options={"ftol": 1e-15, "gtol": 1e-12})
states = 1+Gam@res.x
direct = 1+np.sum(states[:-1]**2)+np.sum(res.x**2)+p*states[-1]**2
assert abs(direct-res.fun) < 1e-12
assert np.all(np.abs(states[:-1]) <= 2) and abs(states[-1]) <= .5
assert res.fun <= 1.654508497187474+1e-9 <= n_h*.25

Ac = np.array([[1., 1.], [0., 1.]])
Bc = np.array([[.5], [1.]])
Pc = solve_discrete_are(Ac, Bc, np.eye(2), np.ones((1, 1)))
eta = .611480067553223/np.linalg.eigvalsh(Pc)[-1]
assert abs(eta-.16982051864) < 1e-9
print("Riccati versus Lyapunov weight:", p_ric, 25/3)
print("Largest A^2 row value on O_1:", worst)
print("Horizon-7 value and terminal state:", res.fun, states[-1])
print("Cart horizon constant eta:", eta)

Eight exercises with complete solutions

The result to retain is the chain of explicit obligations: feasible shifted continuation, a terminal action that pays its own cost, and value bounds that translate decreasing cost into decreasing state error. The next chapter changes the premise that the transition model is known and asks how data can identify what the controller needs.