Reachable tubes and receding horizons

A trajectory that ends outside an obstacle may have passed through it. A control sequence that satisfies today's finite constraints may leave tomorrow's optimization infeasible. Both failures come from certifying a narrower event than the one the controller actually needs. Reachable tubes address the first mismatch; invariant terminal sets begin to address the second.

This chapter completes the reachability discussion before introducing model predictive control, or MPC. The running cart now approaches a boundary with uncertain acceleration. We first compute a safety margin with a transparent analytical model, then examine a finite-horizon controller that deliberately postpones a constraint problem until it can no longer recover.

Grounded in AA203 lecture 11. Despite the recording's MPC title, approximately its first 40 minutes concern level-set reachability, tubes, and an aircraft pursuit example. The recording states a persistent-feasibility lemma at the end and explicitly postpones its proof to lecture 12. The cart safety derivation, relative-coordinate derivation, and finite numerical counterexamples below are original teaching constructions. The official slides add the placement of MPC between open-loop and closed-loop methods, the control and prediction horizons, polyhedral constraint sets, the terminal cost and terminal set as design choices, explicit quadratic and one-norm or infinity-norm costs, the online algorithm with its infeasibility branch, the strategy of analysis through terminal conditions, set-builder feasible sets, and maximal invariant sets, and the closing pages of the lecture 10 deck, delivered in this recording, add the reach value and feedback laws, the identical-vehicle game, and the alternatives to grid solvers; where their notation differs, the text translates it.

A signed terminal function converts set membership into an outcome 4:05

Let \(\mathcal T\) be a closed target set and choose a continuous function \(h:\mathbb R^n\to\mathbb R\) such that \(\mathcal T=\{x:h(x)\leq0\}\). The sign, rather than the magnitude alone, encodes membership. For a circular protected region of radius \(R>0\) in relative coordinates \(x_r,y_r\), take \(h(x_r,y_r,\theta_r)=\sqrt{x_r^2+y_r^2}-R\). The heading \(\theta_r\) affects future motion but not instantaneous membership. Equivalently the circle satisfies \(x_r^2+y_r^2\leq R^2\); this squared-radius form resolves an imprecise spoken description in the transcript.

A continuous representation is straightforward for closed sets using distance functions; an arbitrary nonclosed set need not be the nonpositive sublevel set of a continuous function. This is a useful qualification to the informal statement that every target can be encoded. Furthermore, replacing \(h\) by \(100h\) preserves membership while multiplying the value scale. A positive value can represent a margin, but it is not automatically a calibrated probability of safety.

Use the previous chapter's dynamics \(\dot x=f(x,u,d)\), controller input \(u\), disturbance input \(d\), initial time \(t<0\), and final time zero. If the controller wants to avoid the target at the final time, its objective is to keep \(h(x(0))\) positive. With a nonanticipative disturbance response strategy \(\Gamma\), the source's value convention is

There is no running cost: only the endpoint is being scored. Under the differential-game assumptions discussed in chapter 10, the local equation is \(J_t+H(x,\nabla J)=0\), with terminal data \(J(x,0)=h(x)\) and Hamiltonian

Here \(p\) is a value gradient, not the cart's position. The controller maximizes the margin, while the adversary minimizes it. For a goal-reaching task, the controller instead minimizes the target function and the disturbance maximizes it. The swap reflects changed objectives and the chosen information pattern; it is not an algebraic license to commute minimization and maximization in any game.

If extrema are attained, the nonpositive sublevel set of \(J_{\rm end}\) identifies states from which the adversary can force endpoint target membership in this formulation. Without attainment, equality cases require care: an infimum of zero can describe an arbitrarily close approach rather than an attained collision. A positive value and an implementing policy have a clearer robust interpretation. The policy remains essential; being outside an unsafe backward set does not make every control safe.

The avoid and reach problems can be stated side by side. Let \(x(\cdot)\) solve \(\dot x=f(x,u,d)\) from \(x(t)=\bar x\). The avoid set contains the states from which some nonanticipative disturbance strategy forces the target against every control; the reach set contains the states from which every such strategy can be answered by a control that reaches the target:

The avoid value is \(J_{\rm end}\). The reach value reverses both optimizations, and so does its Hamilton–Jacobi–Isaacs equation:

The controller now drives the endpoint value down and the disturbance pushes it up. In both problems the disturbance keeps the information advantage of a nonanticipative strategy; only the objectives change. With attained extrema, \(\mathcal A(t)=\{x:J_{\rm end}(x,t)\le0\}\) and \(\mathcal R(t)=\{x:J_{\rm reach}(x,t)\le0\}\). A controller that must avoid the target starts outside \(\mathcal A(t)\); a controller that must reach it starts inside \(\mathcal R(t)\).

Once the value is stored, the feedback law is the input that attains the outer optimization of the Hamiltonian at the current gradient:

Where the value is differentiable, the avoid law cannot let it decrease. Along any trajectory, \(\tfrac{d}{ds}J_{\rm end}(x(s),s)=\partial_sJ_{\rm end}+\nabla J_{\rm end}^\mathsf Tf=\nabla J_{\rm end}^\mathsf Tf-H\), and with \(u=u^\star_{\rm avoid}\) every disturbance gives \(\nabla J_{\rm end}^\mathsf Tf\ge H\); the worst disturbance holds the value constant, as the braking margin below is held constant. The reach law gives the reverse inequality, so a state with \(J_{\rm reach}(x,t)\le0\) ends with \(h(x(0))\le0\). Evaluating either law costs one small optimization per query. At a kink of the viscosity solution \(\nabla J\) does not exist, so an implementation uses a one-sided or numerical gradient and a rule for choosing among maximizers.

A tube scores the least margin anywhere along a trajectory 14:04

For collision avoidance, replace the endpoint outcome by \(\min_{\tau\in[t,0]}h(x(\tau))\). The resulting value is

A positive trajectory minimum means every point of that trajectory lies strictly outside the target. A negative minimum detects even a temporary incursion followed by escape. With the same optimization convention, \(J_{\rm tube}\leq J_{\rm end}\), because each trajectory's minimum is no greater than its endpoint value and taking suprema and infima preserves that ordering. Thus the unsafe tube includes the corresponding unsafe endpoint set.

The source gives the autonomous tube equation in the form

The extra minimum encodes the ability of the outcome to retain an earlier worse margin. The lecture presents its interpretation rather than a full viscosity-solution derivation. This chapter follows that boundary: the equation is stated for the source's setting, not asserted as an interchangeable formula for time-varying targets, state-dependent stopping rules, or reach-avoid games with separate goal and obstacle sets.

An original deterministic example separates the two outcomes without any numerical PDE. Let one-dimensional position follow \(z(s)=-2+4s\) for \(0\leq s\leq1\), and let the target be \([-1,1]\), represented by \(h(z)=|z|-1\). The final value is \(h(z(1))=1\), yet the trajectory minimum is minus one at \(s=1/2\). Endpoint success has concealed a full passage through the target. Refining only the final-time calculation cannot correct an incorrectly chosen objective.

The StanfordASL reachability implementation represents reachable states through value-function sublevel sets and solves Hamilton–Jacobi equations numerically. Its availability does not remove the need to choose the correct target, dynamics, disturbance bounds, and set-versus-tube outcome. Those are modeling decisions made before a solver is called.

Relative coordinates retain the heading that changes collision risk 22:36

The lecture illustrates two planar vehicles with constant speeds and controlled turning rates. For vehicle \(a\), \(\dot x_a=v_a\cos\theta_a\), \(\dot y_a=v_a\sin\theta_a\), and \(\dot\theta_a=u_a\); vehicle \(b\) has analogous equations. The avoidance target depends on their relative distance. Relative heading still matters because approaching vehicles and aligned vehicles can have the same separation but very different futures.

Here is an explicit reduction, supplied as a derivation rather than a transcription of a displayed plot. Let \(r=R(-\theta_a)(q_b-q_a)\), where \(q_a,q_b\in\mathbb R^2\) are positions and \(R(\alpha)\) is the planar rotation matrix. Write \(r=(x_r,y_r)\) and \(\theta_r=\theta_b-\theta_a\). Differentiating both the displacement and the moving coordinate frame gives

The terms \(u_a y_r\) and \(-u_a x_r\) arise because the reference frame rotates. Omitting them silently changes the physical model. At equal speeds and zero relative heading with no turns, relative position is constant. At relative heading \(\pi\), the longitudinal approach rate is \(-v_b-v_a\). This explains why a backward unsafe region can protrude well beyond the instantaneous collision circle for approaching configurations.

The demonstrated game uses two identical vehicles: equal speeds \(v_a=v_b=v\) and equal turn-rate bounds \(u_a,u_b\in[-u_{\max},u_{\max}]\). Vehicle \(a\) is the evader and plays the controller, \(u=u_a\); vehicle \(b\) is the pursuer and plays the disturbance, \(d=u_b\). Write \(p=(p_1,p_2,p_3)\) for the value gradient in the coordinates \((x_r,y_r,\theta_r)\), which the official slides call \((x_1,x_2,x_3)\). Both inputs enter \(p^\mathsf Tf\) linearly, with coefficients \(p_1y_r-p_2x_r-p_3\) for \(u_a\) and \(p_3\) for \(u_b\), so each optimization in the avoid Hamiltonian is attained at a bound:

The evader's feedback law turns at full rate, \(u_a^\star=u_{\max}\operatorname{sign}(p_1y_r-p_2x_r-p_3)\), and the worst pursuer also turns at full rate, \(u_b=-u_{\max}\operatorname{sign}(p_3)\); where a coefficient vanishes, every admissible rate attains the optimum. Because \(H\) separates into a term in \(u_a\) and a term in \(u_b\), the order of maximization and minimization does not change it in this game, although it can in games whose Hamiltonian couples the inputs. This is the game of two identical vehicles in Section 2.6.1 of Ian M. Mitchell's A Toolbox of Level Set Methods, Version 1.1 (UBC Computer Science technical report TR-2007-11, 2007), the manual of the ToolboxLS MATLAB package. There the evader's turn rate is the control input avoiding a collision cylinder around the \(\theta_r\) axis, whose radius is the protected radius \(R\) of the first section, and the pursuer's turn rate is the disturbance input trying to reach it; the computed example uses radius 5, speeds 5, and turn-rate bounds 1.

The recording displays a three-dimensional value boundary and fixed-heading slices, then simulates policies from several initial conditions. One trajectory temporarily enters the protected region and later leaves it; the lecturer identifies this as acceptable to the endpoint formulation but unacceptable to the intended all-time safety task. The numerical shape and apparent settling time belong to that demonstration's parameters. They are not a general theorem that every reachable set becomes stationary after a finite horizon.

An exact braking margin exposes disturbance assumptions

Return to a one-dimensional cart. Let \(q\geq0\) be remaining distance to a boundary and \(v\geq0\) the closing speed. The dynamics are \(\dot q=-v\) and \(\dot v=u+d\), with actuator bound \(|u|\leq U\), disturbance bound \(|d|\leq D\), and \(U>D\geq0\). The target is boundary contact, \(q\leq0\). The model assumes instantaneous control and state knowledge and uses normalized distance and time units.

Under strongest braking \(u=-U\), the worst opposing disturbance is \(d=D\). The guaranteed deceleration is \(b=U-D>0\). Until the speed reaches zero, \(v(s)=v_0-bs\) and \(q(s)=q_0-v_0s+\tfrac12bs^2\). Stopping occurs after \(v_0/b\), giving the smallest distance

This is not just an intuitive braking formula. Against constant \(d=D\), no admissible controller can decrease velocity faster than \(-b\); therefore it cannot lose less distance before stopping. Conversely, maximum braking attains this bound. After stopping, a feedback that prevents positive closing speed can preserve distance within this idealized continuous-time model. Thus \(B>0\) is a strict all-time avoidance condition for initially nonnegative closing speed. Equality reaches the boundary and is unsafe if contact belongs to the target.

Along the worst-case maximum-braking trajectory, \(\dot B=-v-(v/b)(-b)=0\). The margin stays constant while physical distance decreases because the remaining stopping distance decreases at exactly the same rate. This is the control meaning of a value boundary: it combines position and velocity rather than checking position alone.

For \(U=1\), \(D=0.25\), \(v_0=1\), and \(q_0=0.8\), the guaranteed stopping distance is \(2/3\), leaving margin \(0.133333\ldots\). A nominal calculation with \(D=0\) predicts stopping distance \(1/2\). Starting at \(q_0=0.6\) would pass that nominal check and fail the robust one. This original example is exact for its model; it is not a physical vehicle certification.

For a finite horizon of duration \(T\), the worst distance lost before time \(T\) is \(v_0s_\star-\tfrac12bs_\star^2\), where \(s_\star=\min(T,v_0/b)\). A short-horizon tube can consequently label a state safe over its horizon even though eventual contact is unavoidable. Choosing a tube instead of a set fixes the within-horizon event; it does not automatically fix insufficient horizon length.

Grid computation separates numerical approximation from model guarantees 39:08

The lecture emphasizes the dimensional cost of grid-based PDE methods. With \(m\) grid points per coordinate in an \(n\)-dimensional state, one value slice has \(m^n\) entries. At \(m=101\), six dimensions already require about \(1.06\times10^{12}\) entries. A single eight-byte scalar per entry uses roughly 8.5 terabytes before derivative arrays and solver workspace. Reducing two vehicles to three relative coordinates is therefore a mathematical and computational simplification, not merely prettier notation.

The value \(J(x,t)\) depends on the \(n\) state coordinates and on time, so a traditional PDE solver computes it on an \((n+1)\)-dimensional grid. An explicit time-marching scheme keeps only the few slices its update needs, which is why the slice size above sets the working memory. The time history matters when the feedback law of the first section is executed, because the law at time \(s\) uses \(\nabla J(\cdot,s)\); storing \(K\) time samples multiplies the slice size by \(K\). For the three relative coordinates with \(m=101\), one slice has \(1{,}030{,}301\) entries, and \(K=100\) stored slices hold about \(1.03\times10^8\) entries, roughly 0.82 gigabytes at eight bytes each.

Decomposition, neural approximation, and sampling can extend computational reach, but their guarantees depend on the particular approximation and error analysis. The recording's practical five-or-six-dimensional rule of thumb describes common grid limitations, not an impossibility theorem. A computed positive value is only as trustworthy as the discretization, numerical error control, boundary treatment, dynamics, and disturbance model that produced it.

The alternatives to a global grid each give up a different part of that result. One can sacrifice global optimality: local trajectory optimization around a nominal maneuver, as in earlier chapters, scales to many more dimensions but says nothing about states far from that maneuver. One can give up guarantees and accept an approximate value whose error is not bounded. Neural-network PDE solvers and sampling-based methods replace the grid by a learned or sampled representation, and reinforcement learning, developed in the course's final part, estimates values and policies from simulated or measured experience rather than from a solved PDE. Where a method produces an approximate value, its zero sublevel set estimates the reachable set, and it certifies the set only if a separate argument bounds the approximation error; local trajectory optimization produces no value away from its nominal maneuver.

A useful finite counterpart is the robust predecessor of a set \(S\):

The controller chooses an action before the unknown disturbance is revealed. On a fully enumerated finite transition system, this quantifier can be checked exactly. On samples from a continuous disturbance interval, checking only those samples is exact only when additional structure justifies it, such as affine dependence and convex target inequalities permitting vertex checks. The executable example below is deliberately a finite model.

MPC turns repeated open-loop optimization into a feedback law 44:10

At time \(t\), measure the current state, optimize a finite sequence, apply its first action, and repeat at the next measurement. That is the basic receding-horizon construction. It avoids tabulating a policy over every possible state, while allowing the next decision to react to an observed deviation. It does not compute a full contingent policy within each nominal prediction horizon.

This construction sits between the two families of methods developed earlier in the course. Open-loop methods, indirect ones through necessary conditions and direct ones through transcription into a finite-dimensional program, compute one input sequence for one initial state. They are comparatively fast, but a sequence cannot respond to a state it did not anticipate. Closed-loop methods, dynamic programming in discrete time and the Hamilton–Jacobi–Bellman and Hamilton–Jacobi–Isaacs equations in continuous time, compute a policy for every state; the reachability analysis of the preceding sections belongs to this branch, and the grid arithmetic above shows its computational price. Model predictive control solves only open-loop problems, but it solves a new one from every measured state, and that measurement closes the loop. The broader name receding-horizon control covers this construction and related variants, with emphasis on the moving window rather than on the prediction model.

The window need not use one horizon for every purpose. A common variant distinguishes the prediction horizon \(N_p\), the number of future steps over which predicted behavior enters the cost and constraints, from the control horizon \(N_m\le N_p\), the number of steps over which the input is free. Inputs after step \(N_m-1\) are held at the last free value, so the optimizer chooses \(N_m\) input vectors while the cost and constraints still see \(N_p\) predicted steps. For a scalar input with \(N_p=20\) and \(N_m=5\), the optimization has 5 decision variables instead of 20. Because the held inputs restrict the admissible sequences, the restriction can only raise the optimal cost or shrink the set of feasible states, never the reverse. In the same variant the predicted quantities are often outputs, measured functions of the state, compared with a reference trajectory rather than with the origin. At the next sampling time both horizons shift forward by one step. This chapter takes \(N_m=N_p=N\), predicts the full state, and regulates it to the origin; reference tracking returns in chapter 12. Each applied input is held over its sampling interval \([t,t+1)\), so for a continuous-time plant the model \(x^+=Ax+Bu\) must be the sampled map under that hold, as for the exact held-input cart model of chapter 7.

For the source's linear time-invariant model \(x^+=Ax+Bu\), let \(X\) be the admissible state set, \(U\) the admissible input set, \(X_f\subseteq X\) the terminal set, \(c(x,u)\) a stage cost, \(p_f(x)\) a terminal cost, and \(N\) the number of predicted transitions. We use \(p_f\) to avoid confusing terminal cost with the earlier value gradient. At each measurement solve

The regulation task asks the closed loop to drive the state to the origin while satisfying \(x(t)\in X\) and \(u(t)\in U\) at every time \(t\geq0\). Here \(x(t)\in\mathbb R^n\) and \(u(t)\in\mathbb R^{n_u}\); we write \(n_u\) for the input dimension, which the official slides call \(m\), because \(m\) already counts grid points per coordinate. The sets are polyhedra, intersections of finitely many closed half-spaces: \(X=\{x:G_xx\le g_x\}\) and \(U=\{u:G_uu\le g_u\}\), with the inequalities read componentwise. A box is the simplest case; the scalar bound \(|u|\le0.5\) used below is the polyhedron \(\{u:u\le0.5,\ -u\le0.5\}\). Linear dynamics keep these constraints linear in the decision variables, because each predicted state is affine in the inputs:

Each condition \(G_xx_k\le g_x\) is therefore a finite set of linear inequalities in \(u_{0:N-1}\), and so is \(x_N\in X_f\) when \(X_f\) is a polyhedron. The constraints in the planning problem bind only predictions. Whether the measured closed-loop states satisfy them at every time is the persistent-feasibility question taken up below.

The policy is \(\pi_N(x)=u_0^\star(x)\), and the actual closed loop is \(x(t+1)=Ax(t)+B\pi_N(x(t))\). The local prediction index \(k\) restarts at zero after each measurement. In the fuller notation, \(x_{t+k\mid t}\) means the prediction for time \(t+k\) made at time \(t\). Predictions for the same future time made at two different planning times need not agree.

Time invariance is what justifies restarting the local index. Because \(A\), \(B\), \(c\), \(p_f\), \(X\), \(U\), and \(X_f\) do not depend on \(t\), the problem posed at time \(t\) differs from the problem posed at time zero only through its initial state \(x_0=x(t)\). One value function \(V_N\) and one policy \(\pi_N\) therefore serve every planning time, and the closed loop is the time-invariant map \(f_{\rm cl}(x)=Ax+B\pi_N(x)\). If any of these data change with time, as with a moving obstacle or a time-varying reference, the planning problem carries the time index, the policy becomes \(\pi_t\), and the closed loop becomes \(f_{\rm cl}(x,t)\). The official slides write \(J_t^\star(x(t))\) for the optimal value, \(p\) for the terminal cost, and \(U_0^\star(x(t))=\{u_0^\star,\ldots,u_{N-1}^\star\}\) for the optimal input sequence; the correspondence is \(J_0^\star=V_N\), \(p=p_f\), and \(U_0^\star(x)=u_{0:N-1}^\star(x)\).

Inputs carry the same double index: \(u_{t+k\mid t}\) is the input for time \(t+k\) computed at time \(t\), and only \(u_{t\mid t}\) is ever applied. The two predictions \(x_{3\mid1}\) and \(x_{3\mid2}\) of the state at time 3 can differ for two separate reasons. A disturbance or model error can move the measured state \(x(2)\) away from the prediction \(x_{2\mid1}\), so the second plan starts elsewhere. Even with an exact model and \(x(2)=x_{2\mid1}\), the plan made at time 2 optimizes over a window that ends one step later, with its own terminal cost and constraint, and it may select different inputs from the same state. Equality of the two predictions is possible but not guaranteed.

The feedback law can be nonlinear even when the plant is linear, because active constraints change with the measured state. With quadratic costs and polyhedral constraints, each planning problem is a convex quadratic program if the state and terminal weights are positive semidefinite and the input weight is positive definite. One-norm and infinity-norm costs lead to different optimization representations. MPC describes the repeated decision framework; it does not identify a unique solver or objective.

The two standard cost families make these statements concrete. The quadratic family uses \(p_f(x_N)=x_N^\mathsf TP_fx_N\) and \(c(x_k,u_k)=x_k^\mathsf TQx_k+u_k^\mathsf TR_uu_k\) with \(P_f\succeq0\), \(Q\succeq0\), and \(R_u\succ0\); the official slides write \(P\), \(Q\), and \(R\), and we write \(P_f\) and \(R_u\) because \(R\) already denotes a radius and a rotation in this chapter. The norm family uses \(p_f(x_N)=\|P_fx_N\|_\ell\) and \(c(x_k,u_k)=\|Qx_k\|_\ell+\|R_uu_k\|_\ell\) with \(\ell=1\) or \(\ell=\infty\), where now each matrix has full column rank. The one-norm sums the absolute values of the components; the infinity-norm takes the largest absolute component. Full column rank of \(Q\) makes \(\|Qx\|_\ell\) vanish only at \(x=0\), so the stage cost penalizes every nonzero state. Full column rank of \(R_u\) makes \(\|R_uu\|_\ell\) grow at least in proportion to \(\|u\|_\ell\), so large inputs are always penalized.

Each norm term becomes linear after an epigraph reformulation. For a one-norm term \(\|Qx_k\|_1\), introduce a vector \(\epsilon_k\) with \(-\epsilon_k\le Qx_k\le\epsilon_k\) componentwise and minimize the sum of its entries. For an infinity-norm term, introduce a scalar \(\epsilon_k\) with \(-\epsilon_k\mathbf 1\le Qx_k\le\epsilon_k\mathbf 1\), where \(\mathbf 1\) is the vector of ones, and minimize \(\epsilon_k\). At an optimum each entry of a one-norm auxiliary vector equals the absolute value of the component it bounds, so the entries sum to the norm, and an infinity-norm auxiliary scalar equals the norm, since any larger value only adds cost. With polyhedral \(X\), \(U\), and \(X_f\), the norm-cost planning problem is therefore a linear program, while the quadratic family gives a quadratic program.

The two families differ in whether the policy \(\pi_N(x)=u_0^\star(x)\) is well defined without further rules. In the quadratic family each predicted state is affine in the inputs, so the objective is a convex quadratic function of \(u_{0:N-1}\) plus the strictly convex term \(\sum_ku_k^\mathsf TR_uu_k\). The sum is strictly convex and grows without bound as the inputs grow, so on a nonempty closed convex feasible set it has exactly one minimizer. A linear program that is bounded below on a nonempty polyhedron also attains its minimum, but its minimizer need not be unique. Take the unstable scalar plant \(x^+=2x+u\) of the dead-end example, with \(|x|\le1\), \(|u|\le0.5\), \(N=1\), \(X_f=X=[-1,1]\), stage cost \(|x|+|u|\), and terminal cost \(|x_1|\). At \(x=0.1\) the objective is \(0.1+|u|+|0.2+u|\), which equals \(0.3\) for every \(u\in[-0.2,0]\) and exceeds it for every other admissible input. A solver may return \(u=0\), giving \(x^+=0.2\), or \(u=-0.2\), giving \(x^+=0\). The closed loop is defined only after a selection rule is fixed. With the quadratic costs \(x^2+u^2\) and \(x_1^2\), the objective \(0.01+u^2+(0.2+u)^2\) has the unique minimizer \(u=-0.1\) and the successor \(0.1\).

As an online algorithm, each sampling period performs four operations. Measure \(x(t)\). Solve the planning problem from \(x_0=x(t)\). If it has no solution, report infeasibility and stop; otherwise apply \(u_0^\star\) over \([t,t+1)\). Wait for time \(t+1\) and repeat. For both cost families, with the closed convex constraint sets assumed here, a feasible planning problem attains its minimum, so the absence of an optimal sequence means exactly that \(x(t)\) lies outside \(X_0\), the set of states with a feasible plan defined in the invariance section. The stop branch is part of the algorithm rather than a rare numerical event: the dead-end example reaches it after one step without any solver error. A deployed controller needs a fallback for that branch, which the nominal formulation does not supply. The word online means that the optimization runs during operation at every sampling time; chapter 12 describes the explicit alternative, which solves the parametric problem offline and stores the resulting law.

Only the first action is executed, but the remaining sequence need not be discarded computationally. It can serve as a shifted warm start. A warm start accelerates an optimization attempt; it is not automatically a feasible backup after a disturbance. The lecture focuses on nominal full-state feedback. Robust MPC, estimation, delayed actuation, and solver deadlines require additional design.

The text by Rawlings, Mayne, and Diehl, also named in the lecture, develops MPC theory and computation in a unified treatment. Here the important connection is that terminal design is part of the control argument, rather than simply a numerical convenience appended to a trajectory optimizer.

That text is Model Predictive Control: Theory, Computation, and Design (2nd edition, Nob Hill Publishing, 2017). A second standard reference, F. Borrelli, A. Bemporad, and M. Morari, Predictive Control for Linear and Hybrid Systems (Cambridge University Press, 2017), develops the constrained linear and hybrid setting, including the polyhedral and invariant-set computations used in the remaining sections.

Three ingredients of the planning problem are design choices rather than data of the regulation task: the terminal cost \(p_f\), the terminal set \(X_f\), and the horizon \(N\). The task fixes the dynamics, the stage cost, and the sets \(X\) and \(U\), and it is posed over an unbounded future; the terminal ingredients stand in for the part of that future beyond the window. The horizon trades computation against foresight. A longer window lets the optimizer see consequences that a short one ignores, at the price of more decision variables, while a short window can act greedily, as the next section shows. The horizon also interacts with the terminal set, because a plan exists only if \(X_f\) can be reached in \(N\) steps. For the scalar plant above with the terminal equality \(X_f=\{0\}\), the state \(x=0.4\) has a feasible plan only when \(N\geq3\); Exercise 8 derives the feasible interval for every \(N\). Chapter 12 turns these choices into conditions that guarantee feasibility and stability.

A feasible short plan can select a state with no feasible continuation 53:41

Consider an original scalar unstable plant \(x^+=2x+u\), with \(|x|\leq1\) and \(|u|\leq0.5\). Use horizon \(N=1\), terminal set \(X_f=X=[-1,1]\), zero terminal cost, and stage cost \(x^2+u^2\). For a fixed measured state, minimizing that stage cost means using the feasible input with smallest magnitude.

Starting at \(x=0.4\), input \(u=0\) is feasible and uniquely minimizes the input penalty. The next state is \(0.8\). But from \(0.8\), the smallest achievable next state is \(2(0.8)-0.5=1.1\), which violates the state constraint. Today's exact optimum has created tomorrow's infeasibility. There is no solver error and no disturbance in this example.

Replace the terminal set by \(C=[-0.5,0.5]\). From \(x=0.4\), one-step feasibility now requires \(0.8+u\leq0.5\), so the minimizing feasible action is \(u=-0.3\) and the next state is \(0.5\). From that boundary, \(u=-0.5\) keeps the state at \(0.5\) forever. Feasibility is preserved, but convergence to the origin is absent. These are distinct control properties, and a terminal set alone does not prove both.

Indeed \(C\) is control invariant: for every \(x\in C\), the admissible action \(u=-x\) gives \(x^+=x\in C\). It is also the maximal invariant subset of \(X\). If \(x>0.5\), even maximal negative input yields \(x^+\geq2x-0.5>x\); repeated application drives the lower bound beyond one. The negative side is symmetric. The boundary points are viable but cannot be driven inward with these actuator limits. Stronger regulation claims therefore need an appropriate region of attraction.

The two failures have names. Persistent feasibility asks that the planning problem remain solvable at every future measurement; the first terminal set violates it. Stability, in the asymptotic sense that regulation needs, asks that small initial deviations produce small excursions and that closed-loop trajectories converge to the origin; the second terminal set preserves feasibility and still violates it. With \(X_f=C\), the minimizing input is \(u=0\) whenever \(2x\in C\), that is, for \(|x|\le0.25\), and \(u=0.5-2x\) on \([0.25,0.5]\), with the mirror image \(u=-0.5-2x\) on \([-0.5,-0.25]\). The closed loop therefore doubles every state of magnitude at most \(0.25\) and maps \([0.25,0.5]\) to \(0.5\). From \(x(0)=0.01\) the trajectory is \(0.02,0.04,0.08,0.16,0.32,0.5\), after which it remains at \(0.5\). Initial deviations of any positive size reach the boundary, so the origin is an unstable equilibrium of this closed loop, not merely an unreached target.

Failure to converge and instability are nevertheless different properties. For the plant \(x^+=x+u\) with \(X=U=[-1,1]\), \(N=1\), \(X_f=X\), zero terminal cost, and stage cost \(u^2\), which is the quadratic family with the admissible weight \(Q=0\), the optimizer chooses \(u=0\) at every state. Every state is then an equilibrium. The origin is stable, because no trajectory moves, but it attracts no other state. Asymptotic stability, the property regulation requires, needs both that small deviations stay small and that trajectories converge.

Even the scalar closed loop with \(X_f=C\) is a piecewise affine map with three pieces on \(X_0=[-0.5,0.5]\), and the pieces were found by solving the optimization by hand. In general \(f_{\rm cl}\) is defined only implicitly, through the minimizer of a constrained program, and it changes whenever the cost, the terminal ingredients, or the horizon change. Analyzing it directly for every candidate design is therefore difficult. The alternative, begun in the next section and completed in chapter 12, is to derive conditions on the terminal cost \(p_f\) and the terminal set \(X_f\) under which persistent feasibility and closed-loop stability hold by construction, without computing \(f_{\rm cl}\).

Predecessors and invariance describe whether another plan will exist 1:04:20

Let \(X_0\) denote all initial states with at least one feasible \(N\)-step sequence. For a deterministic controlled transition \(f\), define \(\operatorname{Pre}(S)=\{x:\exists u\in U,\ f(x,u)\in S\}\). This definition does not by itself impose \(x\in X\); intersect with the admissible current-state set when required. For an autonomous map \(f(x)\), omit the existential input choice.

In set-builder form, with \(x_{k+1}=Ax_k+Bu_k\),

A planning problem has a solution only if, and for the cost families above exactly when, the current state lies in \(X_0\). The set \(\operatorname{Pre}(S)\) is also called the one-step controllable set to \(S\), and the official slides write the transition map as \(\phi\) rather than \(f\). In the autonomous case no input constraint enters, because there is no input.

A set \(C\subseteq X\) is control invariant if every \(x\in C\) admits some admissible input with successor in \(C\). A set is positively invariant under a fixed closed-loop map if that particular map keeps it inside itself. The distinction is existential versus already selected control. A poorly chosen policy can leave a control-invariant set.

These notions state the design goal precisely. The MPC law is persistently feasible if every closed-loop trajectory that starts in \(X_0\) remains in \(X_0\): \(x(0)\in X_0\) implies \(x(t)\in X_0\) for all \(t\geq0\). Equivalently, \(X_0\) is positively invariant under the closed-loop map \(f_{\rm cl}\). Because \(f_{\rm cl}\) depends on the cost through the selected minimizer, this property can depend on the cost. In the dead-end example with \(X_f=X\), Exercise 4 gives \(X_0=[-0.75,0.75]\); from \(x=0.4\), every input in \([-0.5,-0.05]\) would have kept the successor in \(X_0\), but the stage cost selected \(u=0\) and the successor \(0.8\notin X_0\). That shows the cost deciding one transition, not the property as a whole: in this example persistent feasibility fails for every cost, because from \(x=0.75\in X_0\) the only feasible input is \(u=-0.5\), whose successor \(1\) lies outside \(X_0\). Invariant-set theory supplies sufficient conditions that hold for every cost.

Invariant sets are not unique, so the useful object is the largest one. The maximal control invariant set \(C_\infty\subseteq X\) is a control invariant set that contains every control invariant subset of \(X\). It exists because a union of control invariant sets is control invariant: a state in the union lies in some member \(C_i\), and the input that keeps it in \(C_i\) keeps it in the union. Hence \(C_\infty\) is the union of all control invariant subsets of \(X\). For an autonomous map the same argument defines the maximal positive invariant set \(O_\infty\subseteq X\), which contains every positive invariant subset of \(X\). For a linear plant with \(0\in X\) and \(0\in U\), the singleton \(\{0\}\) is control invariant, because \(A0+B0=0\); it is always a member of the family whose union is \(C_\infty\), and it is the only nonempty member when the inputs cannot counteract the dynamics anywhere else, as for \(x^+=2x+u\) with \(U=\{0\}\) and \(X=[-1,1]\).

These sets can be computed by a backward recursion. Set \(\Omega_0=X\) and \(\Omega_{k+1}=\Omega_k\cap\operatorname{Pre}(\Omega_k)\). Then \(\Omega_k\) is the set of states that admit inputs keeping the state in \(X\) for \(k\) transitions. Because \(\operatorname{Pre}\) is monotone, \(S\subseteq T\) implying \(\operatorname{Pre}(S)\subseteq\operatorname{Pre}(T)\), induction gives \(C_\infty\subseteq\Omega_k\) for every \(k\). If two consecutive iterates coincide, then \(\Omega_k\subseteq\operatorname{Pre}(\Omega_k)\), so \(\Omega_k\) is control invariant and equals \(C_\infty\). For the scalar plant \(x^+=2x+u\) with \(|u|\le0.5\), a successor interval \([2x-0.5,2x+0.5]\) meets \([-r,r]\) exactly when \(|x|\le(r+0.5)/2\). Hence \(\Omega_k=[-r_k,r_k]\) with \(r_0=1\) and \(r_{k+1}=(r_k+0.5)/2\), so \(r_k=0.5+2^{-(k+1)}\): the radii are \(1\), \(0.75\), \(0.625\), and so on. The intersection of the iterates is \(C=[-0.5,0.5]\). Since \(C_\infty\subseteq\Omega_k\) for every \(k\), \(C_\infty\subseteq C\), and since \(C\) is control invariant, \(C\subseteq C_\infty\); this confirms that \(C\) is the maximal control invariant set, yet no finite iterate equals it. Finite termination is therefore not guaranteed, even in one dimension. For polyhedral sets and linear dynamics each \(\operatorname{Pre}\) step is a polyhedral projection, and the Multi-Parametric Toolbox 3 (MPT3), a MATLAB toolbox for parametric optimization, computational geometry, and model predictive control, provides invariant-set computations of this kind.

The recording ends by defining the truncated feasible set \(X_1\), containing states from which the remaining \(N-1\) transitions can satisfy constraints and reach \(X_f\). Its stated lemma is: if \(X_1\) is control invariant, then an initially feasible nominal MPC law remains feasible. The proof and the more practical terminal-set condition are developed in chapter 12, where the recording itself develops them. The core intuition is that invariance supplies one admissible extra transition before using the remaining finite plan.

Formally,

the construction of \(X_0\) with one fewer transition. For \(N=1\) no input remains to be chosen, so \(X_1=X\cap X_f\). The lemma's hypothesis concerns this induced set rather than a quantity the designer sets directly, which is why chapter 12 replaces it by a condition on \(X_f\). The scalar example exhibits both outcomes at \(N=1\). With \(X_f=X=[-1,1]\), \(X_1=[-1,1]\) is not control invariant, since the smallest successor of \(0.8\) is \(1.1\); the hypothesis fails, and the dead end shows that persistent feasibility fails as well. With \(X_f=C\), \(X_1=C\) is control invariant, and the closed loop computed in the previous section never leaves \(X_0=C\). Because the lemma is only sufficient, the first failure is a separate fact: a failed hypothesis alone would not have proved infeasibility. Exercise 7 repeats the comparison at \(N=2\).

For the earlier scalar example, choosing a terminal set changes which future states the optimizer is allowed to select. The difference between \(X=[-1,1]\) and \(C=[-0.5,0.5]\) is precisely the difference between immediate constraint satisfaction and indefinite viability. This geometric information complements a cost: lowering a finite objective cannot compensate for choosing a state from which no admissible continuation exists.

Executable checks: exact margins and a deliberate MPC failure

This program uses only finite enumeration and closed-form scalar optimization. The braking sample checks agree with an exact quadratic trajectory; they are not being used to infer a continuous guarantee from an arbitrary time grid. The one-step MPC interval follows directly from its linear inequalities.

import numpy as np

U, D, v, q = 1., .25, 1., .8
b = U-D
stop = v/b
s = np.linspace(0, stop, 1001)
distance = q-v*s+.5*b*s*s
margin = q-v*v/(2*b)
assert abs(distance.min()-margin) < 1e-12
assert abs(margin-2/15) < 1e-12
assert .6-v*v/(2*U) > 0
assert .6-v*v/(2*b) < 0

def one_step(x, terminal_radius):
    lo = max(-.5, -terminal_radius-2*x)
    hi = min(.5, terminal_radius-2*x)
    if lo > hi:
        return None
    return min(max(0., lo), hi)

u = one_step(.4, 1.)
assert u == 0.
assert one_step(2*.4+u, 1.) is None
x = .4
trajectory = [x]
for _ in range(10):
    u = one_step(x, .5)
    assert u is not None
    x = 2*x+u
    trajectory.append(x)
assert np.allclose(trajectory[1:], .5)

states = range(-3, 4)
actions = (-1, 0, 1)
disturbances = (-1, 1)
target = {-1, 0, 1}
nominal = {x for x in states if any(x+u in target for u in actions)}
robust = {x for x in states
          if any(all(x+u+d in target for d in disturbances)
                 for u in actions)}
assert nominal == {-2, -1, 0, 1, 2}
assert robust == {-1, 0, 1}
print("Braking margin:", margin)
print("Feasible but nonregulating MPC trajectory:", trajectory)
print("Finite robust predecessor:", sorted(robust))

The second program checks the receding-horizon calculations of the last three sections with exact rational arithmetic. Its input is the scalar plant \(x^+=2x+u\) with \(|x|\le1\) and \(|u|\le0.5\). Every set it handles is a symmetric interval stored as its radius, so pre is the exact predecessor formula \(r\mapsto(r+0.5)/2\) derived in the invariance section, and intersection with \(X\) or with the previous iterate is a minimum of radii. It verifies the maximal-invariant iterates \(r_k=0.5+2^{-(k+1)}\), the terminal-equality radii of Exercise 8, the horizon-two sets of Exercise 7 and its infeasible successor from \(x=0.6\), an independent integer check of the predecessor formula by enumerating states on a grid of step \(0.001\) and inputs on a grid of step \(0.01\) for the radii \(1\), \(0.75\), and \(0.5\), the closed loop with \(X_f=C\) and its escape from \(0.01\), and the flat one-norm objective at \(x=0.1\). The last check uses an input grid of step \(0.01\); because the objective is piecewise linear with breakpoints \(-0.2\) and \(0\) on the grid, the grid minimizers are exactly the grid points of the continuous minimizing interval. Any failed assertion stops the chapter check, and the program prints the first iterate radii and the closed-loop trajectory.

from fractions import Fraction as F

half = F(1, 2)

def pre(r):
    return (r+half)/2

omega = [F(1)]
for _ in range(8):
    omega.append(min(omega[-1], pre(omega[-1])))
assert all(r == half+F(1, 2**(k+1)) for k, r in enumerate(omega))
assert all(r > half for r in omega)

reach = [F(0)]
for _ in range(4):
    reach.append(pre(reach[-1]))
assert reach[1:] == [half-F(1, 2**(n+1)) for n in range(1, 5)]
assert reach[2] < F(2, 5) <= reach[3]

big1 = min(1, pre(F(1)))
big0 = min(1, pre(big1))
c1 = min(1, pre(half))
assert (big1, big0) == (F(3, 4), F(5, 8))
assert 2*big1-half > big1
assert c1 == min(1, pre(c1)) == half
step = [F(6, 5)+F(k, 100) for k in range(-50, 51)]
x1 = [v for v in step if abs(v) <= big1]
assert min(x1) == F(7, 10) and all(v > big0 for v in x1)

for r in (1000, 750, 500):
    ok = [x for x in range(-1000, 1001)
          if any(abs(2*x+u) <= r for u in range(-500, 501, 10))]
    assert F(ok[-1], 1000) == pre(F(r, 1000)) == -F(ok[0], 1000)

def mpc_c(x):
    lo, hi = max(-half, -half-2*x), min(half, half-2*x)
    return min(max(F(0), lo), hi)

x = F(1, 100)
path = [x]
while len(path) < 8:
    x = 2*x+mpc_c(x)
    path.append(x)
assert path[1:6] == [F(2**k, 100) for k in range(1, 6)]
assert path[6:] == [half, half]

grid = [F(k, 100) for k in range(-50, 51)]
cost = {u: F(1, 10)+abs(u)+abs(F(1, 5)+u) for u in grid}
best = min(cost.values())
assert best == F(3, 10)
assert [u for u in grid if cost[u] == best] == grid[30:51]
print("Invariant iterate radii:", [float(r) for r in omega[:4]])
print("Closed loop from 0.01:", [float(v) for v in path])

Eight exercises with complete solutions

The practical checkpoint is to state the event, horizon, information pattern, and continuation property before interpreting a positive numerical result. Chapter 12 turns the continuation intuition into a recursive-feasibility proof and adds a distinct argument for convergence.