Bounded controls, switching structure, and boundary solvers

Clipping the previous answer misses the target

Our cart starts at rest at position zero and must stop at position \(D=1\) after \(T=2.2\) seconds. Its dynamics remain \(\dot p=v,\dot v=u\), and the acceleration magnitude cannot exceed \(U=1\). The unconstrained minimum-energy acceleration is \(u_0(t)=6D(T^{-2})(1-2t/T)\). Its peak is approximately \(1.2397\), exceeding the bound.

Cutting that signal to the interval \([-1,1]\) preserves its antisymmetry, so final velocity remains zero. But it removes forward acceleration near the beginning and braking near the end. The displacement becomes smaller than one. The time profile is now feasible with respect to the actuator but infeasible with respect to the target. A projection in input space need not preserve a dynamical endpoint constraint.

The correct bounded minimum-energy profile will also look like a clipped line. The difference is that its line's slope must be chosen again, consistently with the terminal state. This chapter derives that slope and compares the resulting trajectory with minimum-time and minimum-fuel controls. Similar-looking input plots can solve different objectives.

The problem belongs to a specific branch of the course. An open-loop method seeks a control signal \(u(t)\) indexed only by time and computed from a known initial state; a closed-loop method seeks a feedback law that prescribes a control for every state the system might occupy. This chapter continues the indirect open-loop methods of chapters 3 and 4. They first derive necessary conditions for the infinite-dimensional problem and then solve the resulting differential equations, numerically when no closed form is available, so they optimize before they discretize. The direct methods of chapter 6 reverse the order: they represent the trajectory by finitely many decision variables and then optimize. Model predictive control, which re-solves an open-loop problem from each measured state, and the closed-loop methods built on dynamic programming and Hamilton–Jacobi equations come later. The bound \(|u|\le U\) is the first constraint that the indirect route must accommodate.

The Hamiltonian is minimized pointwise along a coupled trajectory 10:55

Recording: 10:55–24:04. Let \(x(t)\in\mathbb R^n\) be state, \(u(t)\in\mathcal U\subseteq\mathbb R^m\) an admissible control, and \(\dot x=f(x,u,t)\) the dynamics. Let \(\ell(x,u,t)\) be running cost and \(\lambda(t)\) the costate. In the normal minimization convention used here, \(H=\ell+\lambda^\top f\). The smooth interior condition \(H_u=0\) is replaced by

The comparison variable \(w\) is a possible input value at that time. State and costate are held at their candidate values during this minimization, but those candidate values must themselves solve the state and costate differential equations and the endpoint conditions. One cannot pick an arbitrary costate, minimize its Hamiltonian, and declare the resulting trajectory optimal.

The condition being replaced belongs to the unbounded problem of chapter 4. There one seeks an admissible control \(u^\star\) whose trajectory \(x^\star\) of \(\dot x=f(x,u,t)\) minimizes

with the initial time \(t_0\) and the initial state \(x(t_0)\) fixed, a twice continuously differentiable terminal cost \(\phi\), and neither state nor input confined to a bounded region. Its necessary conditions are \(\dot x^\star=H_\lambda\), \(\dot\lambda^\star=-H_x\), and \(H_u=0\) on \([t_0,T]\), all evaluated at \((x^\star(t),u^\star(t),\lambda^\star(t),t)\), together with the terminal relation

Here \(\delta x_f\) and \(\delta T\) are the admissible variations of the terminal state and the terminal time, and \(\phi_x,\phi_T\) are the partial derivatives of the terminal cost. Whatever the problem leaves free must have a vanishing coefficient: a free terminal state with fixed time gives \(\lambda^\star(T)=\phi_x\), and a fixed terminal state with free time gives \(H(T)+\phi_T=0\). The minimum principle keeps both differential equations and this terminal relation unchanged. Only \(H_u=0\) is replaced, by the pointwise minimization displayed above.

The official slides write the same problem with \(h\) for the terminal cost, \(g\) for the running cost, \(\mathbf p\) for the costate, \(t_f\) for the final time, and \(U\) for the admissible input set. This chapter keeps \(\phi\), \(\ell\), \(\lambda\), \(T\), and \(\mathcal U\) from chapter 4, because \(p\) already denotes the cart's position and \(U\) its acceleration bound. A slide expression such as \(\mathbf p^\top\mathbf b_i\) is therefore the switching coefficient \(\sigma_i=\lambda^\top b_i\) defined in the next section.

Bounds enter practical problems in two ways. Input constraints usually model actuation limits: a thruster or motor cannot deliver arbitrarily large force, which is the origin of the cart's \(|u|\le U\). State constraints usually encode safety requirements, such as keeping a vehicle outside an obstacle. This chapter treats only input bounds. A state constraint restricts the trajectory rather than the value chosen at each instant, which makes the necessary conditions substantially more complicated, and indirect methods are seldom the preferred tool for it. A common practical route plans a geometric path with simplified dynamics and uses that plan to inform the trajectory computation; the direct methods of chapter 6 impose state constraints as inequalities at discrete times.

Why a bound invalidates \(H_u=0\) is clearest in finite dimensions. For a differentiable function \(G:\mathbb R^n\to\mathbb R\), the increment satisfies \(G(y+\Delta y)-G(y)=\nabla G(y)^\top\Delta y+o(\|\Delta y\|)\). A local minimizer \(y^\star\) requires a nonnegative increment for every small admissible displacement \(\Delta y\). If \(y^\star\) is interior, both \(\Delta y\) and \(-\Delta y\) are admissible, so the linear term must be nonnegative in both directions, which forces \(\nabla G(y^\star)=0\). If \(y^\star\) lies on the boundary, some displacements leave the feasible set and only inward ones are tested. Minimizing \(G(y)=y\) on \([0,1]\) gives \(y^\star=0\) with \(G'(0)=1\ne0\): only \(\Delta y\ge0\) is admissible, and the surviving condition \(G'(0)\,\Delta y\ge0\) holds.

The same argument applies to the cost functional. A control \(u^\star\) gives \(J\) a relative minimum if \(\Delta J=J(u)-J(u^\star)\ge0\) for every admissible control \(u\) sufficiently close to \(u^\star\). Writing \(u=u^\star+\delta u\), where the variation \(\delta u\) is itself a function of time, the increment separates as \(\Delta J(u^\star,\delta u)=\delta J(u^\star,\delta u)+\text{higher-order terms}\), with the first variation \(\delta J\) linear in \(\delta u\). The variation \(\delta u\) is arbitrary only if \(u^\star(t)\) lies strictly inside the bounds at every time in \([t_0,T]\); then \(\delta u\) and \(-\delta u\) are both admissible, \(\delta J=0\) follows, and chapter 4's route to \(H_u=0\) applies. When \(u^\star\) rests on a bound during a subinterval, admissible variations there point only inward, so an admissible \(\delta u\) can have an inadmissible negative \(-\delta u\). Every bounded cart solution worked numerically in this chapter (\(T=2.2\)) rests on its bound during a subinterval, although a bound can also remain inactive throughout. The necessary condition weakens to

For a problem with fixed final time and free final state, such as the damped example of the next section, this inequality becomes pointwise. When the state and costate equations hold and \(\lambda(T)=\phi_x\), the variation of chapter 4's costate-augmented objective reduces to \(\delta J=\int_{t_0}^{T}H_u^\top\delta u\,dt\). Take \(w\in\mathcal U\) with \(\mathcal U\) convex and place the variation \(\delta u=\epsilon(w-u^\star)\), admissible for \(0\lt\epsilon\le1\) by convexity, on a short interval beginning at \(t\) and zero elsewhere. Dividing \(\delta J\ge0\) by \(\epsilon\) and by the interval length, then letting both shrink, gives \(H_u(t)^\top(w-u^\star(t))\ge0\) at almost every \(t\). This is the variational inequality stated below. For a scalar interval bound it reads \(H_u\le0\) where \(u^\star\) sits at the upper bound, \(H_u\ge0\) at the lower bound, and \(H_u=0\) strictly inside, which recovers the unbounded condition wherever the bound is inactive. In the next section's saturation law, \(u^\star=-1\) when \(\lambda_2\gt1\), and there \(H_u=u^\star+\lambda_2=\lambda_2-1\gt0\), as the lower-bound case requires. With a constrained terminal state the same pointwise conclusion holds, but deriving it requires the existence of a nontrivial multiplier, which belongs to the full proof.

The minimization is global over admissible input values at each time. It does not make the entire trajectory a global optimum. Pontryagin's principle provides necessary conditions under its hypotheses. For measurable controls the conditions are normally understood almost everywhere; isolated switching instants do not determine integral costs or state trajectories. The recording motivates the stronger condition through one-sided variations. A full proof uses more than the statement that a directional derivative is nonnegative: short-duration needle variations can replace the input by a distant admissible value.

Liberzon's formal statement includes a nontrivial objective/costate multiplier pair, allowing abnormal cases in which the objective multiplier vanishes. We use the normal case with objective multiplier one for the explicit formulas below. His maximum convention is equivalent after a consistent sign change; the word “maximum” does not mean he is maximizing the original cost.

Bounds do not imply that an optimum must touch them. If the unconstrained optimum is feasible, a bound may remain inactive throughout. Conversely, when an optimum is on the boundary, the derivative of \(H\) with respect to \(u\) need not be zero. For a differentiable convex Hamiltonian on a convex input set, its minimizer obeys \(H_u^\top(w-u^\star)\ge0\) for every feasible \(w\). This variational inequality is enough in that convex input subproblem; the nonconvex case requires the full minimization statement.

The state and costate equations remain \(\dot x=H_\lambda\) and \(\dot\lambda=-H_x\). The terminal boundary relation remains the one derived in chapter 4. An autonomous Hamiltonian is constant along suitable extremals, but free time implies a zero value only when the endpoint condition has no additional explicit time-cost contribution. The previous chapter's \(\phi=\alpha T^2/2\) gives \(H=-\alpha T\), a direct counterexample to an unqualified “free time means zero Hamiltonian” rule.

The source's bounded quadratic example produces a saturation law 24:04

Recording: 24:04–40:37. The lecture considers \(\dot x_1=x_2,\dot x_2=-x_2+u\), with cost \(\tfrac12\int(x_1^2+u^2)\,dt\), fixed duration, free terminal state, and \(|u|\le1\). Its Hamiltonian is

Only \(u^2/2+\lambda_2u\) depends on the input. Completing the square gives \(\tfrac12(u+\lambda_2)^2-\lambda_2^2/2\). Its unconstrained minimizer is \(-\lambda_2\). The constrained minimizer is its projection onto the interval:

This derivation explains the shape as a function of costate. It does not give the costate as a function of time. To finish the source problem, one must solve \(\dot\lambda_1=-x_1\), \(\dot\lambda_2=-\lambda_1+\lambda_2\), the two state equations, the specified initial state, and \(\lambda(T)=0\) because terminal state is free with no terminal penalty. The source explicitly stops at the profile rather than claiming a completed trajectory solve.

A useful generalization assumes control-affine dynamics \(f=a(x,t)+B(x,t)u\), with column \(b_i\) of the \(n\)-by-\(m\) matrix \(B\). Define the switching coefficient \(\sigma_i=\lambda^\top b_i\). For a diagonal positive-definite input weight \(R\), running effort is \(\tfrac12u^\top Ru\), and independent box bounds \(M_i^-\le u_i\le M_i^+\) yield

Both the diagonal weight and the product structure of the feasible input box enable coordinatewise minimization. With coupled actuator constraints or a nondiagonal weight, the input minimization remains a constrained quadratic problem but generally cannot be solved by clipping each coordinate independently.

Minimum time drives a linear Hamiltonian to an endpoint 42:52

Recording: 42:52–57:03. For minimum time, the running cost is one, so \(J=\int_0^T1\,dt=T\). With control-affine dynamics, the input-dependent part of the Hamiltonian is \(\sum_i\sigma_i u_i\). A linear function over an interval reaches its minimum at the lower endpoint when its slope is positive and at the upper endpoint when its slope is negative:

This is the source's bang-bang structure. It describes saturated actuator commands, not necessarily maximum velocity: velocity is a state resulting from the commands. If \(\sigma_i=0\), the Hamiltonian is momentarily indifferent to that input coordinate. A zero at an isolated instant can simply be a switch. A singular arc requires indifference over a nonzero time interval, where additional analysis is needed. An isolated crossing and a singular interval are not interchangeable.

For the cart, \(\lambda_p\) is constant and \(\dot\lambda_v=-\lambda_p\), so the switching coefficient \(\sigma=\lambda_v\) is affine. A nonzero affine function can cross zero at most once. In the normal minimum-time rest-to-rest problem with positive displacement, the feasible optimal pattern is \(u=U\) followed by \(u=-U\), switching at \(T/2\). Integrating gives displacement \(D=UT^2/4\), hence

The velocity bound \(v(t)\le\min(Ut,U(T-t))\) gives an independent global proof: its triangular area is the maximum possible rest-to-rest displacement in time \(T\), and the bang-bang control attains it. At \(D=U=1\), \(T_{\min}=2\). For negative displacement, reverse the input signs and replace \(D\) by \(|D|\) in the duration. General nonlinear control-affine systems can have singular arcs and more complicated switching; the cart's single-switch result relies on its special costate equations.

Linear time-invariant dynamics make the singular case tractable; Donald E. Kirk develops this case in Section 5.4 of Optimal Control Theory: An Introduction (Prentice-Hall, 1970). Let \(\dot x=Ax+Bu\) with constant matrices \(A\in\mathbb R^{n\times n}\) and \(B\) and input columns \(b_i\). The costate equation \(\dot\lambda=-A^\top\lambda\) gives \(\lambda(t)=e^{-A^\top t}\lambda(0)\), so \(\sigma_i(t)=\lambda(0)^\top e^{-At}b_i\), and its derivative of order \(l\) is \((-1)^l\lambda(t)^\top A^lb_i\). Suppose \(\sigma_i\) vanished on an interval. At an interior time \(t_1\) of that interval every derivative would vanish, so \(\lambda(t_1)\) would be orthogonal to \(b_i,Ab_i,\ldots,A^{n-1}b_i\). If the pair \((A,b_i)\) is controllable, these \(n\) vectors span \(\mathbb R^n\); then \(\lambda(t_1)=0\), and uniqueness for the linear costate equation gives \(\lambda\equiv0\). A normal extremal would then have \(H=1\), contradicting the zero-Hamiltonian condition of this free-time problem without terminal cost, and an abnormal extremal would violate the nontriviality of its multipliers. Hence, when every pair \((A,b_i)\) is controllable, a time-optimal control has no singular interval. Each \(\sigma_i\) is then a real-analytic function that is not identically zero, so it has finitely many zeros, and its input finitely many switches, on any bounded interval.

When every eigenvalue of \(A\) is real, the count sharpens: each \(u_i\) switches at most \(n-1\) times. Let \(\mu_1,\ldots,\mu_\rho\) be the distinct eigenvalues, with multiplicities \(n_1,\ldots,n_\rho\) summing to \(n\). Every entry of \(e^{-At}\) is a combination of terms \(t^le^{-\mu_jt}\) with \(l\lt n_j\), so \(\sigma_i(t)=\sum_jq_j(t)e^{-\mu_jt}\), where each \(q_j\) is a polynomial of degree below \(n_j\). Such a function, if not identically zero, has at most \(n-1\) real zeros. Multiplying by \(e^{\mu_1t}\) leaves the zeros unchanged and turns the first group into the polynomial \(q_1\). Differentiating \(n_1\) times annihilates \(q_1\) and preserves the form and degrees of the other groups, whose exponents \(\mu_1-\mu_j\) are nonzero; by Rolle's theorem each differentiation loses at most one zero. If the result vanishes identically, the function was \(q_1(t)e^{-\mu_1t}\) with at most \(n_1-1\) zeros; otherwise induction on \(n\) bounds its zeros by \(n-n_1-1\), and the original function has at most \(n-1\). The induction starts from a single nonzero exponential, which has no zero. A switch requires a zero of \(\sigma_i\), which proves the bound. The cart's \(A\) has the double eigenvalue \(0\), so \(n=2\) permits one switch, which is the transfer derived above.

Both hypotheses matter. Complex eigenvalues remove the bound: for a spring-restrained cart \(\dot p=v,\ \dot v=-p+u\), the costate equations are \(\dot\lambda_p=\lambda_v\) and \(\dot\lambda_v=-\lambda_p\), so \(\ddot\lambda_v=-\lambda_v\). The switching coefficient \(\lambda_v\) is sinusoidal and changes sign every \(\pi\) time units, so a long transfer switches many times; its pair \((A,b)\) is still controllable, and singular intervals remain impossible. If instead some pair \((A,b_i)\) is uncontrollable, \(\sigma_i\) vanishes identically for every nonzero \(\lambda(0)\) orthogonal to the controllable subspace of that pair, and along such a costate the minimum principle does not determine \(u_i\). None of these structural statements asserts that a time-optimal control exists; existence is a separate question, taken up in the section on existence and uniqueness.

Minimum fuel creates an interval in which doing nothing is optimal 57:03

Recording: 57:03–1:08:34. Fix the duration and use the effort model \(\sum_i c_i|u_i|\), with \(c_i>0\) and bounds containing zero. The input-dependent Hamiltonian separates into \(c_i|u_i|+\sigma_i u_i\). For \(u_i\ge0\), its slope is \(c_i+\sigma_i\); for \(u_i\le0\), its slope is \(\sigma_i-c_i\). Comparing both half-intervals gives

At \(\sigma_i=c_i\), every value in \([M_i^-,0]\) minimizes that scalar Hamiltonian; at \(\sigma_i=-c_i\), every value in \([0,M_i^+]\) does. Those tie cases must not be silently forced to zero. Away from ties, the pattern is bang-off-bang. “Fuel” is a modeling name for the absolute-input integral; it describes actual consumption only when that model fits the actuator.

For our positive-displacement cart with symmetric bound \(U\), an original complete solution uses \(+U\) for time \(b\), zero for \(T-2b\), and \(-U\) for the final time \(b\). The terminal velocity is zero. Summing the two acceleration-triangle areas and the coasting rectangle gives

The square root requires \(T\ge T_{\min}\). At the minimum duration the coast vanishes; with more time, less total impulse is needed. For \(D=U=1,T=2.2\), \(b\approx0.641742431\), the coast lasts approximately \(0.916515139\), and fuel cost is approximately \(1.283484861\).

Why is this globally fuel-minimal? Let \(P=\int_0^Tu_+(t)\,dt\), where \(u_+=\max(u,0)\). Rest-to-rest velocity requires the same total magnitude of negative input, so fuel is \(2P\). For a given \(P\) and magnitude limit \(U\), displacement \(\int_0^T(T-t)u(t)\,dt\) is maximized by placing positive input as early as possible and negative input as late as possible: moving an equal small impulse to an earlier positive time increases its weight, and moving negative impulse later reduces its adverse weight. The resulting extremal arrangement gives \(D\le P(T-P/U)\). On the feasible range \(0\le P\le UT/2\), that expression increases with \(P\). Solving the equality therefore supplies the smallest possible impulse and proves the formula.

Original extension: solve the bounded minimum-energy cart completely

Now minimize \(\tfrac12\int_0^Tu^2dt\) with the same cart endpoints and bound. The factor one half simplifies Hamiltonian derivatives without changing the minimizing control. In a normal extremal, the costate \(\lambda_v\) is affine, and the optimal input is its clipped negative. By symmetry, seek

Set \(a=T/2\) and \(r=U/k\), the half-width of the central unsaturated ramp. In the constrained regime, \(0<r<a\): input is \(+U\) until \(t=a-r\), decreases linearly through zero, and reaches \(-U\) at \(t=a+r\). Antisymmetry ensures zero total impulse and terminal velocity. The remaining endpoint requirement determines \(r\), rather than allowing us to reuse the unbounded slope.

Pairing equal times on opposite sides of the midpoint gives

Therefore \(r=\sqrt{3(a^2-D/U)}\) and \(k=U/r\). The regime \(0<r<a\) corresponds to

At the upper threshold, \(r=a\), the saturated intervals disappear and the unconstrained affine input becomes feasible. At the lower threshold, \(r\downarrow0\), the ramp becomes a switch and the trajectory approaches the unique minimum-time bang-bang transfer. The finite normal-costate formula is for durations strictly above the lower threshold; it should not be evaluated by dividing by zero at that limiting feasibility boundary.

The squared-input effort, without the one-half factor, is

For \(D=U=1,T=2.2\), this yields \(r\approx0.793725393\), slope \(k\approx1.259881577\), and effort \(E^\star\approx1.141699476\). The original unbounded slope \(12D/T^3\approx1.126972202\) is too shallow after clipping. Its clipped displacement is approximately \(0.947546519\), explaining the opening failure.

A global certificate follows from convexity. Choose \(\lambda_v(t)=-k(T/2-t)\). The proposed input minimizes \(\tfrac12u^2+\lambda_v(t)u\) pointwise on the bound. Any alternative feasible input differs by \(\delta u\) with \(\int\delta u\,dt=0\) and \(\int(T-t)\delta u\,dt=0\), preserving final velocity and position. Because \(\lambda_v\) is affine in time, \(\int\lambda_v\delta u\,dt=0\). Integrating the pointwise Hamiltonian inequality therefore gives no smaller total effort for any feasible alternative. Strict convexity makes the optimal input unique up to changes on measure-zero sets.

The minimum-fuel bang-off-bang input at this duration has squared effort \(U\int|u|dt\approx1.283484861\), larger than the minimum-energy answer. It uses less absolute impulse but concentrates that input at the bound. The cost function chooses the shape; neither “gentle” nor “fast” substitutes for writing the objective.

Clipping depends on the metric and on the constraints 68:34

Recording: 1:08:34–1:10:40. The source's multivariable minimum-energy law assumes diagonal \(R\). To see why, take \(R=\begin{bmatrix}2&1\\1&2\end{bmatrix}\), switching vector \(\sigma=(-4,0)^\top\), and box \([-1,1]^2\). The unconstrained minimizer of \(\tfrac12u^\top Ru+\sigma^\top u\) is \((8/3,-4/3)^\top\). Coordinatewise clipping gives \((1,-1)^\top\), with cost \(-3\).

At \(u_1=1\), minimizing over \(u_2\) instead gives derivative \(1+2u_2=0\), hence \(u_2=-1/2\). The point \((1,-1/2)^\top\) has cost \(-3.25\). Its gradient is \((-2.5,0)^\top\), consistent with the upper-bound multiplier \(2.5\) on the first coordinate and no active bound on the second. Strict convexity certifies it globally. The correct operation is projection in the \(R\)-weighted geometry, not ordinary componentwise clipping.

There are therefore two separate reasons a clipping shortcut can fail: it can solve the wrong instantaneous metric, or it can use costates from a different trajectory problem. The scalar bounded-cart derivation avoided both by minimizing the correct Hamiltonian and solving the endpoint relation again.

Necessary conditions presuppose an optimum that may not exist

Every argument so far assumed that an optimal control exists and asked what it must satisfy. Neither existence nor uniqueness is guaranteed in general. Two scalar minimum-fuel problems with free final time and the bound \(|u|\le1\) separate the two failures. In both, a scalar state \(x(t)\) starts at \(x_0\ne0\) and must reach the origin at some final time \(T\) that is not prescribed, and the cost is \(J=\int_0^T|u|\,dt\). The first has dynamics \(\dot x=u\); the second has \(\dot x=-x+u\).

For \(\dot x=u\), integrating the dynamics gives \(\int_0^Tu\,dt=-x_0\), hence \(J\ge\bigl|\int_0^Tu\,dt\bigr|=|x_0|\), with equality exactly when \(u\) never takes the sign of \(x_0\). Take \(x_0=1\). The control \(u=-1\) on \([0,1]\), the control \(u=-1/2\) on \([0,2]\), and coasting with \(u=0\) on \([0,4]\) followed by \(u=-1\) on \([4,5]\) all reach the origin at the lower-bound cost \(1\). The optimal cost is unique, but the optimal control is not, and neither is the optimal final time: every \(T\ge1\) is the final time of some optimal control.

The minimum principle registers this degeneracy as a tie. The Hamiltonian \(H=|u|+\lambda u\) does not depend on \(x\), so the costate is constant, and free final time with autonomous data and no terminal cost requires \(H=0\) along an extremal. If \(|\lambda|\lt1\), the minimizer is \(u=0\) and the state never moves. If \(|\lambda|\gt1\), the minimizer is \(u=-\operatorname{sign}\lambda\) with \(H=1-|\lambda|\lt0\), which violates \(H=0\); an abnormal extremal, with \(H=\lambda u\), fails the same test because its minimum value is \(-|\lambda|\ne0\). Only \(\lambda=\pm1\) satisfies \(H=0\); \(\lambda=-1\) admits only \(u\in[0,1]\), which cannot move \(x_0=1\) toward the origin, so \(\lambda=1\) remains, and then \(|u|+u=0\) for every \(u\in[-1,0]\). This is precisely the tie case \(\sigma=c\) of the minimum-fuel law and of Exercise 1. The pointwise condition admits every nonpositive control, and the endpoint requirement \(\int_0^Tu\,dt=-1\) then selects the entire optimal family rather than one member.

For \(\dot x=-x+u\), variation of constants gives \(x(T)=e^{-T}\bigl(x_0+\int_0^Te^tu(t)\,dt\bigr)\), so reaching the origin requires \(\int_0^Te^tu\,dt=-x_0\). Since \(e^t\le e^T\) on the interval, \(|x_0|\le e^T\int_0^T|u|\,dt\), and every admissible transfer satisfies

Yet the cost can be made arbitrarily small. With \(x_0=1\), coast with \(u=0\) for a duration \(\tau_c\), during which the decay brings the state to \(e^{-\tau_c}\), and then apply \(u=-1\). The state \(\bigl(e^{-\tau_c}+1\bigr)e^{-(t-\tau_c)}-1\) reaches zero after a burn of duration \(\tau_b=\ln\bigl(1+e^{-\tau_c}\bigr)\), which is also the cost. For \(\tau_c=0,1,5,10\), the costs are approximately \(0.693147\), \(0.313262\), \(0.006715\), and \(4.5399\times10^{-5}\); each exceeds its lower bound \(e^{-T}\), for instance \(0.006693\) at \(\tau_c=5\). The infimum over admissible controls is therefore \(0\), but every control that reaches the origin pays a positive cost, so no control attains the infimum. Waiting longer always lowers the cost, and no finite final time is best. Exercise 7 shows that the minimum principle has no solution here at all, neither normal nor abnormal.

That outcome is consistent with what the principle claims. It constrains an optimum that exists; for a problem without one it may return no candidate, as here, or candidates that are not optimal. Existence must be established separately, either from a general theorem whose hypotheses are checked for the problem or, as in this chapter's cart proofs, by a direct argument that bounds the cost of every feasible control and exhibits one that attains the bound. The second example shows what such hypotheses must exclude: its input set is compact and its dynamics linear, but the final time is unbounded, and the cost keeps decreasing as the time grows. Uniqueness is likewise separate. Strict convexity gave the bounded minimum-energy cart a unique optimal input, whereas the absolute-value cost of the first example is convex but not strictly convex, and its optimal set is a continuum.

A boundary solver returns a solution of equations, not an optimality certificate 70:06

Recording: 1:10:06–1:17:58. Indirect shooting guesses missing initial values, integrates forward, and adjusts the guess using terminal residuals. Indirect collocation represents state and costate functions on a mesh and imposes their differential equations and endpoint conditions together. Both are numerical routes to a boundary-value solution. Neither alone distinguishes a minimizing extremal from another stationary solution.

The recording introduces SciPy's solve_bvp. Its inputs include a right-hand-side function, a boundary-residual function, an increasing mesh, and an initial guess of all dependent variables at mesh points. The implementation uses fourth-order collocation with residual control. A returned success flag concerns its numerical convergence test, not optimality of an underlying control problem.

The solver accepts a two-point problem in standard form: a first-order system \(\dot z=F(z,t,q)\) for \(z(t)\in\mathbb R^{n_z}\) on a known interval \([t_0,T]\), with boundary residuals \(r_{\mathrm{bc}}\bigl(z(t_0),z(T),q\bigr)=0\) that involve the solution only at the two ends. The vector \(q\in\mathbb R^{n_q}\) collects unknown constant parameters and is passed as the optional fifth argument p. The official slides call these parameters \(\mathbf p\) and the right-hand side \(g\), symbols that the same deck also uses for the costate and the running cost. The parameters are not optimized. They are further unknowns, determined together with \(z\) by the boundary residuals, and the problem is square only when the residual function returns \(n_z+n_q\) equations. An indirect method fits this form after the control has been eliminated through the minimum condition: \(z\) stacks state and costate, so \(n_z=2n\), and \(r_{\mathrm{bc}}\) collects the initial state and the terminal relations. A free final time does not fit directly, because the interval itself is unknown. Rescaling time to \([0,1]\) and carrying the duration either as a parameter or as an extra state with zero derivative removes the obstacle, as chapter 6 develops for the free-time cart. U. M. Ascher and R. D. Russell, “Reformulation of boundary value problems into ‘standard’ form,” SIAM Review 23(2), 238–254 (1981), collect conversions of this kind for other nonstandard conditions.

The fourth-order collocation scheme is the three-stage Lobatto IIIA formula. SciPy's implementation represents the solution as a continuously differentiable cubic spline whose derivative equals \(F\) at every mesh node and at the midpoint of every mesh interval. On an interval \([t_j,t_{j+1}]\) of width \(h_j\), with \(F_j\) the right-hand side at the node \(t_j\) and \(F_{j+1/2}\) its value at the spline's midpoint, the collocation residual is \(z_{j+1}-z_j-\tfrac{h_j}{6}\bigl(F_j+4F_{j+1/2}+F_{j+1}\bigr)\): Simpson's rule applied to the differential equation, with its three stages at the two ends and the midpoint. Residual control is a separate mechanism. After a collocation solve, the solver estimates how far the spline's derivative departs from \(F\) between the collocation points and adds mesh nodes where that departure exceeds the tolerance.

The source's demonstration solves \(\dot z_1=z_2,\dot z_2=-|z_1|\), with \(z_1(0)=0,z_1(4)=-2\). Here \(z_1,z_2\) are simply boundary-value variables, not automatically a state/costate pair. Boundary residuals are \((z_1(0),z_1(4)+2)^\top\). The derivative \(z_2'=-|z_1|\le0\) makes \(z_2\) nonincreasing, a useful independent qualitative check.

An original analytic audit reveals two solution branches. The negative branch is \(z_1(t)=-2\sinh(t)/\sinh(4)\), with \(z_2=z_1'\). It remains negative after the initial point, so \(z_1''=z_1=-|z_1|\). A branch that first rises above zero is

Value and first derivative match at \(t=\pi\), and both pieces satisfy the equation and endpoint conditions. Different initial guesses can lead the solver to different branches. A visually plausible curve and a small boundary residual are therefore insufficient to establish uniqueness. In this demonstration there is not even an objective to minimize: it is an equation-solving example.

Executable branch checks with independent closed forms

The original code below reproduces the equation-solving task using two initial guesses, then compares each returned curve with its separately derived analytic branch. It requires NumPy and SciPy, has no file or network side effects, and raises an error if convergence or accuracy checks fail. The denser evaluation mesh is distinct from the adaptive solver mesh.

import numpy as np
from scipy.integrate import solve_bvp

def rhs(t, z):
    return np.vstack((z[1], -np.abs(z[0])))

def boundary(za, zb):
    return np.array([za[0], zb[0] + 2])

mesh = np.linspace(0, 4, 41)
dense = np.linspace(0, 4, 1001)
for branch in ["negative", "positive"]:
    if branch == "negative":
        guess = np.vstack((-mesh / 2, -np.ones_like(mesh) / 2))
        exact = -2 * np.sinh(dense) / np.sinh(4)
    else:
        guess = np.vstack((2 * np.sin(mesh), 2 * np.cos(mesh)))
        A = 2 / np.sinh(4 - np.pi)
        exact = np.where(dense <= np.pi, A * np.sin(dense),
                         -A * np.sinh(dense - np.pi))
    result = solve_bvp(rhs, boundary, mesh, guess,
                       tol=1e-7, max_nodes=5000)
    assert result.success, result.message
    values = result.sol(dense)
    assert np.max(np.abs(boundary(values[:, 0], values[:, -1]))) < 1e-8
    assert np.max(np.abs(values[0] - exact)) < 1e-6
    assert np.max(np.diff(values[1])) < 1e-7
    print(branch, "max position error:", np.max(np.abs(values[0] - exact)))

The checked run obtained maximum position errors below \(4\times10^{-9}\) on this dense comparison grid for both branches. Those are observed finite-grid errors, not rigorous uniform error bounds. The supporting numerical checks also compare bounded Hamiltonian minimizers against a dense input grid and verify the cart's displacement, impulse, and effort formulas. A future change to solver settings or initial guesses must be checked again rather than inheriting these results automatically.

Exercises with full solutions

Structure first, trajectory second, certificate third

The source's archetypes give useful control structure: bang-bang for a linear input Hamiltonian, bang-off-bang for absolute-input cost, and saturation for separable quadratic cost. To obtain a trajectory, those laws must be coupled back to the state–costate boundary problem. To establish optimality, one must then apply an appropriate sufficiency argument or explicitly retain the result as a candidate.

Our bounded-cart examples permit global certificates because their dynamics and endpoint constraints are linear and their objectives are convex or admit a direct extremal-area argument. The general nonlinear problem need not share those properties. State constraints were explicitly deferred in this recording; adding them to an indirect formulation requires more than inserting a clipping function. The next recording finishes free-time numerical reformulation and begins direct methods, where the trajectory is represented by finite decision variables before optimization.

The chapter follows a complete local machine transcript checked against official lecture-5 slides. Source-slide existence/nonuniqueness examples and free-time rescaling were not delivered before the recording ended and are not attributed to it here. Independent bounded-cart solutions, global proofs, tie-case analysis, the coupled-weight counterexample, and two analytic solver branches are original teaching additions. See the section evidence map. The official slides add the course placement of indirect methods, the statement of the unbounded problem with its full terminal relation, the one-sided-variation argument, the reference to Kirk for linear time-optimal problems, the existence and uniqueness examples, and the solver's standard form with unknown parameters and Lobatto collocation; the existence and uniqueness section rests on the slides alone, and their free-time reformulation is developed in chapter 6.