Active constraints and variations of a trajectory

An acceleration bound changes which improvement is possible

In the preceding chapter, a cart started at rest, used acceleration \(a\) for one second and \(-a\) for the next, and stopped at position \(a\). With position measured in meters and time in seconds, the normalized objective was

Here \(a\) is the normalized acceleration, the first term measures effort, and the second penalizes terminal position error. The unconstrained optimum \(a=1/2\) has cost one. Now impose a physical acceleration magnitude limit \(U>0\): \(-U\le a\le U\). If \(U=1/4\), the best feasible answer is \(a=1/4\), whose cost is \(1.25\). Its derivative \(F'(1/4)=-2\) is not zero. Increasing \(a\) would improve the objective, but that direction violates the upper bound.

This is not the same as missing a stationary point. The minimum lies where the objective pushes against a constraint. For \(U>1/2\), neither bound is active at the optimum; for \(U=1/2\), the upper bound is active but exerts no first-order cost pressure; for \(U<1/2\), it is active and costly. The distinction between “touching a bound” and “paying for a bound” will be encoded by a multiplier.

The bounded two-pulse problem is still finite-dimensional. Later in this chapter we let the acceleration vary continuously, so the optimization chooses a whole signal. We will retain the same principle—no admissible first-order improvement—while changing the mathematical space in which a direction lives.

An active set is a set of indices, not a feasible region 03:37

Recording: 3:37–14:17. Write a finite optimization problem as

The decision is \(z\), the objective is \(F\), the \(m\) equality functions are \(h_i\), and the \(r\) inequality functions are \(g_j\). Assume these functions are continuously differentiable near the candidate. For a feasible point \(z\), define the active set

An inactive inequality has \(g_j(z)<0\), leaving strict slack. An active set contains constraint indices; it is neither the region satisfying the constraints nor a list of all feasible points. For example, use \(g_1(x,y)=-x\), \(g_2(x,y)=-y\), and \(g_3(x,y)=x+y-3\). These define a closed triangle. At its origin, \(\mathcal A(0,0)=\{1,2\}\); at \((1,1)\), the active set is empty; at \((1,2)\), it is \(\{3\}\).

Why can inactive constraints be ignored in a sufficiently small local argument? If \(g_j(z^\star)<0\), continuity supplies a neighborhood in which it remains negative. There are finitely many inactive constraints, so intersect their neighborhoods. Inside that intersection, deleting them does not change feasibility. Thus a local optimum remains a local optimum of the problem with inactive inequalities removed, within an appropriately small neighborhood.

This observation does not justify deleting inactive constraints for a global solve. A step large enough to leave that neighborhood can activate one of them. Nor does it identify the active set before solving the problem. Active-set algorithms must propose, test, and revise candidates. Finally, converting the active inequalities to equalities restricts the feasible set further: an original local optimum remains optimal on that restriction, but the restricted problem also has candidates that fail the original one-sided test. The multiplier sign condition is essential for rejecting them.

Four conditions keep the geometry and the signs consistent 14:17

Recording: 14:17–23:35. Define the Lagrangian with one multiplier for each constraint:

The stationarity condition follows from the active-set reduction of the previous section. A local minimizer \(z^\star\) of the inequality-constrained problem is also a local minimizer of the equality-constrained problem that keeps the equalities \(h_i(z)=0\), replaces each active inequality by the equality \(g_j(z)=0\) for \(j\in\mathcal A(z^\star)\), and drops the inactive inequalities. If the gradients \(\nabla h_i(z^\star)\) and \(\nabla g_j(z^\star)\), \(j\in\mathcal A(z^\star)\), are linearly independent, the Lagrange multiplier theorem for equality constraints applies to that reduced problem and supplies multipliers \(\lambda_1,\ldots,\lambda_m\) and \(\mu_j\), \(j\in\mathcal A(z^\star)\), with

Setting \(\mu_j=0\) for every inactive index \(j\notin\mathcal A(z^\star)\) extends the second sum to all \(r\) inequalities without changing its value. The result is the stationarity condition \(\nabla_z\mathcal L=0\) stated below, together with complementary slackness. This step does not give \(\mu_j\ge0\). The reduced problem treats each active constraint as an equality, and an equality multiplier may have either sign, so the reduction alone cannot distinguish a minimizer that presses against a bound from a point at which moving into the feasible region would lower the cost. Nonnegativity needs a separate sign argument that uses the one-sided feasible directions of the original inequalities; the sign example below shows what fails without it.

The equality multipliers \(\lambda_i\) are unrestricted in sign; the inequality multipliers \(\mu_j\) will be nonnegative under the convention \(g_j\le0\). If \(z^\star\) is a local minimum and the gradients of all equalities together with the active inequalities are linearly independent, the Karush–Kuhn–Tucker conditions require multipliers satisfying

The stated linear-independence constraint qualification is a sufficient regularity condition for necessity and gives unique multipliers at a given candidate. Other constraint qualifications exist; necessity should not be asserted without one or another suitable hypothesis. The chapter-2 example \(h(s)=s^2=0\) already showed how a degenerate representation can defeat multiplier necessity.

Complementary slackness says an inactive inequality must have zero multiplier. Its converse is false: an active constraint can also have zero multiplier. At the cart threshold \(U=1/2\), the optimizer touches the upper acceleration bound but would choose the same acceleration if that bound were removed. Zero multiplier captures this first-order indifference.

To understand the sign, minimize \(F(s)=s\) subject to \(g(s)=-s\le0\), so \(s\ge0\). At the optimum zero, stationarity is \(1-\mu=0\), giving \(\mu=1\). Reverse the objective to \(-s\). Treating the boundary as an equality still gives the candidate zero, but stationarity now requires \(\mu=-1\). This negative multiplier rejects the candidate, correctly: moving into the feasible half-line lowers the objective. Equality tangency alone cannot distinguish those cases.

There is no rule that at least one inequality must be active at a minimum. With an empty active set and no equalities, all multipliers vanish and KKT reduces to \(\nabla F=0\). This is exactly how an interior optimum is included. Conversely, KKT points need not be minima in nonconvex problems: minimizing \(-s^2\) over \([-1,1]\) gives a stationary interior candidate at zero, but the true minima occur at the endpoints.

Work both cases, then certify what survives 25:45

Recording: 25:45–33:45. The lecture's example minimizes \(F(x,y)=x^2+y^2\) subject to \(g(x,y)=2x+y-2\le0\). The gradients are \(\nabla F=(2x,2y)^\top\) and \(\nabla g=(2,1)^\top\). If the constraint is active, stationarity gives \(x=-\mu\) and \(y=-\mu/2\). Substitution into \(2x+y=2\) yields \(-5\mu/2=2\), hence \(\mu=-4/5\). That violates dual feasibility, so the active candidate is rejected.

If the constraint is inactive, complementary slackness forces \(\mu=0\), then stationarity gives \((x,y)=(0,0)\). This point has strict slack \(g=-2\) and objective zero. Since a sum of squares is nonnegative everywhere, it is immediately the unique global minimizer. The source's active/inactive calculation is a candidate filter; the nonnegativity argument supplies a separate certificate.

For the bounded cart, let \(g_+(a)=a-U\) and \(g_-(a)=-a-U\), with multipliers \(\mu_+,\mu_-\). Since the desired displacement is positive and \(U>0\), the optimum is nonnegative and the lower bound is inactive. Stationarity is

If \(U\ge1/2\), choose \(a^\star=1/2\) and both multipliers zero. If \(0<U<1/2\), choose \(a^\star=U\), \(\mu_-=0\), and \(\mu_+=4-8U>0\). This verifies every KKT condition. The optimum is the clipped scalar answer \(a^\star=\min(1/2,U)\), but the clipping is justified by this one-dimensional convex geometry; arbitrary componentwise clipping of a multivariable equality-constrained answer generally breaks constraints.

Here KKT is also sufficient. More generally, suppose \(F\) and each \(g_j\) are convex, each equality is affine, and a feasible point satisfies KKT. Its Lagrangian is convex in \(z\), so stationarity makes that point a global minimizer of \(\mathcal L\). For any other feasible \(y\), nonnegative multipliers and \(g_j(y)\le0\) imply \(\mathcal L(y,\lambda,\mu)\le F(y)\). Complementarity gives \(\mathcal L(z^\star,\lambda,\mu)=F(z^\star)\). Therefore

This proves global optimality. The regularity condition needed to guarantee existence of multipliers is distinct from the sufficiency proof once valid multipliers have already been found. Boyd and Vandenberghe provide the broader convex-duality framework; the proof here makes the required signs explicit.

The bounded cart's value is \(V(U)=4U^2-4U+2\) below \(U=1/2\), and \(V(U)=1\) above it. In the active regime, \(V'(U)=8U-4=-\mu_+\): expanding the available acceleration reduces cost at a rate given by the multiplier. At the threshold the first derivative joins continuously at zero, but the second derivative jumps from eight to zero. Even this simple problem shows why an active-set change can alter sensitivity.

The decision can be an entire function 33:49

Recording: 33:49–46:22. An indirect optimal-control method first derives continuous-time necessary conditions and then solves the resulting equations, often numerically. A direct method first represents the trajectory with finitely many variables and then optimizes that finite approximation. The phrases “optimize then discretize” and “discretize then optimize” describe this order of operations. Neither phrase alone guarantees numerical accuracy or global optimality.

Both routes belong to the open-loop branch of optimal control, which computes one control signal, a function of time, for one given initial condition. Indirect methods came first historically, and the course develops them before direct methods, the route most often used in current numerical practice. The closed-loop branch instead computes a policy that prescribes the control as a function of the current state; dynamic programming and the Hamilton–Jacobi–Bellman equation, developed in later chapters, are its principal tools. Model predictive control lies between the two branches: it re-solves an open-loop problem from each measured state and applies only the beginning of each plan.

The indirect route sometimes gives a more direct answer, because its necessary conditions can be solved analytically or semi-analytically for some problems; the cubic cart trajectory derived later in this chapter is one such case. A closed-form control law needs little computation when it is deployed, which matters on flight computers and other processors with limited capacity, and an indirect solution can also supply the initial guess for a direct method. The analytical route is not available for every problem. When it fails, the necessary conditions must themselves be solved numerically, with the difficulties examined in the chapter on boundary solvers. D. E. Kirk's Optimal Control Theory: An Introduction (Dover, 2004), Chapter 4, develops the functionals, increments, variations, and Euler equation of this chapter for an engineering readership.

The problem these conditions address can be stated in general form. A control signal \(u(t)\) drives a state vector \(x(t)\) through the differential equation \(\dot x(t)=f(x(t),u(t),t)\) on \([t_0,t_f]\), and the performance measure is

Here \(f\) gives the dynamics, \(\ell\) is the running cost rate, and \(\phi\) is the terminal cost. A control is admissible if it satisfies the control constraints over the whole interval, and a trajectory is admissible if it satisfies the state constraints and the boundary conditions. The task is to find an admissible control \(u^\star\) that causes the system to follow an admissible trajectory \(x^\star\) minimizing \(J\). The official course slides write \(h\) for the terminal cost and \(g\) for the running cost; this book writes \(\phi\) and \(\ell\), because \(h_i\) already names equality constraints in this chapter and \(g\) will name the integrand of the variational problem below. For the continuous cart, the state consists of position and velocity, \(f\) returns the velocity and the acceleration \(u\), the running cost is \(u^2\), there is no terminal cost, and the rest-to-rest conditions admit only trajectories that start and stop at the required positions with zero velocity.

For a function of finitely many variables, an unconstrained local minimizer is found by setting the gradient to zero, which produces candidate stationary points, and then examining second- or higher-order derivatives to decide whether each candidate is a minimum, a maximum, or neither. The calculus of variations generalizes this procedure to functionals, real-valued maps whose argument is a whole function, by examining variations, which are perturbations of that argument function. The development below follows the finite-dimensional sequence step by step. It defines functionals, closeness, and local extrema, then an analog of the differential, and then a stationarity condition, the fundamental theorem of the calculus of variations, which becomes a differential equation once it is specialized to integral functionals. Applying the result to the controlled dynamics above is the subject of the next chapter.

Let \(x:[t_0,t_f]\to\mathbb R\) be a scalar function on a fixed finite interval, with \(t_0<t_f\). A functional \(J\) assigns a real number to the whole function \(x\), rather than to one sampled value \(x(t)\). For example, \(J[x]=\int_{t_0}^{t_f}x(t)\,dt\) is linear because integration respects both addition and scalar multiplication. The effort functional \(J[u]=\int_0^T u(t)^2\,dt\) is not linear: scaling \(u\) by two scales its cost by four.

A functional's domain is part of its definition. Continuous signals, continuously differentiable trajectories, and square-integrable controls are different classes. Boundary conditions further restrict which candidates are admissible. A numerical vector of samples is a representation of a function, not automatically the function itself: it may omit sharp oscillations or constraint violations between samples.

Write \(\Omega\) for the domain of \(J\), the class of admissible functions. The functional is linear if and only if

for all real numbers \(\alpha_1,\alpha_2\) and all \(x^{(1)},x^{(2)}\in\Omega\) for which \(\alpha_1x^{(1)}+\alpha_2x^{(2)}\) also lies in \(\Omega\). The integral functional above satisfies this definition because the integral of a linear combination is the same combination of integrals; the choice \(\alpha_2=0\) isolates homogeneity, and \(\alpha_1=\alpha_2=1\) isolates additivity. The closure qualification matters once boundary conditions enter. If \(\Omega\) contains only functions with \(x(t_0)=x_0\) and \(x(t_f)=x_f\), and at least one of these prescribed values is nonzero, then the sum of two admissible functions violates that condition, so \(\Omega\) is not a linear space. The difference of two admissible functions, however, vanishes at both endpoints, and such differences do form a linear space. This is why the first variation defined below is linear in the perturbation rather than in the candidate trajectory.

Our continuous cart chooses acceleration \(u(t)\) over \(0\le t\le T\), where \(T>0\) is the duration. Dynamics determine velocity and position. Without actuator bounds, the minimum-energy rest-to-rest solution derived later in this chapter uses a linearly varying acceleration, rather than two constant pulses. Allowing a richer class of controls can reduce the optimum because it enlarges the feasible set. It does not follow that every numerical refinement finds the correct continuous optimum; approximation and solver error must still be checked.

Closeness of trajectories depends on what the norm observes 46:22

Recording: 46:22–52:09. A norm assigns a nonnegative size \(\|x\|\) to a function, vanishes only for the zero function in the chosen space, scales as \(\|\alpha x\|=|\alpha|\|x\|\), and obeys \(\|x+y\|\le\|x\|+\|y\|\). The lecture gives the uniform norm on continuous functions:

The maximum exists on this compact interval because \(x\) is continuous. Distance is measured by \(\|x-y\|_\infty\). A local minimum of \(J\) means that for some radius \(\epsilon>0\), every admissible \(x\) with \(\|x-x^\star\|<\epsilon\) has \(J[x]\ge J[x^\star]\). The norm therefore determines which competitors are “nearby.”

A local maximum reverses the inequality: \(J[x]\le J[x^\star]\) for every admissible \(x\) within some positive distance of \(x^\star\). A function at which \(J\) has a local minimum or a local maximum is called an extremal, and \(J[x^\star]\) is then a relative extremum of \(J\). Some texts reserve the word extremal for any solution of the Euler–Lagrange equation derived later in this chapter; the two usages differ because that equation is only a necessary condition. Here an extremal is always a local minimizer or a local maximizer.

Here is an original caution against carrying finite-dimensional intuition too far. On the normalized interval \([0,1]\), define \(x_n(t)=n^{-1}\sin(2\pi nt)\), where \(n\) is a positive integer. Then \(\|x_n\|_\infty=1/n\to0\). But \(\dot x_n(t)=2\pi\cos(2\pi nt)\), and

for every \(n\). Uniformly tiny position wiggles do not imply tiny velocity effort. If an objective depends on derivatives, the topology used for differentiability must control them appropriately. For the first-derivative functional below, we work on continuously differentiable functions with the normalized norm \(\|x\|_{C^1}=\|x\|_\infty+\|\dot x\|_\infty\). Physical units would require scaling these terms before adding them. The source's norm example introduces the idea; this additional choice makes our Taylor argument precise.

For square-integrable functions, an integral norm identifies functions that differ only on a set of measure zero. In that space, “zero norm” means zero almost everywhere, not necessarily at every individual time. We will not need that functional-analytic machinery for the smooth examples here, but it explains why the domain must accompany a norm definition.

A first variation is a linear map, not the full change 52:24

Recording: 52:24–1:04:53. For an admissible perturbation \(\eta(t)\), the full increment is \(\Delta J[x;\eta]=J[x+\eta]-J[x]\). Differentiability means there is a bounded linear map \(DJ[x]\), acting on perturbations, such that

The first variation is \(\delta J[x;\eta]=DJ[x](\eta)\). It is linear in the perturbation \(\eta\), although it generally depends nonlinearly on the base trajectory \(x\). Equivalently, along a fixed direction, \(\delta J[x;\eta]=\left.\frac{d}{d\epsilon}J[x+\epsilon\eta]\right|_{\epsilon=0}\), provided the stated differentiability holds. Existence of each directional derivative alone is a weaker claim; our smooth examples satisfy the stronger condition.

The definition is the infinite-dimensional analog of the differential. For a differentiable \(F:\mathbb R^n\to\mathbb R\), the increment satisfies \(F(z+\Delta z)-F(z)=\nabla F(z)^\top\Delta z+o(\|\Delta z\|)\), where \(\Delta z\) is a change of the decision vector and the linear term \(\nabla F(z)^\top\Delta z\) is the differential. At an unconstrained local minimizer the differential vanishes for every \(\Delta z\), which is equivalent to \(\nabla F(z^\star)=0\). The first variation plays the role of the differential, the perturbation \(\eta\) plays the role of \(\Delta z\), and the condition proved below, that the first variation vanish in every admissible direction, plays the role of the zero-gradient condition. The analogy stops short at one step. In finite dimensions a vanishing differential converts at once into finitely many equations, whereas for functionals a further argument, carried out in the next section, is needed to turn the vanishing first variation into an equation that can be solved.

Other texts, including the official course slides, write the perturbation as \(\delta x\), call it the variation of \(x\), write its time derivative as \(\delta\dot x\), and write a functional's value with parentheses, \(J(x)\). In this chapter these are \(\eta\), \(\dot\eta\), and \(J[x]\). The slides also write the increment as \(\delta J(x,\delta x)+g(x,\delta x)\,\|\delta x\|\) and call \(J\) differentiable at \(x\) when \(g(x,\delta x)\to0\) as \(\|\delta x\|\to0\). That is the definition above, apart from the boundedness of the linear map, with \(g(x,\eta)=r_x(\eta)/\|\eta\|\) for \(\eta\ne0\), and the letter \(g\) in it is unrelated to the integrand \(g\) of the next section. In the fundamental lemma of the next section, the slides write \(h\) for the continuous function that this chapter calls the residual \(r\).

The remainder is little-o of the perturbation norm; it need not be quadratic. In one dimension, \(F(s)=|s|^{3/2}\) has derivative zero at zero, with remainder \(|s|^{3/2}\). Dividing that remainder by \(|s|\) tends to zero, but dividing it by \(s^2\) diverges. The recording's informal “quadratically” phrasing should therefore be understood as “higher order than linear” unless stronger smoothness is supplied.

To make the distinction concrete, on \([0,1]\) take

The perturbation vanishes at both endpoints. Expanding the squares exactly gives

Thus the first variation is \(1/12\), while the full increment also includes a positive quadratic term. With small negative \(\epsilon\), the first term dominates and lowers cost. The straight line \(x(t)=t\) is therefore not stationary for this objective, despite satisfying the prescribed endpoint positions. An endpoint condition does not make an arbitrary interpolating curve optimal.

At an interior local extremum within an affine admissible class, every permitted direction can be taken with both signs. The scalar function \(\epsilon\mapsto J[x^\star+\epsilon\eta]\) has a local extremum at zero, so its derivative vanishes: \(\delta J[x^\star;\eta]=0\) for every such direction. With one-sided constraints, only a one-sided derivative inequality is generally available. “First variation equals zero” must not be imposed along forbidden directions.

This result is the fundamental theorem of the calculus of variations. Let \(x\) be a scalar or vector function of \(t\) in the class \(\Omega\), with the norm applied to vector functions, for instance, as the maximum over \(t\) of the Euclidean length. Let \(J\) be differentiable at a function \(x^\star\in\Omega\). Call a perturbation \(\eta\) admissible at \(x^\star\) if \(x^\star+\eta\in\Omega\), and suppose that \(x^\star\) is not constrained by any boundary of \(\Omega\), in the sense that \(x^\star+\epsilon\eta\in\Omega\) for every admissible \(\eta\) and every real \(\epsilon\). If \(x^\star\) is an extremal, then \(\delta J[x^\star;\eta]=0\) for every admissible \(\eta\). The hypothesis about boundaries concerns the boundary of the admissible class, not the endpoint conditions at \(t_0\) and \(t_f\). A fixed-endpoint class satisfies it, because adding any multiple of a perturbation that vanishes at both ends preserves the endpoint values. A class defined by an inequality such as \(|x(t)|\le1\) violates it at any candidate that touches the bound, exactly as an active inequality restricted the admissible directions in the KKT conditions.

The proof is by contradiction and uses the differentiability definition directly. Suppose that \(J[x]\ge J[x^\star]\) for every admissible \(x\) with \(\|x-x^\star\|\lt\rho\), where \(\rho\gt0\) is the neighborhood radius, and that \(\delta J[x^\star;\eta]\ne0\) for some admissible \(\eta\ne0\). For real \(\epsilon\), linearity gives \(J[x^\star+\epsilon\eta]-J[x^\star]=\epsilon\,\delta J[x^\star;\eta]+r_{x^\star}(\epsilon\eta)\). Because \(|r_{x^\star}(\epsilon\eta)|/(|\epsilon|\,\|\eta\|)\to0\) as \(\epsilon\to0\), there is a \(\bar\epsilon\gt0\) such that \(|r_{x^\star}(\epsilon\eta)|\lt|\epsilon|\,|\delta J[x^\star;\eta]|\) whenever \(0\lt|\epsilon|\lt\bar\epsilon\). Choose such an \(\epsilon\) whose sign is opposite to that of \(\delta J[x^\star;\eta]\) and which also satisfies \(|\epsilon|\,\|\eta\|\lt\rho\). The linear term then equals \(-|\epsilon|\,|\delta J[x^\star;\eta]|\) and outweighs the remainder, so the increment is negative. The function \(x^\star+\epsilon\eta\) is admissible, lies inside the neighborhood, and costs less, which contradicts the local minimum. For a local maximizer, choose the sign of \(\epsilon\) equal to that of \(\delta J[x^\star;\eta]\). The argument needs both signs of \(\epsilon\), and the boundary hypothesis is what supplies them. Kirk gives a proof by contradiction in Section 4.1 of the text cited earlier.

Integration by parts turns stationarity into a differential equation 65:31

Recording: 1:05:31–1:22:50. Consider the fixed-endpoint problem

The integrand \(g\) is twice continuously differentiable in its arguments. We take \(x\) and perturbations in \(C^1\), and assume a candidate \(x^\star\) is smooth enough that the derivatives appearing below exist continuously. Subscripts \(g_x\) and \(g_{\dot x}\) denote partial derivatives with respect to the first and second arguments, evaluated along the candidate. A perturbation \(\eta\) is admissible only if \(\eta(t_0)=\eta(t_f)=0\).

This is the simplest optimal-control problem of the general form stated earlier in this chapter. Take the dynamics \(\dot x=u\), so that choosing the control is the same as choosing the state trajectory directly. The integrand \(g(x,\dot x,t)\) is then the running cost \(\ell(x,u,t)\) with \(u\) replaced by \(\dot x\), and there is no terminal cost. Fixing \(t_0\), \(t_f\), \(x_0\), and \(x_f\) removes every endpoint effect. The next chapter relaxes these endpoint assumptions and then restores general dynamics.

Both \(J[x+\eta]\) and \(J[x]\) integrate over the same fixed interval, so their difference combines into one integral:

The fixed limits matter for this step. If the final time could also change, the two integrals would have different upper limits, and the increment would contain an additional boundary contribution, which the next chapter derives.

Taylor's theorem bounds what the linear part leaves out. At each fixed time, expand \(g\) in its first two arguments about \((x(t),\dot x(t))\):

Here \(R(t)\) is the pointwise Taylor remainder, and \(M\) bounds the absolute values of the three second partial derivatives \(g_{xx}\), \(g_{x\dot x}\), and \(g_{\dot x\dot x}\) over the set of points \((x(t)+\xi,\dot x(t)+\zeta,t)\) with real offsets satisfying \(|\xi|+|\zeta|\le1\) and \(t\in[t_0,t_f]\). The bound holds at every \(t\) with \(|\eta(t)|+|\dot\eta(t)|\le1\), because the segment along which Taylor's theorem evaluates the second derivatives then stays in this set. The set is compact because \(x\) and \(\dot x\) are continuous, and \(g\) is twice continuously differentiable, so \(M\) is finite. For \(\|\eta\|_{C^1}\le1\), the pointwise bound gives \(|R(t)|\le\tfrac M2\|\eta\|_{C^1}^2\), so the integrated remainder is at most \(\tfrac M2(t_f-t_0)\|\eta\|_{C^1}^2\). Divided by \(\|\eta\|_{C^1}\), it tends to zero. The linear part is bounded, as the definition requires: \(|\delta J[x;\eta]|\le(t_f-t_0)\max(\|g_x\|_\infty,\|g_{\dot x}\|_\infty)\,\|\eta\|_{C^1}\), where the uniform norms of \(g_x\) and \(g_{\dot x}\) along the candidate are finite by continuity. The functional is therefore differentiable in the \(C^1\) norm of the norms section, and the linear part of the expansion, displayed next, is its first variation. Under the assumption \(g\in C^2\) the remainder is bounded by a multiple of the squared norm, which is the stronger smoothness that the previous section said quadratic behavior requires.

Taylor expansion inside the integral gives

Both terms are linear in \(\eta\). The derivative perturbation \(\dot\eta\) is not an independent choice: it is the time derivative of the same function. Integration by parts moves that derivative onto its coefficient:

The bracket means the final-time value minus the initial-time value. It vanishes here because the endpoint perturbations vanish, not because the coefficient \(g_{\dot x}\) vanishes. This distinction becomes decisive when endpoints are free.

At a stationary curve, the remaining integral is zero for every smooth perturbation with zero endpoints. Let \(r(t)=g_x-dg_{\dot x}/dt\), a continuous residual. If \(r\) were positive at an interior time, continuity would make it positive throughout a small interval. A smooth nonnegative bump supported in that interval would give \(\int r\eta\,dt>0\), contradicting stationarity. A negative residual is ruled out the same way. Hence \(r=0\) in the interior, and by continuity also at the endpoints when defined there. This is the fundamental lemma of the calculus of variations in the form we need.

Stated on its own, the lemma reads as follows. If \(r\) is continuous on \([t_0,t_f]\) and \(\int_{t_0}^{t_f}r(t)\eta(t)\,dt=0\) for every continuously differentiable \(\eta\) with \(\eta(t_0)=\eta(t_f)=0\), then \(r(t)=0\) for every \(t\in[t_0,t_f]\). The class of test functions is the one that the derivation supplies: admissible perturbations are continuously differentiable and vanish at both ends. A version that assumes the integral vanishes for every continuous \(\eta\) would not apply directly here, because the derivation does not establish that hypothesis for perturbations with nonzero endpoint values without a further approximation argument; that version is also immediate, since the choice \(\eta=r\) gives \(\int r^2\,dt=0\). The restricted version needs a test function from the restricted class, and one can be written explicitly. If \(r\) is positive on an interval \((t_1,t_2)\) with \(t_0\le t_1\lt t_2\le t_f\), the function equal to \((t-t_1)^2(t_2-t)^2\) on \([t_1,t_2]\) and to zero elsewhere is continuously differentiable, because both it and its derivative vanish at \(t_1\) and \(t_2\). It is zero at \(t_0\) and \(t_f\) and positive inside \((t_1,t_2)\), so it makes the integral positive. In normalized time it has the same quartic shape, \(\tau^2(1-\tau)^2\), as the perturbation used in the numerical check later in this chapter. Applied to the residual, the lemma gives the equation displayed next, which is called the Euler–Lagrange equation or, more briefly, the Euler equation.

The resulting Euler–Lagrange equation is

It is a necessary condition for a sufficiently smooth extremum, not automatically a minimum certificate. Nor is it always a genuine second-order ODE: that interpretation requires nondegeneracy in the derivative argument. If \(g\) does not depend on \(\dot x\), the equation is algebraic. If \(g=x\dot x\), the integral is an endpoint term and the equation is identically satisfied. The lecture's common second-order interpretation applies to the regular examples, not every possible integrand.

For the earlier quadratic functional, \(g=(x^2+\dot x^2)/2\), so the equation is \(\ddot x=x\). With \(x(0)=0,x(1)=1\), solving this ODE gives \(x^\star(t)=\sinh(t)/\sinh(1)\), where \(\sinh(t)=(e^t-e^{-t})/2\). To prove it is the unique global minimum, substitute any feasible \(x=x^\star+\eta\). The cross term vanishes by the derived stationarity, leaving

Equality forces \(\eta=0\). This exact quadratic identity turns a necessary condition into a global proof. It also explains why the simple straight line lost to a curved path: the objective penalizes both the size of the state and its rate of change.

Original extension: the minimum-energy cart is a cubic trajectory

The lecture stops at the first-derivative Euler–Lagrange equation. We now extend the same integration-by-parts reasoning to the cart, whose acceleration is the second derivative of position. This is an independently derived bridge to later optimal-control lectures, not additional content attributed to this recording.

Let \(p(t)\) be position for \(t\in[0,T]\), with prescribed displacement \(D\), fixed \(T>0\), and four endpoint conditions

Minimize \(J[p]=\int_0^T\ddot p(t)^2\,dt\). We use sufficiently smooth trajectories for the derivation; the final comparison works for feasible trajectories with square-integrable second derivative. There is no acceleration bound in this optimization. Let \(\eta\) be a perturbation with both \(\eta\) and \(\dot\eta\) zero at the endpoints. The first variation is \(2\int_0^T\ddot p\,\ddot\eta\,dt\). Integrating by parts twice gives

The endpoint term vanishes, and the fundamental lemma yields \(p^{(4)}=0\). Thus the candidate is cubic. Write \(p(t)=a_0+a_1t+a_2t^2+a_3t^3\), where the \(a_i\) are coefficients. The initial conditions give \(a_0=a_1=0\). The final two conditions solve to \(a_2=3D/T^2\) and \(a_3=-2D/T^3\). With normalized time \(\tau=t/T\),

The velocity is a parabola, the acceleration decreases linearly, and the effort is

A global certificate again comes from expanding a square. Every feasible perturbation has zero endpoint position and velocity, so the same integration by parts gives \(\int_0^T\ddot p^\star\ddot\eta\,dt=0\). Therefore \(J[p^\star+\eta]-J[p^\star]=\int_0^T\ddot\eta^2\,dt\ge0\). Equality makes \(\eta\) affine; its endpoint conditions then make it identically zero. This proves uniqueness within the stated feasible class.

At \(D=1,T=2\), the continuous optimum costs \(1.5\), while the feasible equal-duration constant pulses \(u=1,-1\) cost \(2\). The richer control class saves one quarter of that pulse cost. But its peak acceleration is \(6|D|/T^2=1.5\). With an acceleration limit of one, this attractive solution is infeasible, even though the pulse construction is feasible. A global optimum of an unconstrained model is not automatically an admissible answer to the bounded model. The interactive laboratory displays this distinction as the duration and bound change.

Check the increment, the constraints, and the cost gap separately

This original example evaluates a known feasible perturbation \(\eta(t)=A\tau^2(1-\tau)^2\), where \(A\) is a signed position amplitude. Its position and velocity vanish at both endpoints. Its acceleration is \(\ddot\eta=(A/T^2)(2-12\tau+12\tau^2)\). The predicted extra effort is \(4A^2/(5T^3)\), obtained by integrating the squared polynomial. The code compares that exact result with numerical quadrature; it does not search for a minimum by trying a few curves.

import numpy as np

D, T, A = 1.0, 2.0, 1.0
tau = np.linspace(0.0, 1.0, 20001)
t = T * tau
p = D * (3 * tau**2 - 2 * tau**3)
v = 6 * D / T * tau * (1 - tau)
u = 6 * D / T**2 * (1 - 2 * tau)
eta = A * tau**2 * (1 - tau)**2
eta_dot = A / T * (2 * tau - 6 * tau**2 + 4 * tau**3)
eta_ddot = A / T**2 * (2 - 12 * tau + 12 * tau**2)
np.testing.assert_allclose([p[0], v[0], p[-1], v[-1]], [0, 0, D, 0])
np.testing.assert_allclose([eta[0], eta[-1], eta_dot[0], eta_dot[-1]], 0)
energy = np.trapezoid(u**2, t)
cross_term = np.trapezoid(2 * u * eta_ddot, t)
gap = np.trapezoid((u + eta_ddot)**2 - u**2, t)
assert abs(energy - 12 * D**2 / T**3) < 1e-7
assert abs(cross_term) < 1e-10
assert abs(gap - 4 * A**2 / (5 * T**3)) < 1e-7
print("Base effort:", energy, "extra effort:", gap)

Inputs \(D,T,A\) determine the target, duration, and comparison curve; the shown example uses a positive duration and a uniform normalized mesh. Outputs are quadrature estimates. NumPy is the only nonstandard dependency. The arrays have no side effects beyond printing. The companion Python checks repeat the identity for negative displacement, zero displacement, and signed perturbations, and verify the finite-dimensional active-set multipliers. The proof covers all admissible perturbations; these computations test particular formulas and implementation choices.

Exercises with full solutions

The common principle—and the assumptions that keep it valid

Finite KKT conditions and first-variation stationarity both express the absence of an admissible first-order improvement. Their details differ because finite inequalities produce one-sided directions, while fixed-endpoint function spaces permit signed perturbations that vanish at the endpoints. Constraint qualifications, function spaces, smoothness, and boundary conditions determine which directions the argument may use.

The recording establishes necessary conditions. This chapter supplied additional global certificates only for explicitly convex or quadratic examples. It did not prove that every stationary trajectory minimizes cost, that every discretization preserves constraints, or that the unconstrained cubic respects an actuator limit. The following lecture introduces more general endpoint conditions and constrained variational problems, providing the route to state dynamics and costates.

Source boundary: the complete local machine transcript was read sequentially and cross-checked against the end of lecture-2 slides and lecture-3 slides. Corrected technical terms include Karush–Kuhn–Tucker and calculus of variations. Empty active sets, zero active multipliers, remainder order, and differential-equation degeneracy are clarified explicitly. The source map records those distinctions and the original teaching constructions. The official lecture-3 slides add the open-loop placement and advantages of indirect methods with the Kirk reading, the general optimal-control problem, the linear-functional definition, extremals, the increment-and-remainder form of differentiability, the fundamental theorem with its proof by contradiction, the single-integral increment and its Taylor expansion, and a standalone statement of the fundamental lemma, with notation translated where it differs, while the deck's last pages, on the type of the Euler equation, the shortest-path example, and the vector Euler equation, are delivered at the start of the next recording and treated in the next chapter.