Local geometry, descent, and equality constraints

A stopped cart with a nonzero gradient

Return to the cart with position \(p\), velocity \(v\), and commanded acceleration \(u\), governed by \(\dot p=v\) and \(\dot v=u\). Start at rest at the origin. Hold one acceleration \(u_0\) for one second and another \(u_1\) for the next second. For this chapter, divide position by one meter, time by one second, acceleration by one meter per second squared, and cost by one square meter per second cubed. All displayed optimization variables are therefore dimensionless. Restoring units changes weights and derivatives; it does not change the following logic.

Exact integration, rather than Euler approximation, gives the terminal position and velocity:

Here \(z\in\mathbb R^2\) is the decision vector; \(c\) maps decisions to terminal position, and \(E\) maps them to terminal velocity. Require a stopped cart, \(Ez=0\), but penalize a missed position softly. For desired displacement \(D\in\mathbb R\) and nonnegative weight \(q\), minimize

The first term is integrated squared acceleration over the two unit intervals. The second is a terminal error penalty. There is no acceleration bound in this chapter's model; such a bound would add inequalities. The equality forces \(u_1=-u_0\). Writing \(a=u_0\) reduces the objective to \(F(a)=2a^2+q(a-D)^2\). With \(q=2\) and \(D=1\), the best feasible choice is \(a=1/2\), hence \(z^\star=(1/2,-1/2)^\top\), terminal position \(1/2\), and cost \(1\).

Now compute the gradient in the original two-dimensional decision space:

This is no contradiction. Moving in the negative-gradient direction would improve the objective by giving the cart a positive terminal velocity, violating the stop condition. Every feasible infinitesimal direction has the form \(d=(s,-s)^\top\), with scalar \(s\), and therefore \(\nabla F(z^\star)^\top d=0\). The optimizer has no first-order improvement along the feasible line. Understanding this geometric statement is the bridge from finite optimization to optimal control, where dynamics impose many coupled equality constraints.

Necessary conditions ask what a minimum cannot do 01:53

Recording: 1:53–13:44. First remove constraints. Let \(F:\mathbb R^n\to\mathbb R\) be differentiable near an interior local minimizer \(z^\star\). The integer \(n\) is the number of scalar decision variables. Interior means every sufficiently small displacement is admissible. For a direction \(d\in\mathbb R^n\) and positive scalar \(\epsilon\), differentiability gives

The remainder \(o(\epsilon)\), for fixed \(d\), divided by \(\epsilon\) tends to zero as \(\epsilon\to0\). If the gradient \(g=\nabla F(z^\star)\) were nonzero, choosing \(d=-g\) would make the leading term \(-\epsilon\|g\|_2^2\), where \(\|\cdot\|_2\) is the Euclidean norm. For sufficiently small \(\epsilon\), this negative linear term dominates the remainder and contradicts local minimality. Therefore \(g=0\). The lecture establishes the same conclusion by considering positive and negative coordinate perturbations.

The coordinate route writes the argument as a pair of one-sided inequalities. At a local minimum the first-order cost variation cannot be negative in any direction: if \(\nabla F(z^\star)^\top d\lt0\) for some \(d\), the expansion above makes \(F(z^\star+\epsilon d)\lt F(z^\star)\) for all sufficiently small \(\epsilon\). Hence, for every \(d\in\mathbb R^n\) with components \(d_i\),

Let \(\hat e_i\) be the \(i\)th unit coordinate vector of \(\mathbb R^n\). The choice \(d=\hat e_i\) leaves only the \(i\)th term and gives \(\partial F(z^\star)/\partial z_i\ge0\); the choice \(d=-\hat e_i\) gives \(\partial F(z^\star)/\partial z_i\le0\). The only number satisfying both inequalities is zero, so every partial derivative vanishes and \(\nabla F(z^\star)=0\). Both signs are essential. If only one-sided perturbations are admissible, each coordinate yields a single inequality. At the endpoint zero of the half-line \(s\ge0\), for instance, only directions \(d\gt0\) are admissible, and the conclusion weakens to \(F'(0)\ge0\), as the boundary example below shows.

A stationary point is a point with zero gradient. Stationarity is a filter, not a certificate: \(F(s)=-s^2\) is stationary at its strict maximum \(s=0\), and \(F(s)=s^3\) is stationary there without any local extremum. A constrained boundary changes the argument because the direction \(-g\) might be forbidden. For example, minimizing \(F(s)=s\) over \(s\ge0\) has minimizer zero and derivative one. The cart example makes the same point for an equality-defined feasible set rather than a one-sided interval.

If \(F\) is twice continuously differentiable, define its symmetric Hessian \(H(z)=\nabla^2F(z)\). At any point \(z\), the second-order expansion \(F(z+\epsilon d)-F(z)=\epsilon\nabla F(z)^\top d+\tfrac12\epsilon^2d^\top H(z)d+o(\epsilon^2)\) refines the first-order one by a curvature term. Its first-order term is the one the coordinate argument used; its second-order term decides the sign only once the first-order term vanishes. At a stationary point, Taylor expansion in a fixed direction yields

A negative value of \(d^\top H(z^\star)d\) would contradict minimality for sufficiently small nonzero \(\epsilon\). Thus the Hessian must be positive semidefinite, written \(H(z^\star)\succeq0\): its quadratic form is nonnegative for every direction. A negative eigenvalue provides an explicit direction that fails this test. The conditions \(g=0\) and \(H\succeq0\) remain only necessary. Both \(s^4\) and \(-s^4\) have zero first and second derivatives at zero, yet one has a minimum and the other a maximum.

When local information becomes a guarantee 13:44

Recording: 13:44–24:50. A positive definite Hessian supplies the missing strictness. Suppose \(g=0\) and \(H(z^\star)\succ0\). Let \(\mu>0\) be its smallest eigenvalue. For a small arbitrary displacement \(e\), twice continuous differentiability gives

Choose a neighborhood in which \(|r(e)|\le(\mu/4)\|e\|_2^2\). Since \(e^\top H(z^\star)e\ge\mu\|e\|_2^2\), the difference is at least \((\mu/4)\|e\|_2^2>0\) for every nonzero \(e\) in that neighborhood. This proves a strict local minimum. The proof's hinge is a uniform lower bound over all directions, not just positive curvature along a few sampled axes. Positive definiteness is sufficient but unnecessary: \(s^4\) again has a strict minimum with zero Hessian.

Convexity strengthens the conclusion from local to global. A set \(C\subseteq\mathbb R^n\) is convex when the segment \((1-\theta)x+\theta y\) lies in \(C\) for every \(x,y\in C\) and every \(\theta\in[0,1]\). A function \(F\) on that set is convex when

For a differentiable convex function on an open convex domain, consider that inequality with \(\theta>0\), subtract \(F(x)\), divide by \(\theta\), and let \(\theta\) decrease to zero. Differentiability identifies the limit, producing the supporting-plane inequality

Consequently, an unconstrained stationary point is globally optimal. More generally, if the inner product on the right is nonnegative for every feasible \(y\), \(x\) is globally optimal over that convex feasible set. For the stopped-cart problem the feasible set is a line, the objective has positive definite Hessian, and the gradient is orthogonal to every displacement along that line. This already certifies the answer globally.

The two halves of this argument combine into a single statement. Let \(C\subseteq\mathbb R^n\) be open and convex, and let \(F\) be continuously differentiable and convex on \(C\). Then a point \(z^\star\in C\) is a global minimum of \(F\) over \(C\) if and only if \(\nabla F(z^\star)=0\). Sufficiency is the supporting-plane inequality with a zero gradient: \(F(y)\ge F(z^\star)\) for every \(y\in C\). Necessity is the first-order condition of the previous section: because \(C\) is open, a global minimum over \(C\) is an interior local minimum, so every sufficiently small displacement is admissible and the gradient must vanish. Each hypothesis does separate work. Without openness, \(F(s)=s\) over \(s\ge0\) is convex and continuously differentiable, yet its minimizer zero has \(F'(0)=1\). Without convexity, \(F(s)=s^3\) on \(\mathbb R\) is stationary at zero, which is not a minimum. For the cart's reduced objective \(4(a-\tfrac12)^2+1\), convex on all of \(\mathbb R\), the single zero of \(F'(a)=8a-4\) at \(a=1/2\) is therefore the global minimizer, and no second-derivative test is needed.

A separate segment argument shows that any local minimum of a convex function over a convex set is global: a putative better point would create better points arbitrarily close to the local minimum. Strict convexity makes the inequality strict for distinct endpoints and \(0<\theta<1\), ruling out two distinct minimizers. It does not guarantee that a minimizer exists. The strictly convex function \(e^s\) on all of \(\mathbb R\) has infimum zero approached as \(s\to-\infty\), never attained. Existence, uniqueness, local optimality, and global optimality are different questions. Boyd and Vandenberghe's Convex Optimization develops this distinction and the supporting-plane characterization systematically.

A downhill direction is not a safe step length 27:04

Recording: 27:04–39:39. An iterative algorithm generates decisions \(z^{(j)}\), where \(j=0,1,\ldots\) counts iterations. Given a direction \(d^{(j)}\) and positive step length \(\alpha_j\), update \(z^{(j+1)}=z^{(j)}+\alpha_jd^{(j)}\). At the current point, write \(g=\nabla F(z^{(j)})\). A strict descent direction satisfies \(g^\top d^{(j)}<0\). Differentiability ensures improvement for all sufficiently small positive steps in that fixed direction; it says nothing about a large step.

The update is an instance of iterative descent. Starting from an initial guess \(z^{(0)}\), which may be as uninformed as the origin, the method generates \(z^{(1)},z^{(2)},\ldots\) with the aim that

Updates in which \(d^{(j)}\) satisfies \(g^\top d^{(j)}\lt0\) whenever \(g\ne0\), and in which \(\alpha_j\gt0\), form the class of gradient methods. The gradient enters through this sign test; the direction itself need not be \(-g\). Every gradient method must therefore settle two choices, the direction and the step length. The rest of this section studies the step length for the direction \(-g\), and the next section treats both choices in general.

The official course slides use different symbols for this material. They write the decision as \(x\), the objective as \(f\), the iteration counter as a superscript, as in \(x^{k+1}=x^k+\alpha^kd^k\), the positive definite scaling matrix of the next section as \(D^k\), a constant step length as \(s\), and the transpose as a prime. This chapter writes \(z\), \(F\), the parenthesized superscript \((j)\), \(M\), a constant \(\alpha\), and \(^\top\), because \(f\) denotes the linear coefficient of the quadratic objective introduced below and \(D\) the cart's desired displacement. The correspondence is one-to-one: \(x^k=z^{(j)}\), \(\alpha^k=\alpha_j\), \(f=F\), \(D^k=M\), \(s=\alpha\), and \(\nabla f(x)'d=g^\top d\).

The simplest gradient method makes the first-order decrease explicit. For \(g\ne0\), consider the half-line \(z_\alpha=z^{(j)}-\alpha g\) with \(\alpha\ge0\). Differentiability gives

so the predicted decrease is proportional to the squared gradient norm, and every sufficiently small positive \(\alpha\) lowers the objective. Nothing in the linear term limits \(\alpha\). The prediction keeps falling as \(\alpha\) grows, and only the neglected remainder, which the expansion controls for small \(\alpha\) alone, can reverse it.

For a general direction the same expansion reads \(F(z^{(j)}+\alpha d)=F(z^{(j)})+\alpha g^\top d+o(\alpha)\). Write \(g^\top d=\|g\|_2\|d\|_2\cos\omega\), where \(\omega\) is the angle between \(g\) and a nonzero \(d\). Then \(d\) is a strict descent direction exactly when \(\omega\gt90^\circ\). Because \(g\) is normal to the level set of \(F\) through \(z^{(j)}\), these are the directions that point to the side of its tangent hyperplane on which \(F\) decreases. At the cart's feasible optimum in the opening section, \(g=(-2,-2)^\top\) and the feasible direction \((1,-1)^\top\) make an angle of exactly \(90^\circ\). That right angle is the geometric form of the statement that the feasible line offers no first-order improvement.

Two boundaries accompany this framing. First, if \(g=0\), no direction satisfies the strict inequality, so a gradient method has no admissible move and stops. The stopping point is stationary, and the stationarity section showed that it may be a maximum or neither: started at \(s=0\), a gradient method for \(F(s)=-s^2\) never leaves the maximum. Second, monotone decrease is an aim rather than a guarantee of reaching a minimizer. A strictly decreasing sequence of objective values can converge to a level above the minimum, and the step-length discussion in the next section constructs such a sequence for the cart.

Our reduced cart objective provides an exact experiment. With \(q=2\) and \(D=1\),

Gradient descent with a constant \(\alpha>0\) updates \(a_{j+1}=a_j-\alpha(8a_j-4)\). Define the error \(e_j=a_j-1/2\). Subtracting the optimum from both sides gives the complete dynamics of this optimizer:

For any nonzero starting error, convergence occurs exactly when \(|1-8\alpha|<1\), or \(0<\alpha<1/4\). At \(\alpha=1/4\), the error changes sign without shrinking; the objective stays constant. At \(\alpha=0.3\), the magnitude grows by a factor of \(1.4\) each iteration. At \(\alpha=1/8\), the optimum is reached in one step because this particular objective is quadratic with constant curvature eight. The direction remains downhill before each update even when an oversized update makes the final objective worse.

These regimes separate the requirement on the direction from the requirement on the step. Most often the step length is chosen so that

and a gradient method with this property is called gradient descent in the strict sense. For the cart, \(F(a)-1=4(a-\tfrac12)^2\), so the objective gap obeys \(F(a_{j+1})-1=(1-8\alpha)^2\,(F(a_j)-1)\). Strict decrease from a nonoptimal point therefore holds exactly when \((1-8\alpha)^2\lt1\), that is, for \(0\lt\alpha\lt1/4\). The constant steps \(\alpha=1/4\) and \(\alpha=0.3\) define gradient methods that are not descent methods in this sense, even though \(-F'(a_j)\) is a strict descent direction at every nonoptimal iterate. For this quadratic the interval of strict decrease coincides with the interval of convergence. For a general objective, strict decrease at every iteration does not imply convergence to a minimizer.

For a multivariable quadratic \(F(z)=\tfrac12z^\top Hz-f^\top z+\gamma\), where \(H\succ0\), \(f\in\mathbb R^n\), and \(\gamma\) is a constant, the minimizer satisfies \(Hz^\star=f\). Constant-step gradient descent obeys \(e_{j+1}=(I-\alpha H)e_j\), where \(I\) is the identity matrix. In an orthonormal eigenbasis of \(H\), the error along an eigenvector with eigenvalue \(\lambda_i\) is multiplied by \(1-\alpha\lambda_i\). All modes contract exactly when \(0<\alpha<2/\lambda_{\max}\), with \(\lambda_{\max}\) the largest eigenvalue.

Take \(H=\operatorname{diag}(1,100)\). A step of \(0.01\) eliminates the high-curvature error immediately but multiplies the low-curvature error by \(0.99\) each time. Shrinking that remaining component by a factor of \(100\) takes at least \(459\) iterations. This is a conditioning problem, not a mysterious failure of the derivative. A coordinate rescaling can change the apparent geometry and the performance of unscaled gradient descent.

Curvature chooses a direction; line search controls the distance 39:43

Recording: 39:43–51:30. The lecture generalizes negative-gradient directions to \(d=-Mg\), where \(M\) is a symmetric positive definite matrix. Then \(g^\top d=-g^\top Mg<0\) whenever \(g\ne0\). The matrix changes the geometry in which a displacement is regarded as large. Setting \(M=I\) recovers ordinary gradient descent. Where the Hessian is positive definite, choosing \(M=H^{-1}\) gives a Newton direction, usually computed by solving \(Hd=-g\) rather than explicitly forming an inverse.

The choice \(M=I\) is called steepest descent, and the name has a precise meaning. Among directions of unit Euclidean length, the Cauchy–Schwarz inequality gives \(g^\top d\ge-\|g\|_2\), with equality only for \(d=-g/\|g\|_2\). The negative gradient therefore gives the fastest first-order decrease per unit of Euclidean step, not the most direct route to the minimizer. Steepness depends on the norm. Let \(M^{1/2}\) be the symmetric positive definite square root of \(M\), and measure steps by \((d^\top M^{-1}d)^{1/2}\). Substituting \(d=M^{1/2}\tilde d\) turns the unit-length constraint into \(\|\tilde d\|_2=1\) and the directional derivative into \((M^{1/2}g)^\top\tilde d\), so the same inequality makes \(-Mg\) the steepest direction in that norm. A positive definite scaling matrix is thus a choice of the norm in which displacements are compared. For the quadratic with \(H=\operatorname{diag}(1,100)\) of the previous section, at a point with \(z-z^\star=(1,1)^\top\), the gradient is \(H(z-z^\star)=(1,100)^\top\). The Euclidean steepest direction points almost entirely along the second coordinate, whereas the Newton direction \(-H^{-1}g=-(1,1)^\top\) points at the minimizer.

For the strictly convex quadratic above, the Newton direction is exactly \(z^\star-z\), so a unit step solves the problem. For a nonquadratic objective, this is a local quadratic-model prediction. If the Hessian is indefinite, its inverse does not generally produce a descent direction. Even with a positive definite Hessian at the current point, a unit step can leave the region where that local model is useful. Computing and solving with curvature also costs more than evaluating a gradient. One-step quadratic convergence in an ideal example is not a wall-clock claim about a large control problem.

The recording surveys exact line search, constant steps, and diminishing steps. A diminishing sequence with \(\alpha_j\to0\) and \(\sum_j\alpha_j=\infty\) avoids some obvious defects, but those two properties alone are not a convergence theorem for an arbitrary objective or direction sequence. Assumptions about regularity, direction quality, boundedness, and noise still matter. Similarly, a tiny gradient certifies approximate stationarity in the chosen coordinates, not a minimum or small distance to the optimum without additional structure.

The minimization rule, also called exact line search, chooses \(\alpha_j\) so that

It replaces the choice of step by a one-dimensional minimization, which is cheap only when that subproblem is. For a quadratic with Hessian \(H\), the subproblem is explicit: \(\psi(\alpha)=F(z+\alpha d)=F(z)+\alpha g^\top d+\tfrac12\alpha^2d^\top Hd\). When \(d^\top Hd\gt0\), setting \(\psi'(\alpha)=0\) gives \(\alpha^\star=-g^\top d/(d^\top Hd)\), which is positive for a strict descent direction. For the scalar cart, \(d=-F'(a)\) and the constant curvature eight give \(\alpha^\star=F'(a)^2/(8F'(a)^2)=1/8\) at every nonoptimal point, the one-step value of the constant-step analysis. For a nonquadratic objective the rule needs an inner iterative search, and the minimum need not exist: if \(F\) decreases without bound along the ray, no minimizing \(\alpha\) exists.

Diminishing step lengths do not guarantee descent at each iteration. Apply \(\alpha_j=1/(j+1)\) to the scalar cart from \(a_0=0\), where \(F(a_0)=2\). The error multipliers \(1-8\alpha_j\) for \(j=0,\ldots,7\) are \(-7,-3,-5/3,-1,-3/5,-1/3,-1/7,0\), so the costs \(F(a_0),\ldots,F(a_8)\) are \(2,50,442,1226,1226,442,50,2,1\). The cost rises for three iterations because the early steps exceed the stability limit \(1/4\) of the constant-step analysis, holds at \(\alpha_3=1/4\), and falls once \(\alpha_j\lt1/4\). It reaches the optimum exactly at \(j=8\) only because \(\alpha_7=1/8\) equals the reciprocal of the curvature; a diminishing sequence that never takes the value \(1/8\) approaches the optimum only asymptotically. Large early steps are the purpose of the rule, since they move a distant iterate quickly, but on this objective they first move it away.

The condition \(\sum_j\alpha_j=\infty\) guards against the opposite failure, in which the steps shrink so quickly that the iterates stall. Choose \(\alpha_j=2^{-j}/80\) for the cart from \(a_0=0\). Each multiplier \(1-8\alpha_j=1-0.1\cdot2^{-j}\) lies in \([0.9,1)\), so the error shrinks in magnitude and the cost decreases strictly at every iteration. Yet \(\sum_j\alpha_j=1/40\). For numbers \(x_i\in[0,1]\), the product inequality \(\prod_i(1-x_i)\ge1-\sum_ix_i\) bounds the total contraction factor below by \(1-8/40=0.8\). Hence \(|e_j|\ge0.4\), \(a_j\le0.1\), and \(F(a_j)\ge1.64\) for every \(j\). The iterates converge to \(a\approx0.0935\), where the cost is about \(1.661\) and the derivative about \(-3.25\): the limit is not even stationary. Together the two conditions do complementary work. Letting \(\alpha_j\to0\) makes the steps eventually smaller than any stability limit such as \(1/4\), and a divergent sum lets the iterates travel any distance the problem requires.

As an explicit extension beyond the recording's survey, consider Armijo backtracking. Fix a sufficient-decrease fraction \(\eta\in(0,1)\), a shrink factor \(\beta\in(0,1)\), and an initial trial step \(\alpha=1\). Repeatedly replace \(\alpha\) by \(\beta\alpha\) until

The right side demands a fraction of the improvement predicted by the linear model. Why does this terminate? Differentiability gives \(F(z+\alpha d)=F(z)+\alpha g^\top d+o(\alpha)\). Because \((1-\eta)g^\top d<0\), sufficiently small positive \(\alpha\) makes the remainder too small to violate the test. Geometric shrinking eventually reaches that neighborhood. This proves termination of a single search at a point with a strict descent direction. It does not by itself prove that repeated searches reach a global minimum.

For the cart at \(a=0\), the negative gradient is \(d=4\), and \(F(0)=2\). Choose \(\eta=0.1\) and \(\beta=1/2\). Trials \(\alpha=1,1/2,1/4\) give costs \(50,10,2\), respectively; their Armijo upper bounds are \(0.4,1.2,1.6\), so all fail. The next trial \(\alpha=1/8\) gives \(a=1/2\), cost \(1\), and upper bound \(1.8\), so it succeeds. The calculation exposes both the prediction and the actual measured improvement.

Three questions about these methods lie outside this chapter: under what conditions the iterates converge, and to what; when to stop; and how fast convergence occurs. The guarantees of that analysis are typically convergence to stationary points, not to minimizers, for the reasons given in the stationarity section. Common termination rules are a fixed iteration budget, a small decrease \(F(z^{(j)})-F(z^{(j+1)})\le\tau\), and a small gradient norm \(\|g\|_2\le\tau\), for a chosen tolerance \(\tau\gt0\). Each rule tests a symptom rather than optimality. On the stalled sequence above, the decrease is about \(1.3\times10^{-4}\) at iteration \(10\) and about \(1.3\times10^{-7}\) at iteration \(20\), so a decrease test with \(\tau=10^{-6}\) stops at a cost above \(1.64\) while the optimum is one. The gradient test would not stop there, since \(|F'(a_j)|=8|e_j|\ge3.2\), but a small gradient bounds the distance to a minimizer only under further structure, as noted above. Convergence rates depend on the direction, the step rule, and the starting point: the factor \(|1-8\alpha|\) per iteration and the \(459\) iterations for \(\operatorname{diag}(1,100)\) are rates of particular quadratics, not general bounds. Dimitri Bertsekas's Nonlinear Programming (3rd edition, Athena Scientific, 2016) develops convergence, termination, and rate analysis for gradient methods.

Gradient methods also presuppose that the gradient can be computed. Derivative-free methods use objective values only. Coordinate descent minimizes along one coordinate direction at a time, cycling through the coordinates, and each one-dimensional minimization can be carried out by comparing function values. On a quadratic \(\tfrac12z^\top Hz-f^\top z\) with \(H\succ0\), exact minimization over coordinate \(i\) sets \(z_i=(f_i-\sum_{l\ne i}H_{il}z_l)/H_{ii}\), where \(f_i\) and \(H_{il}\) are entries of \(f\) and \(H\); one cycle is a Gauss–Seidel sweep for \(Hz=f\). With \(H=\operatorname{diag}(1,100)\), one cycle reaches the minimizer whatever the conditioning, because the coordinate axes are eigenvectors. Coupling changes this. The cart objective without the stopping constraint has Hessian \(H=2I+2qcc^\top\), which for \(q=2\) equals \(\begin{bmatrix}11&3\\3&3\end{bmatrix}\). Let \(\delta_1,\delta_2\) be the coordinate errors at the start of a cycle. Minimizing over the first coordinate leaves \(\delta_1=-(H_{12}/H_{11})\delta_2\), and minimizing over the second then leaves \(-(H_{21}/H_{22})\delta_1=H_{12}^2\delta_2/(H_{11}H_{22})\). Each cycle therefore multiplies the second error by \(9/33=3/11\), and coordinate descent converges geometrically without reaching the minimizer in finitely many cycles from a generic start.

The Nelder–Mead method, introduced by J. A. Nelder and R. Mead in “A simplex method for function minimization” (The Computer Journal, 1965), uses no derivatives at all. It maintains \(n+1\) points in \(\mathbb R^n\), the vertices of a simplex, and at each iteration replaces the worst of them by reflecting it through the centroid of the others, by expanding or contracting along that line, or else shrinks the whole simplex toward the best vertex. Neither derivative-free method is analyzed further in this chapter.

Equality constraints remove directions, not necessarily gradients 51:30

Recording: 51:30–56:07 and 1:07:45–1:14:57. Consider minimizing a differentiable \(F(z)\) subject to \(h(z)=0\), where \(h:\mathbb R^n\to\mathbb R^m\) collects \(m\) scalar equality constraints. Its Jacobian \(J_h(z)\) is the \(m\)-by-\(n\) matrix whose rows are the transposed constraint gradients. If \(z(s)\) is a differentiable feasible curve passing through \(z^\star\) at \(s=0\), differentiating \(h(z(s))=0\) gives

Thus tangent directions lie in the nullspace of the constraint Jacobian. If its rows are linearly independent, the regular level-set theorem also identifies this nullspace with the tangent space of a smooth local feasible manifold. This regularity condition is what makes first-order geometry faithfully describe the available local motions.

The nullspace in this argument has a name in the first-order theory. The subspace of first-order feasible variations at \(z^\star\) is \(\ker J_h(z^\star)=\{\Delta z\in\mathbb R^n:J_h(z^\star)\Delta z=0\}\), where \(\ker\) denotes the nullspace. The official slides write it \(V(x^\star)\); this chapter reserves \(V\) for the optimal value of the sensitivity section. The name records exactly what membership guarantees. Because \(h(z^\star)=0\), a first-order expansion gives \(h(z^\star+\Delta z)=J_h(z^\star)\Delta z+o(\|\Delta z\|_2)=o(\|\Delta z\|_2)\) for such a variation: the displaced point satisfies the constraints to first order, but in general not exactly. A feasible point at which the constraint gradients, the rows of \(J_h\), are linearly independent is called regular. At a regular point every first-order feasible variation is the velocity of some feasible curve, which is the content of the level-set statement above. At a point that is not regular, the first-order subspace can be larger than the set of available motions.

At a constrained local optimum the objective derivative along every such feasible curve vanishes: \(\nabla F(z^\star)^\top d=0\). A vector orthogonal to the nullspace of a matrix lies in its row space. Therefore there exists a multiplier vector \(\lambda\in\mathbb R^m\) such that

The stationarity equation admits two equivalent readings. In the first, \(\nabla F(z^\star)=-J_h(z^\star)^\top\lambda\) lies in the subspace spanned by the constraint gradients. With a single regular constraint and \(\nabla F(z^\star)\ne0\), the two gradients are collinear; since each gradient is normal to its own level set, the objective's level set through \(z^\star\) is tangent there to the feasible set. In the second, \(\nabla F(z^\star)\) is orthogonal to every first-order feasible variation, so the first-order cost change \(\nabla F(z^\star)^\top\Delta z\) vanishes on \(\ker J_h(z^\star)\). The second reading is the unconstrained condition with the admissible displacements restricted. Without constraints every \(\Delta z\) is admissible, and orthogonality to all of them forces \(\nabla F(z^\star)=0\).

The sign of \(\lambda\) is a convention fixed by writing the Lagrangian as \(\mathcal L(z,\lambda)=F(z)+\lambda^\top h(z)\). Unlike inequality multipliers, equality multipliers have no sign restriction. With linearly independent constraint gradients, the multiplier for a given stationary point is unique because \(J_h^\top\) has trivial nullspace. There may still be several stationary points.

Regularity cannot simply be omitted. Minimize \(F(s)=s\) subject to \(h(s)=s^2=0\). The only feasible point is zero, hence it is globally optimal. Yet \(F'(0)=1\) and \(h'(0)=0\), so no multiplier solves \(1+\lambda\cdot0=0\). The linearized constraint permits every direction even though the true feasible set permits no motion at all. This is a failure of the first-order constraint representation, not evidence that the optimum is invalid.

In the cart problem, \(h(z)=Ez\), so \(J_h=E\) has full row rank. At the computed optimum, \(g=(-2,-2)^\top\) and \(\lambda=2\) satisfy \(g+E^\top\lambda=0\). The multiplier cancels exactly the gradient component normal to the feasible line. The tangent component is already zero. This is the constrained analogue of unconstrained stationarity.

The derivation above follows the feasible-direction viewpoint: at a local minimum, no small move along a direction that keeps the point feasible can lower the cost. Two standard proofs instead transform the constrained problem into an unconstrained one, and the cart problem already exhibits both. The elimination approach treats the constraints as \(m\) equations in \(n\) unknowns and solves them for \(m\) of the variables. Assume \(F\) and \(h\) are continuously differentiable, written \(C^1\), and that \(z^\star\) is regular. Because the \(m\) rows of \(J_h(z^\star)\) are linearly independent, some \(m\) of its columns form an invertible \(m\)-by-\(m\) submatrix. Reorder the variables as \(z=(y,w)\), with \(y\in\mathbb R^m\) collecting those columns and \(w\in\mathbb R^{n-m}\) the rest, and split the Jacobian accordingly into \(J_y=\partial h/\partial y\) and \(J_w=\partial h/\partial w\). The implicit function theorem supplies a \(C^1\) map \(\phi\) with \(h(\phi(w),w)=0\) near \(w^\star\), \(\phi(w^\star)=y^\star\), and derivative \(-J_y^{-1}J_w\) at \(w^\star\). The reduced objective \(R(w)=F(\phi(w),w)\) is unconstrained and has a local minimum at \(w^\star\), so

where \(J_y^{-\top}\) is the transposed inverse and both Jacobian blocks are evaluated at \(z^\star\). Define \(\lambda=-J_y^{-\top}\nabla_yF(z^\star)\). The \(y\)-components of the multiplier equation, \(\nabla_yF+J_y^\top\lambda=0\), hold by this definition; the \(w\)-components, \(\nabla_wF+J_w^\top\lambda=0\), are the reduced stationarity just displayed. Together they are \(\nabla F(z^\star)+J_h(z^\star)^\top\lambda=0\). For the cart, \(y=u_1\), \(w=u_0\), and \(\phi(u_0)=-u_0\) is the substitution of the opening section; \(J_y=1\) and \(\partial F/\partial u_1=-2\) at the optimum give \(\lambda=2\), the multiplier found above. Regularity is what makes the choice of \(y\) possible. For \(h(s)=s^2\), \(J_h(0)=0\) has no invertible submatrix, and the proof fails exactly where the theorem fails. Exercise 7 shows that the choice of eliminated variables is only local.

The penalty approach disregards the constraints and adds a large cost for violating them. For \(\rho\gt0\), minimize \(F(z)+\rho\|h(z)\|_2^2\) over all \(z\). A minimizer \(z_\rho\) satisfies

which is the multiplier equation with \(2\rho\,h(z_\rho)\) in the role of \(\lambda\). Suppose \(z_\rho\to z^\star\) as \(\rho\to\infty\) and \(z^\star\) is regular. Then \(J_hJ_h^\top\) is invertible near \(z^\star\), and multiplying the display by \(J_h(z_\rho)\) gives \(2\rho\,h(z_\rho)=-(J_hJ_h^\top)^{-1}J_h\nabla F\), evaluated at \(z_\rho\). The right side converges, so the scaled violations converge to a vector \(\lambda\) that satisfies the multiplier equation at \(z^\star\). The delicate step of the full proof, omitted here, is guaranteeing that penalized minimizers exist and converge to the given local minimum rather than elsewhere; it adds a term penalizing distance from \(z^\star\) and restricts the minimization to a small closed ball. Exercise 2 carries out the cart case. There the scaled violation is \(2\rho b=4\rho/(3+2\rho)\), which tends to \(2=\lambda\), while the violation \(b\) itself never vanishes. The penalty argument also extends to inequality constraints by penalizing only the positive part of each violation, and in that setting it yields the sign of the inequality multipliers that the next chapter requires.

First-order conditions cannot distinguish a constrained minimum from a constrained maximum. The AA203 course notes state the second-order conditions, following Bertsekas's Nonlinear Programming. Assume \(F\) and \(h\) are twice continuously differentiable, let \((z^\star,\lambda)\) satisfy the first-order conditions, and write

for the Hessian of the Lagrangian in \(z\), where \(h_i\) and \(\lambda_i\) are the \(i\)th components of \(h\) and \(\lambda\). If \(z^\star\) is a regular local minimum, then \(d^\top\nabla^2_{zz}\mathcal L(z^\star,\lambda)d\ge0\) for every \(d\in\ker J_h(z^\star)\). Conversely, if \(d^\top\nabla^2_{zz}\mathcal L(z^\star,\lambda)d\gt0\) for every nonzero \(d\in\ker J_h(z^\star)\), then \(z^\star\) is a strict local minimum, whether or not it is regular. Both conditions restrict the curvature test to first-order feasible variations, and both test the Lagrangian rather than the objective.

The necessary condition shows why the Lagrangian is the right object. For \(d\in\ker J_h(z^\star)\), regularity and the implicit function theorem, now applied with twice continuously differentiable data, provide a twice differentiable feasible curve \(z(s)\) with \(z(0)=z^\star\) and \(z'(0)=d\). Along it \(h(z(s))=0\), so \(F(z(s))=\mathcal L(z(s),\lambda)\). Differentiating twice at \(s=0\) gives

and the last term vanishes by first-order stationarity. A local minimum along the curve requires a nonnegative second derivative, which is the stated condition. The curve's acceleration \(z''(0)\) carries the curvature of the constraint. It drops out only because the multiplier term has been added; differentiating \(F\) alone leaves \(d^\top\nabla^2F(z^\star)d+\nabla F(z^\star)^\top z''(0)\), whose second term is generally nonzero at a constrained optimum.

Sufficiency follows by contradiction. If \(z^\star\) were not a strict local minimum, there would be feasible points \(z_k\ne z^\star\), \(k=1,2,\ldots\), converging to \(z^\star\) with \(F(z_k)\le F(z^\star)\). Write \(z_k=z^\star+t_kd_k\) with \(t_k=\|z_k-z^\star\|_2\to0\) and \(\|d_k\|_2=1\), and pass to a subsequence along which \(d_k\) converges to a unit vector \(\bar d\). Dividing \(h(z_k)-h(z^\star)=0\) by \(t_k\) and letting \(k\to\infty\) gives \(J_h(z^\star)\bar d=0\). Because both points are feasible and \(\nabla_z\mathcal L(z^\star,\lambda)=0\), Taylor's theorem gives \(F(z_k)-F(z^\star)=\mathcal L(z_k,\lambda)-\mathcal L(z^\star,\lambda)=\tfrac12t_k^2d_k^\top\nabla^2_{zz}\mathcal L(z^\star,\lambda)d_k+o(t_k^2)\le0\). Dividing by \(t_k^2\) and letting \(k\to\infty\) yields \(\bar d^\top\nabla^2_{zz}\mathcal L(z^\star,\lambda)\bar d\le0\) for a nonzero \(\bar d\) in the nullspace, contradicting the hypothesis. Regularity is never used, because the multiplier is given rather than constructed.

For the cart, the constraint is linear, so \(\nabla^2_{zz}\mathcal L=\nabla^2F=2I+2qcc^\top\), which is positive definite on all of \(\mathbb R^2\). The sufficient condition holds and certifies a strict local minimum, consistent with the global certificate of the sufficiency section. On the feasible direction \(d=(1,-1)^\top\), \(c^\top d=1\) and \(d^\top\nabla^2F\,d=4+2q\), which for \(q=2\) equals \(8\), the curvature of the reduced objective \(4(a-\tfrac12)^2+1\). The circle example below shows these conditions deciding a case that the objective's Hessian cannot.

The lecture's circle example has two answers to stationarity 56:07

Recording: 56:07–1:07:45. The source illustrates the multiplier construction with

The constraint gradient is \((2z_1,2z_2)^\top\), nonzero everywhere on the circle. Stationarity and feasibility become \(1+2\lambda z_1=0\), \(1+2\lambda z_2=0\), and \(z_1^2+z_2^2=2\). Since \(\lambda=0\) cannot satisfy the first two equations, they imply \(z_1=z_2=-1/(2\lambda)\). Substitution gives \(\lambda^2=1/4\). Thus the two candidates are \((-1,-1)\) with multiplier \(1/2\), and \((1,1)\) with multiplier \(-1/2\).

Both readings of the multiplier equation can be checked at \((-1,-1)\). The constraint gradient there is \((-2,-2)^\top\), so \(\ker J_h=\{s(1,-1)^\top:s\in\mathbb R\}\). The objective gradient \((1,1)^\top\) is orthogonal to this line and equals \(-\tfrac12(-2,-2)^\top\), a multiple of the constraint gradient. A first-order feasible variation \(\Delta z=s(1,-1)^\top\) is not exactly feasible: \(\|(-1,-1)^\top+\Delta z\|_2^2=(-1+s)^2+(-1-s)^2=2+2s^2\), a violation of second order in \(s\). Along the variation the objective equals \(-2\) exactly, because the variation is orthogonal to the gradient of a linear objective.

Both candidates satisfy the necessary equations, but their objective values are \(-2\) and \(2\). To certify them globally, apply Cauchy–Schwarz:

Both endpoints of this bound are attained by the candidates. One is the global minimum and the other the global maximum. Geometrically, lines of constant objective become tangent to the circle at these two points. Tangency alone does not say which way the objective improves.

The second-order conditions answer that question locally. With \(h(z)=z_1^2+z_2^2-2\), \(\nabla^2h=2I\) and \(\nabla^2F=0\), so \(\nabla^2_{zz}\mathcal L=2\lambda I\). At \((-1,-1)\), \(\lambda=1/2\) gives the identity, which is positive on the tangent line, so the sufficient condition certifies a strict local minimum. At \((1,1)\), \(\lambda=-1/2\) gives \(-I\), and \(d=(1,-1)^\top\) yields \(d^\top(-I)d=-2\lt0\). The necessary condition fails, so \((1,1)\) is not a local minimum. The objective's own Hessian is zero at both points and decides neither. Exercise 5 recovers the value \(-2\) independently: along its rotation curve, \(F=2\cos s\) has second derivative \(-2\) at \(s=0\), the curve's velocity there is \((-1,1)^\top\), and \((-1,1)(-I)(-1,1)^\top=-2\).

Although the objective is linear and hence convex, the equality-defined circle is not a convex feasible set: the midpoint of \((-1,-1)\) and \((1,1)\) is the origin, outside the circle. The standard convex-program guarantee therefore does not apply to this formulation. Existence does hold because the objective is continuous and the feasible circle is nonempty and compact. These are different reasons for different claims. Replacing the circle by its closed disk would make a convex problem, but it would also change the feasible set and the interpretation of the multiplier conditions.

A saddle-point linear system solves the cart 74:57

Recording: 1:14:57–1:18:55. The lecture ends by combining equality feasibility and stationarity as derivatives of the Lagrangian. The following concrete matrix calculation extends that formulation. Allow the cart's required final velocity to be \(b\in\mathbb R\), so \(h(z)=Ez-b\). Define \(H=2I+2qcc^\top\) and \(f=2qDc\). Then \(F(z)=\tfrac12z^\top Hz-f^\top z+qD^2\), and its equality-stationary equations are

Both block rows are gradients of a single function. Regard the Lagrangian as a function \(\mathcal L:\mathbb R^{n+m}\to\mathbb R\) of the stacked variables \((z,\lambda)\). Its partial gradients are \(\nabla_z\mathcal L(z,\lambda)=\nabla F(z)+J_h(z)^\top\lambda\) and \(\nabla_\lambda\mathcal L(z,\lambda)=h(z)\), so at a regular local minimum the first-order conditions take the compact form

with \(n\) equations from the first condition, \(m\) from the second, and \(n+m\) unknowns. The second condition restates feasibility rather than adding a requirement. For the cart, \(n=2\), \(m=1\), and \(\mathcal L(z,\lambda)=\tfrac12z^\top Hz-f^\top z+qD^2+\lambda(Ez-b)\). Its gradient in \(z\), \(Hz+E^\top\lambda-f\), is the top block row, and its derivative in \(\lambda\), \(Ez-b\), is the bottom row.

This equality-constrained optimality system is often called a KKT system. The name does not mean that this recording already covered inequality complementarity. Its block structure matters: the top row balances objective gradient and constraint forces; the bottom row enforces the constraint exactly. Minimizing a penalty \(F(z)+\rho(Ez-b)^2\), with finite positive penalty \(\rho\), would produce a different system and generally a nonzero equality residual.

For \(q\ge0\), \(H\succ0\) because \(d^\top Hd=2\|d\|_2^2+2q(c^\top d)^2>0\) for every nonzero \(d\). Also \(E\) has full row rank. These facts prove the block matrix is nonsingular. Suppose a pair \((d,\mu)\) lies in its nullspace. Then \(Hd+E^\top\mu=0\) and \(Ed=0\). Multiplying the first equation by \(d^\top\) gives \(d^\top Hd=0\), forcing \(d=0\). The first equation then reduces to \(E^\top\mu=0\), which forces \(\mu=0\). A square matrix with trivial nullspace is invertible.

Counting equations and unknowns alone would not prove existence or uniqueness. The lecture's brief equal-count observation should be understood as a way to organize candidate equations; nonlinear systems of equal size can have zero, one, or many solutions. Here the positive-definite and rank assumptions supply the missing proof. The objective is strictly convex on a nonempty affine feasible set, and the solution of the system is its unique global minimum.

With \(D=1,q=2,b=0\), the system yields \(z=(1/2,-1/2)^\top,\lambda=2\). One should check both residuals, \(Hz+E^\top\lambda-f\) and \(Ez-b\), rather than trusting a returned vector. In larger problems a badly conditioned system may have small algebraic residual but sensitive decisions; scaling physical units and examining conditioning remain important.

The Lagrangian is not an ordinary objective to minimize jointly over \(z\) and \(\lambda\). At a point with \(h(z)\ne0\), the term \(\lambda^\top h(z)\) can be driven toward either sign without bound by changing \(\lambda\). Multipliers enforce a balance and encode sensitivity. They are not extra controls whose magnitude should automatically be minimized.

A multiplier measures the cost of changing the requirement

The recording briefly points toward sensitivity; this derivation is an extension. Define the optimal value \(V(b)=F(z^\star(b))\), where \(z^\star(b)\) solves the cart problem with terminal velocity \(b\). Our invertible linear system makes both \(z^\star(b)\) and \(\lambda(b)\) differentiable in \(b\). Differentiate \(Ez^\star(b)=b\) to obtain \(E\,dz^\star/db=1\). The chain rule and stationarity then give

At \(b=0\), the derivative is \(-2\). Allowing a small positive terminal velocity lowers cost to first order, because it lets the cart approach its desired position with less braking. The minus sign depends on our convention \(h=Ez-b\); writing the equality with the opposite sign flips the multiplier. A sensitivity statement without the defining constraint convention is incomplete.

This derivative does not authorize silently changing the specification. A controller that must stop cannot replace \(b=0\) by a cheaper \(b>0\). It tells the designer the local tradeoff if the requirement itself is negotiable. With active inequalities, singular constraints, or changing solution branches, the value function may fail to be differentiable; the tidy formula above relies on a smooth, locally unique solution. The duality and sensitivity discussion in Boyd and Vandenberghe places this interpretation in a broader framework.

Executable checks that test the derivation

The following original example solves a three-by-three equality system and independently compares it with scalar elimination. Inputs are the normalized displacement, nonnegative position penalty, and terminal velocity. Outputs are the two accelerations and the multiplier. The code changes no files and performs no network requests. It requires NumPy and is intentionally a dense educational implementation; sparse factorizations, bounds, and nonlinear dynamics are outside its contract.

import numpy as np

def solve_cart(D=1.0, q=2.0, b=0.0):
    if not np.isfinite([D, q, b]).all() or q < 0:
        raise ValueError("Use finite parameters and a nonnegative penalty")
    c = np.array([1.5, 0.5])
    E = np.array([[1.0, 1.0]])
    H = 2 * np.eye(2) + 2 * q * np.outer(c, c)
    K = np.block([[H, E.T], [E, np.zeros((1, 1))]])
    rhs = np.r_[2 * q * D * c, b]
    solution = np.linalg.solve(K, rhs)
    assert np.linalg.norm(K @ solution - rhs) < 1e-10
    return solution[:2], solution[2]

def value(b):
    z, _ = solve_cart(b=b)
    return z @ z + 2 * (np.array([1.5, 0.5]) @ z - 1) ** 2

z, multiplier = solve_cart()
np.testing.assert_allclose(z, [0.5, -0.5])
assert abs(multiplier - 2) < 1e-12
for D in [-2.0, 0.0, 1.5]:
    z, _ = solve_cart(D=D, q=3)
    a = 3 * D / (2 + 3)  # independent scalar elimination, b = 0
    np.testing.assert_allclose(z, [a, -a], atol=1e-12)
eps = 1e-5
derivative = (value(eps) - value(-eps)) / (2 * eps)
assert abs(derivative + multiplier) < 1e-8
print("Cart optimum, equality residual, and sensitivity checks passed")

Residual testing checks that the implemented system was solved; scalar elimination checks that it represents the intended constrained model. Signed and zero displacements test symmetry and a boundary case. The centered finite difference checks the multiplier sign against an independently evaluated value function. The companion numerical laboratory also checks the Armijo trial sequence and the zero-gradient stopping case. These tests support specific finite computations; the preceding proofs establish the general guarantees.

Exercises with full solutions

What these conditions leave unresolved

The recording provides the finite-dimensional language needed for later optimal-control derivations. A stationary equation is a candidate generator. A positive definite Hessian gives a local certificate; convexity can give a global one. A descent direction needs a compatible step length. Equality constraints replace zero gradient by zero gradient along feasible directions, under a regularity assumption.

Our cart's linear dynamics and quadratic objective made these distinctions computable without ambiguity. General nonlinear dynamics can create nonconvex feasible sets and many stationary trajectories. Acceleration and state bounds add inequalities and active-set changes. Numerical discretization adds approximation error. None of those difficulties disappears because a solver returns small first-order residuals. The next recording develops inequality conditions and then begins the passage from vectors to entire trajectories. The interactive trajectory laboratory shows why a mathematically optimal unconstrained input can still violate a physical bound.

Source boundary: topic ordering and time links come from a complete local machine transcription checked against official lecture-1 and lecture-2 slides. Mathematical notation is reconstructed from those slides and explicit derivations, not treated as verbatim speech-recognition output. Administrative details are omitted. The section evidence map separates lecture ideas, external theory, and original examples; no claim of human-authored captions or complete verbal reproduction is made. The official lecture-2 slides add the iterative-descent framing and the angle reading of a descent direction, the strict-decrease meaning of gradient descent, the name steepest descent, the minimization step rule, the remark that diminishing steps need not decrease the cost at every iteration, the analysis and derivative-free methods left outside the course, the penalty and elimination routes to the multiplier theorem, the subspace of first-order feasible variations and the term regular, the pointer to second-order conditions in the course notes, and the Lagrangian gradient in the multipliers; their inequality-constraint pages are developed in the next chapter, and where their notation differs, the text translates it.