Identification and adaptive tracking

A controller can track perfectly while its parameter estimate remains wrong. A regression model can fit every collected transition while failing to identify how the plant responds to a different control policy. Learning for control therefore requires two separate questions: what information does the experiment provide, and what behavior does the coupled learning-and-control system guarantee?

The first part of the course treated dynamics as known. Here the cart's mass, actuation gain, or sampled transition coefficients become unknown. The chapter develops least-squares identification, examines excitation and statistical assumptions, and then derives a model-reference adaptive controller whose tracking guarantee does not require proving that every parameter converges.

Grounded in AA203 lecture 13. The recording ends after the unknown-mass adaptive-tracking example and explicitly postpones model-identification adaptive control to lecture 14. The regression qualifications, placement of the one-half in the Lyapunov function, semidefinite-decrease argument, and numerical experiments below complete details that the source treats informally. The official slides add the course roadmap and the feedback-loop framing, the classification of learning methods, the offline identification pipeline, continuous-time least squares, the explicit covariance and information-matrix calculations, the definition and classical aim of adaptive control, Lyapunov's theorem for global stability, the closed-loop identity and sliding-variable reading of the filtered error, the role of the reference signal in parameter convergence, and Slotine and Li's text as the reference for systematic design; where their notation differs, the text translates it.

Unknown dynamics create several different learning problems 3:41

The preceding chapters solved optimal-control problems by two routes and a bridge between them. Open-loop methods compute a trajectory from one initial condition, either indirectly through necessary conditions or directly through transcription into a finite-dimensional program. Closed-loop methods compute a policy, through dynamic programming or through the Hamilton–Jacobi–Bellman and Hamilton–Jacobi–Isaacs equations. Model predictive control obtains feedback by re-solving open-loop problems from each measured state. Every one of these methods assumed that the dynamics were known, from an analytical model or a differential equation. The remainder of the course relaxes that assumption. In the course roadmap, this chapter sits on the classical branch of control, beside ordinary feedback control: it treats system identification and adaptive control. When learning is instead combined with an optimal-control formulation, the result is adaptive optimal control, which later chapters pursue through model-free and model-based reinforcement learning. The two branches differ in what they establish, a difference that the section on adaptive control makes precise.

The common setting is the feedback loop of reference tracking. A reference signal specifies the behavior that the system should exhibit. A sensor measures the system's output, the measurement is subtracted from the reference, and a controller converts that difference into a control input to the system, which closes the loop. Every earlier design used a model of the block between control input and output to choose the controller. The learning problem begins when that block becomes partly unknown, for instance when the cart carries a payload of unknown mass or meets forces that its model omits. The comparison with the reference still produces an error signal, but the input-to-output map that a model-based design would use is no longer fully available.

Uncertainty does not automatically demand a learning algorithm. Ordinary feedback may compensate sufficiently for a small mismatch. A robust controller may handle a specified uncertainty set without identifying which member is present. Learning instead uses observed transitions to improve a controller or a model. These approaches can be combined: an estimator may refine a model while a robust controller protects against its remaining uncertainty.

Each response has a typical use. A quadrotor flying through mild wind gusts can rely on an ordinary feedback law, such as proportional–integral–derivative control, because the gusts cause small tracking errors that the loop corrects. A vertical-takeoff aircraft such as the Harrier near hover meets large and unpredictable turbulence. A robust design then chooses the control to perform acceptably under the worst disturbance in a specified set, which is the max–min formulation of the differential-game chapter, where the disturbance acts as an adversary. Protection against the worst case can make the controller conservative when the disturbances that actually occur are mild, because the design pays for disturbances that do not arrive. A robotic arm that adjusts its grip from the slip it detected in earlier grasp attempts illustrates the third response: recorded transitions, such as logs of earlier flights or grasps, are used to estimate how the system behaves or how its controller should change.

The lecture distinguishes direct improvement of the policy from learning a model and then designing a policy from that model. It also distinguishes data collected before deployment, adaptation during one continuing run, and learning across repeated episodes. “Zero episodes” means no new deployment interaction in the learning phase, not no historical experience. Offline reinforcement learning can use previously collected episodes, while online adaptive control interacts continuously without a reset. These categories describe available experience rather than mutually exclusive algorithm names.

The two families contain the course's named learning methods. Direct adaptive control, imitation learning, and model-free reinforcement learning use measurements to change the controller itself. System identification, indirect adaptive control, and model-based reinforcement learning first fit a model and then use it to improve the controller, so they introduce the intermediate model-fitting objective examined in the section on prediction as an intermediate objective. This chapter develops system identification and one direct adaptive controller. The next chapter completes indirect adaptive control and begins imitation learning, and later chapters develop model-free and model-based reinforcement learning.

For the cart, an engineer might collect acceleration responses before operation and fit a mass; an adaptive controller might update a force coefficient while carrying an unknown payload; an episodic learner might improve across many resettable simulated transfers. The ability to reset changes the cost of exploration. A dangerous action during a single physical run cannot be undone by labeling the following interval a new episode.

Least squares identifies parameters only in observed directions 16:02

Many problems do not require learning during operation. The standard engineering pipeline runs experiments in advance, fits a data-driven model of the dynamics to the recorded data, and then uses that model in place of the unknown dynamics, with any planning or control method of the earlier chapters and without further online learning. A recorded trajectory \(x_0,\ldots,x_N\) with inputs \(u_0,\ldots,u_{N-1}\) supplies \(N\) input–output pairs, one per transition. The pipeline relies on an engineer in the loop, who designs the experiments, resets the system between them, stops a run that becomes unsafe, and judges when the fitted model is adequate. It also presumes that the plant at deployment is the plant that was identified. When these conditions fail, as with a payload whose mass changes between deliveries, the adaptive methods later in this chapter update the controller during operation instead. Linear regression is the principal tool of the offline pipeline.

Let \(y_i\in\mathbb R\) be an observed response and \(z_i\in\mathbb R^p\) its regressor, with model \(y_i=z_i^\mathsf T\theta+\epsilon_i\). Here \(\theta\in\mathbb R^p\) is an unknown parameter vector and \(\epsilon_i\) is the residual disturbance. Stack the row vectors \(z_i^\mathsf T\) into \(Z\in\mathbb R^{N\times p}\), responses into \(y\in\mathbb R^N\), and residuals into \(\epsilon\). Ordinary least squares minimizes

Expanding gives \(y^\mathsf Ty-2\theta^\mathsf TZ^\mathsf Ty+\theta^\mathsf TZ^\mathsf TZ\theta\). The gradient is \(2Z^\mathsf T(Z\theta-y)\) and the Hessian is \(2Z^\mathsf TZ\succeq0\), so the normal equations are sufficient for a minimum. If \(Z\) has full column rank, the solution is unique:

This formula explains the estimator; it is usually not the preferred numerical implementation. QR or singular-value decomposition avoids explicitly inverting the normal matrix and avoids squaring the condition number as a numerical procedure. The normal matrix is \(p\times p\), not \(N\times N\). Increasing sample count enlarges the data matrix and accumulation work, while parameter dimension determines the size of that inverse.

For the scalar dynamical model \(x_{t+1}=ax_t+bu_t+\epsilon_t\), use \(z_t=(x_t,u_t)^\mathsf T\), \(y_t=x_{t+1}\), and \(\theta=(a,b)^\mathsf T\). For the sampled cart, each successor-state component can be regressed on the current position, velocity, and control. The same algebra applies to a matrix of outputs; full regressor rank is still required to identify every coefficient without additional structural assumptions.

If \(Z\) is rank deficient, a pseudoinverse returns a particular minimum-norm fit. It does not create evidence for the missing parameter directions. Whenever \(Z\delta=0\), parameter vectors \(\theta\) and \(\theta+\delta\) make identical predictions on the collected data. Regularization chooses among such models using an imposed preference, rather than discovering which one the data uniquely imply.

The fixed-design covariance calculation has explicit assumptions 26:45

For a fixed full-rank matrix \(Z\), assume \(\mathbb E[\epsilon\mid Z]=0\) and \(\operatorname{Cov}(\epsilon\mid Z)=\sigma^2I\), with finite \(\sigma^2\). Substituting the data model gives

Taking the conditional expectation proves unbiasedness. Applying the covariance transformation to this linear map and canceling \(Z^\mathsf TZ\) yields

The covariance step uses the rule that a fixed matrix \(A\) maps a random vector with covariance \(\Sigma\) to one with covariance \(A\Sigma A^\mathsf T\), while adding the constant \(\theta\) changes no covariance. Here \(A=(Z^\mathsf TZ)^{-1}Z^\mathsf T\), whose transpose is \(Z(Z^\mathsf TZ)^{-1}\) because \(Z^\mathsf TZ\) is symmetric, so

Writing the normal matrix as a sum of outer products, \(Z^\mathsf TZ=\sum_{i=1}^Nz_iz_i^\mathsf T\), shows what each sample contributes. For the scalar dynamical model of the previous section, with \(z_t=(x_t,u_t)^\mathsf T\) for \(t=0,\ldots,N-1\),

The diagonal entries accumulate the squared state and the squared input. The off-diagonal entry accumulates their product, and a strong correlation between state and input can make the matrix nearly singular even when both diagonal entries are large. The section on excitation shows a feedback experiment in which the matrix is exactly singular.

Under these assumptions, the Gauss–Markov statement is that this estimator has minimal covariance among linear unbiased estimators. It is not a claim of minimal error among all possible biased and nonlinear estimators. If the conditional noise is independent Gaussian with common variance, maximizing the likelihood gives the same least-squares estimate.

Unbiasedness is not consistency. An estimator may be centered correctly while retaining a large variance forever. For the displayed covariance to shrink in every direction, the smallest eigenvalue of \(Z^\mathsf TZ\) must diverge as data accumulate. A large trace or one growing eigenvalue is insufficient. If \(\lambda_{\min}(Z^\mathsf TZ)\geq\alpha N\), then the largest covariance eigenvalue is at most \(\sigma^2/(\alpha N)\).

The two moments answer the convergence question directly under the fixed-design assumptions. The conditional mean-square error is the trace of the covariance,

because the trace sums the \(p\) eigenvalues of \((Z^\mathsf TZ)^{-1}\), each at most \(1/\lambda_{\min}(Z^\mathsf TZ)\). Markov's inequality applied to the squared error then bounds the conditional probability of an error of norm at least \(\rho\gt0\) by \(p\sigma^2/(\rho^2\lambda_{\min}(Z^\mathsf TZ))\). Hence \(\lambda_{\min}(Z^\mathsf TZ)\to\infty\) is sufficient for \(\widehat\theta\to\theta\) in mean square and in probability. This is the precise form of the informal criterion that a system operated forever is eventually identified exactly. Operating forever is not enough: operation must keep adding information in every parameter direction, the requirement that the next section calls persistent excitation, and the fixed-design noise assumptions must hold.

There is a further issue in system identification: the design matrix often contains states generated by past disturbances. A current innovation can be independent of the current regressor while affecting future regressors in the same data matrix. Conditioning on the entire \(Z\) can therefore destroy the simple zero-mean argument above. Finite-sample unbiasedness for a fixed design should not be asserted automatically for one dependent trajectory.

This does not make least squares unusable. It changes the analysis needed to justify its behavior. Simchowitz and colleagues' Learning Without Mixing analyzes least-squares identification from a single dependent linear-system trajectory and develops finite-sample guarantees suited to that setting. Its relevance here is the need for time-series reasoning, rather than importing an independent-design theorem without checking its premises.

Feedback can hide the parameters it stabilizes 33:00

Suppose data are collected under the fixed feedback \(u_t=-kx_t\), where \(k\) is known. The plant then evolves as \(x_{t+1}=(a-bk)x_t+\epsilon_t\). Every regressor is proportional to \((1,-k)^\mathsf T\). Only the combination \(a-bk\) can be identified from this experiment; infinitely many parameter pairs produce the same closed-loop coefficient.

For an original example, take \(k=1\). The pairs \((a,b)=(1.2,0.7)\) and \((0.9,0.4)\) both give closed-loop coefficient 0.5. A noiseless trajectory from \(x_0=1\) is \(x_t=2^{-t}\) under both plants. Collecting a million points from this same feedback experiment does not distinguish them. In addition, the total squared state excitation is finite because \(\sum_{t=0}^\infty4^{-t}=4/3\).

The normal matrix of the previous section shows the defect without computing any estimate. Under \(u_t=-kx_t\), the sums satisfy \(\sum x_tu_t=-k\sum x_t^2\) and \(\sum u_t^2=k^2\sum x_t^2\), so

for every \(N\). Its null direction is \((k,1)^\mathsf T\), the parameter change that leaves \(a-bk\) unchanged, and its other eigenvector is \((1,-k)^\mathsf T\), with eigenvalue \((1+k^2)\sum x_t^2\). For the noiseless trajectory \(x_t=2^{-t}\) with \(k=1\), the sum tends to \(4/3\), so the matrix tends to \(\tfrac43\begin{bmatrix}1 & -1\\ -1 & 1\end{bmatrix}\), whose eigenvalues are \(8/3\) and \(0\). The information in the identifiable direction therefore stays bounded as well, which is the finite total excitation noted above. With noise, the disturbances keep exciting the state and \(\sum x_t^2\) can grow without bound, but the determinant remains exactly zero.

A common finite-window persistent-excitation condition asks for constants \(L\in\mathbb N\), \(\alpha>0\), such that

This ensures recurrent information in every parameter direction. Statistical convergence also requires suitable noise and model assumptions; excitation is not a substitute for them. Merely moving the system is not enough if all motion follows one regressor direction. Conversely, full rank on a finite dataset permits a unique fit but says little about its sensitivity when the smallest singular value is tiny.

Adding a probing input \(u_t=-kx_t+\eta_t\) can break collinearity, where \(\eta_t\) is a designed excitation. That changes both the information and the actual trajectory. Its amplitude cannot be chosen solely to maximize a regression determinant if state and actuator constraints matter. The identification problem is already becoming a control problem.

A useful streaming implementation is recursive least squares. With a positive-definite initial information inverse \(P_0\), initial estimate \(\widehat\theta_0\), and no forgetting, define

The last identity is the rank-one inverse update for accumulated information. A finite \(P_0^{-1}\) corresponds to an initial quadratic regularization, so this recursion matches that regularized batch problem rather than unregularized least squares exactly. It lowers repeated computation; it does not repair uninformative data.

Least squares also has a continuous-time form, in which sums become integrals. Suppose the response \(y(t)\) and the regressor \(z(t)\) are observed as continuous signals. At each time \(t\), minimize the regularized integral error

Setting the gradient to zero gives the normal equation \(G(t)\widehat\theta(t)=P_0^{-1}\widehat\theta_0+\int_0^tz(\tau)y(\tau)\,d\tau\), with information matrix \(G(t)=P_0^{-1}+\int_0^tz(\tau)z(\tau)^\mathsf T\,d\tau\). Because \(\dot G=zz^\mathsf T\), the inverse \(P(t)=G(t)^{-1}\) satisfies \(\dot P=-P\dot GP\). Differentiating the normal equation, \(\dot G\widehat\theta+G\dot{\widehat\theta}=zy\), and solving for \(\dot{\widehat\theta}\) gives the streaming form

The update multiplies a gain, the regressor, and the current prediction error. The adaptation law later in this chapter has the same product structure, but its error is a tracking error rather than a prediction error, a difference that separates model-reference from model-identification adaptive control. For a dynamical model, the continuous-time response is a state derivative, which is rarely measured directly. Differentiating a noisy position measurement amplifies its noise, so a practical continuous-time fit must address that noise, a step the ideal formula omits.

Accurate prediction is an intermediate objective 36:11

The lecture asks how much data is needed, how experiments should be designed, and whether parameter accuracy is the right performance measure. The final objective is useful control. A model error in a direction irrelevant to the task may matter little, whereas a small error that changes a crucial decision can matter greatly.

Consider one-step dynamics \(y=bu\), input bound \(|u|\leq1\), and target \(y=1\). The true gain is \(b=0.01\). A far-away estimate \(\widehat b=0.5\) chooses \(u=1\) and achieves true squared error \(0.9801\). A numerically closer estimate \(\widehat b=-0.001\) chooses \(u=-1\), giving true squared error \(1.0201\). Parameter distance alone ranks these two estimates opposite to their induced task performance. This small constructed case illustrates the lecture's sign-sensitive controller example without claiming a universal relation.

Model-class error creates another limit. Fitting a linear predictor to nonlinear drag may be useful over a narrow operating range and misleading elsewhere. More samples from the same narrow region reduce estimation variance without eliminating extrapolation error. Reporting training residual alone does not establish closed-loop robustness after a new controller visits different states.

Adaptive control proves stability of a coupled loop 40:17

Adaptive control occupies the one-episode setting of the first section. Instead of fitting a model before deployment, it adapts the policy online, during the same continuing run in which the controller acts, so that performance improves as information arrives. The adaptation can follow either family of the first section. A direct scheme adjusts parameters of the control law itself; model-reference adaptive control, developed below, is the standard example. An indirect scheme updates an estimate of the model and then recomputes or re-optimizes the controller from the current estimate; model-identification adaptive control, whose name records this identification step, is the standard example and is completed at the start of the next chapter. Both names cover large families of techniques rather than single algorithms.

Classical adaptive control does not usually pose the adaptive optimal control problem, in which the controller would choose inputs to minimize a cost while a parameter is unknown and would therefore have to weigh how its present actions affect what it later learns. It asks instead whether the coupled system formed by the plant, the controller, and the adaptation law is stable, and whether the quantity the designer cares about, such as a tracking error, converges. The coupling is what makes that question difficult. For the cart below, the closed-loop state contains the plant's position and velocity, the reference model's position and velocity, and the current mass estimate. Although the plant and the reference model are linear, the applied force multiplies the estimate by a signal that depends on the tracking error, and the adaptation law multiplies two such signals. The coupled system is therefore nonlinear, and an eigenvalue test of a linear closed loop cannot certify its stability. A Lyapunov argument can, which is why the next section states the global Lyapunov theorem before applying it.

A semidefinite energy derivative requires an invariance argument 42:47

The analysis tool is Lyapunov's theorem for global asymptotic stability. Write a general state vector as \(\xi\in\mathbb R^n\), keeping \(x\) for a position, and consider an autonomous system \(\dot\xi=f(\xi)\) with \(f\) locally Lipschitz, so that solutions exist and are unique. A controlled system \(\dot\xi=F(\xi,u)\) under a feedback policy \(u=\pi(\xi)\) has this form, with \(f(\xi)=F(\xi,\pi(\xi))\). A point \(\xi^\star\) is an equilibrium if \(f(\xi^\star)=0\). For a continuously differentiable function \(V:\mathbb R^n\to\mathbb R\), the derivative along trajectories is \(\dot V(\xi)=\nabla V(\xi)^\mathsf Tf(\xi)\), which can be evaluated at every state without solving the differential equation. Suppose that

Then \(\xi^\star\) is globally asymptotically stable: trajectories that start near it stay near it, and every trajectory, from any initial state, converges to it. Conditions (1) and (2) make \(V\) behave like a stored energy that every motion away from the equilibrium strictly dissipates. Condition (3), called radial unboundedness, makes the conclusion global. It makes every sublevel set \(\{\xi:V(\xi)\le c\}\) bounded, so a trajectory along which \(V\) decreases stays in a compact set. Without it, \(V\) can decrease while the state escapes. The positive-definite function \(V(\xi)=\xi_1^2/(1+\xi_1^2)+\xi_2^2\) stays below \(1\) on the entire \(\xi_1\)-axis, so its sublevel set \(\{V\le1\}\) contains that unbounded axis, and a decreasing \(V\) alone does not keep the trajectory bounded. The discrete-time Lyapunov argument for model predictive control used bounds required only near the origin; the three conditions here hold on all of \(\mathbb R^n\), which is what makes this conclusion global.

Condition (2) demands strict decrease at every state other than the equilibrium, and both applications in this chapter fail it. The oscillator's energy derivative vanishes at every turning point, as shown next. The adaptive controller's Lyapunov derivative, derived later, vanishes whenever a filtered tracking error is zero, whatever the error in the mass estimate. The adaptive error equations are also time-varying when the reference acceleration varies in time, as it does for a sinusoidal reference, because the reference acceleration enters them, so the theorem for autonomous systems does not apply to them directly. For a reference with zero acceleration, such as a constant position or a constant-velocity ramp, every state with zero tracking error and zero error velocity is an equilibrium of those equations for any mass estimate, so no single equilibrium of the joint state can be globally asymptotically stable. The theorem is therefore the template of the argument; an invariance argument completes it for the oscillator, and a Barbalat-type argument completes it for the adaptive controller.

The adaptive-control discussion returns to continuous-time Lyapunov analysis. For the unforced mass–spring–damper equation \(m\ddot x+d\dot x+kx=0\), with \(m,d,k>0\), the physical energy is

Conditions (1) and (3) hold by inspection, because \(k,m\gt0\) make \(E\) a positive-definite quadratic form in \((x,\dot x)\), with the origin as equilibrium. Condition (2) requires the derivative along trajectories. Differentiating gives \(\dot E=kx\dot x+m\dot x\ddot x=\dot x\,(kx+m\ddot x)\), and the equation of motion replaces \(m\ddot x\) by \(-d\dot x-kx\). The spring terms cancel, which leaves \(\dot E=-d\dot x^2\): only the damper removes energy.

The derivative is nonpositive, but not strictly negative at every nonzero state: at a displaced turning point, \(\dot x=0\) while \(x\neq0\), and \(\dot E=0\). The source's informal strict-negativity statement needs this correction.

To finish the proof, energy sublevel sets are compact and invariant. Inside the set where \(\dot E=0\), velocity is zero. Remaining there requires acceleration also to be zero, and the dynamics then force \(kx=0\). The only invariant trajectory contained in the zero-derivative set is the origin. LaSalle's invariance principle therefore establishes convergence. If \(d=0\), the zero-derivative set is the whole phase space and the oscillator need not converge. The missing condition is consequential.

This distinction prepares the adaptive example. A negative-semidefinite derivative can prove boundedness and tracking convergence with additional reasoning, while leaving parameter error nonzero. One should not infer a globally asymptotically stable joint parameter-and-tracking equilibrium merely from a dissipation term that sees only tracking error.

A reference model specifies the response the plant should imitate 51:04

Model-reference adaptive control, abbreviated MRAC, combines a plant with unknown parameters, a reference model, an adjustable feedback law, and an adaptation mechanism. Here the plant is the cart written as \(m\ddot x=u\), where \(m>0\) is a constant mass and \(u\) is force. A desired response \(x_m\) is generated by

The command \(r(t)\) is supplied externally; gains \(k_d,k_p,k_r\) are designer choices, with positive damping and stiffness producing a stable reference model. The reference output is distinct from the actual plant state. Specifying a reference model is a behavioral objective, not necessarily the solution of an optimal-control cost.

Define tracking error \(e=x-x_m\) and choose \(\lambda>0\). If the mass is known, use

Substitution into the plant gives \(\ddot e+2\lambda\dot e+\lambda^2e=0\). Its characteristic polynomial is \((r+\lambda)^2\), so \(e(t)=(c_0+c_1t)e^{-\lambda t}\), with constants determined by initial errors. Both position and velocity tracking errors converge to zero. The controller has not assumed the initial state already lies on the reference trajectory.

The official slides write \(\tilde x=x-x_m\) for the tracking error that this chapter calls \(e\), \(v\) for the signal this chapter calls \(\nu\), and \(k\) for the reference-model input gain \(k_r\); the chapter reserves \(k\) for a feedback gain and a spring stiffness. The slides also describe the known-mass law as achieving perfect tracking. The precise statement is the one derived above: the error decays exponentially from any initial condition, and it is identically zero only when \(e(0)=\dot e(0)=0\).

An adaptation law cancels a mixed term in the Lyapunov derivative 1:01:52

When \(m\) is unknown, replace it in the force law by an adjustable estimate \(\widehat m\): \(u=\widehat m\nu\). Define parameter error \(\widetilde m=\widehat m-m\) and filtered tracking error \(s=\dot e+\lambda e\). Since \(m\ddot x=(m+\widetilde m)\nu\), direct substitution yields

Two identities carry this substitution. The plant and the force law give \(m\ddot x=\widehat m\nu=m\nu+\widetilde m\nu\), so \(m(\ddot x-\nu)=\widetilde m\nu\). The definition of \(\nu\) gives \(\ddot x-\nu=\ddot e+2\lambda\dot e+\lambda^2e\), and because \(\dot s=\ddot e+\lambda\dot e\) and \(\lambda s=\lambda\dot e+\lambda^2e\), this sum equals \(\dot s+\lambda s\). With a known mass, \(\widetilde m=0\) and the equation reduces to \(\dot s=-\lambda s\), the known-mass error equation written as a first-order equation for \(s\).

The filtered error \(s\) is often called the sliding variable, a name taken from sliding-mode control, in which a controller drives the state onto the set \(s=0\) and keeps it there. In the \((e,\dot e)\) plane that set is the line \(\dot e=-\lambda e\), along which the position error decays exponentially at rate \(\lambda\). The adaptive loop does not place the state on that line. The convergence section shows that the state approaches it as \(s\to0\).

The unknown parameter creates a product of parameter error, control regressor, and tracking error in an energy derivative. Choose adaptation gain \(\gamma>0\) and update

The compatible Lyapunov candidate is

The one-half in the second term is necessary for the stated update law. The official slide writes the same function as \(\tfrac12\big[ms^2+\widetilde m^2/\gamma\big]\), with the one-half multiplying both terms; a reading that drops the bracket attaches the one-half to the first term only and produces an extra factor of two, as Exercise 3 shows. Written out term by term, the slide's function is the expression displayed above.

Because the true mass is constant, \(\dot{\widetilde m}=\dot{\widehat m}\). Differentiate and substitute both coupled equations:

This algebra is the design principle: the adaptation term cancels the unknown mixed term. The true mass appears in the proof's energy function, but the implemented update does not require knowing it. Increasing \(\gamma\) changes adaptation speed and transient behavior; this identity alone is not an optimal tuning rule.

Tracking convergence and parameter convergence are different claims 1:11:32

Assume the reference acceleration is bounded and the ideal continuous-time model, exact measurements, and unsaturated force law apply. The nonincreasing nonnegative \(V\) bounds \(s\) and \(\widetilde m\), and integration gives \(\int_0^\infty s(t)^2\,dt<\infty\). This alone does not imply \(s(t)\to0\); arbitrarily narrow spikes can have finite squared integral.

The filtered error satisfies \(\dot e=-\lambda e+s\). Since \(s\) is bounded, this stable linear equation bounds \(e\), and then \(\dot e=s-\lambda e\) is bounded. Bounded reference acceleration makes \(\nu\) bounded. The equation for \(\dot s\) now bounds \(\dot s\), so \(s\) is uniformly continuous. A Barbalat-type argument shows that a uniformly continuous signal with finite squared integral must tend to zero: otherwise infinitely many fixed-height deviations, each with a minimum duration imposed by continuity, would make the integral diverge.

Finally, variation of constants gives

Split the integral into an old finite interval, whose contribution decays, and a recent tail where \(s\) is uniformly small. This proves \(e\to0\), and \(\dot e=s-\lambda e\to0\). These steps complete the source's convergence inference while keeping its assumptions visible.

Nothing here proves \(\widetilde m\to0\). With the constant reference \(x_m=0\), initial \(x=\dot x=0\), and any fixed estimate \(\widehat m\), we have \(e=s=\nu=u=0\) and \(\dot{\widehat m}=0\). Tracking is exact even if the mass estimate is wrong. Parameter convergence needs informative excitation and additional analysis; it is not a consequence of tracking convergence alone.

The zero reference fails because it never asks the plant to accelerate, and force reveals mass only through acceleration. A reference that keeps demanding acceleration changes the conclusion. Assume, in addition to the hypotheses above, that the reference jerk \(\dddot x_m\) is bounded. Because \(V\) is nonincreasing and bounded below, it has a limit \(V_\infty\). Since \(s\to0\), the identity \(\widetilde m(t)^2=2\gamma\big(V(t)-\tfrac12ms(t)^2\big)\) shows that \(\widetilde m(t)^2\to2\gamma V_\infty\), so \(|\widetilde m(t)|\) converges to a limit \(\mu\ge0\). Next, \(\dot{\widetilde m}=-\gamma\nu s\) is bounded, \(\ddot e=\widehat m\nu/m-\ddot x_m\) is bounded, and therefore \(\dot\nu=\dddot x_m-2\lambda\ddot e-\lambda^2\dot e\) is bounded. Differentiating \(\dot s=\widetilde m\nu/m-\lambda s\) then shows that \(\ddot s\) is bounded, so \(\dot s\) is uniformly continuous. A function with a finite limit whose derivative is uniformly continuous has a derivative that tends to zero; this is Barbalat's lemma in the form needed here. Applied to \(s\), it gives \(\dot s\to0\), and the closed-loop equation gives \(\widetilde m\nu=m(\dot s+\lambda s)\to0\). If \(\mu\) were positive, \(|\widetilde m|\) would eventually exceed \(\mu/2\), which would force \(\nu\to0\); since \(e\) and \(\dot e\) tend to zero, \(\nu-\ddot x_m\to0\), and therefore \(\ddot x_m\to0\). Consequently, if the reference acceleration does not tend to zero, then \(\widehat m\to m\).

This is the one-parameter form of persistent excitation: the signal that multiplies the parameter error, here \(\nu\), must keep returning to values away from zero. The condition is sufficient, not necessary, and it concerns the ideal model. The sinusoidal reference \(x_m=\sin t\) satisfies it, and so does the constant-acceleration reference \(x_m=t^2/2\). A constant reference and a constant-velocity ramp do not: in steady tracking both require zero force, and a zero force is produced by every mass estimate. With several unknown parameters, the regressor must excite every parameter direction, as in the matrix condition of the excitation section, and tracking can converge while the estimate remains wrong in any direction that the reference never excites. A reference simple enough to be tracked by many parameter values therefore leaves those values unidentified, whereas a reference rich enough that only the true value tracks it forces the parameters to converge.

The numerical experiment uses a sinusoidal reference to keep the system informative, and its mass estimate approaches the true value. That is a verified outcome for this fixture, not a proof for every bounded reference. Force limits, noisy differentiation, changing mass, and numerical discretization also lie outside the ideal continuous-time theorem. Practical adaptive designs may add projection, normalization, or robust modifications, but those changes require their own argument.

The example took its ingredients as given: the reference model, the control law, the adaptation law, and the Lyapunov candidate. Constructing them for a new plant is the substance of adaptive-control design, and the construction depends on the plant's structure. Jean-Jacques E. Slotine and Weiping Li, Applied Nonlinear Control (Prentice Hall, 1991), Chapter 8, develops model-reference adaptive control systematically for broader classes of plants, together with the parameter-convergence and robustness questions raised in this section.

Executable checks: nonidentifiability and adaptive dissipation

The first check constructs two exactly indistinguishable closed-loop plants. The second integrates the complete plant-plus-estimator equations, checks tracking, and compares the integrated dissipation with the Lyapunov decrease. It also tests the zero-excitation equilibrium directly.

import numpy as np
from scipy.integrate import solve_ivp

x = .5**np.arange(12)
Z = np.column_stack([x[:-1], -x[:-1]])
assert np.linalg.matrix_rank(Z) == 1
theta1 = np.array([1.2, .7])
theta2 = np.array([.9, .4])
assert np.allclose(Z@theta1, x[1:])
assert np.allclose(Z@theta2, x[1:])
assert np.allclose(Z@(theta1-theta2), 0)

m, lam, gamma = 2., 1., 2.
def dynamics(t, state):
    position, velocity, estimate = state
    e = position-np.sin(t)
    de = velocity-np.cos(t)
    s = de+lam*e
    nu = -np.sin(t)-2*lam*de-lam*lam*e
    return [velocity, estimate*nu/m, -gamma*nu*s]

t = np.linspace(0., 60., 6001)
result = solve_ivp(dynamics, (0, 60), [0., 0., 1.],
                   t_eval=t, rtol=1e-10, atol=1e-12)
assert result.success
position, velocity, estimate = result.y
e = position-np.sin(t)
de = velocity-np.cos(t)
s = de+lam*e
V = .5*m*s*s+(estimate-m)**2/(2*gamma)
assert np.max(np.diff(V)) < 1e-8
dissipation = np.trapezoid(lam*m*s*s, t)
assert abs(V[0]-V[-1]-dissipation) < 1e-6
assert abs(e[-1]) < 1e-6 and abs(de[-1]) < 1e-6
assert abs(estimate[-1]-m) < 1e-6
# Constant zero reference, zero plant state, wrong mass estimate:
nu, s0, wrong_estimate = 0., 0., 7.
assert wrong_estimate*nu == 0
assert -gamma*nu*s0 == 0
print("Sinusoidal-reference final mass estimate:", estimate[-1])
print("Integrated dissipation:", dissipation)
print("Wrong estimate can coexist with exact zero-reference tracking.")

A further check integrates the same closed loop for four references: a constant position \(x_m=1\), a ramp \(x_m=t\), a constant acceleration \(x_m=t^2/2\), and the sinusoid \(x_m=\sin t\). Its inputs are each reference's position, velocity, and acceleration as functions of time, together with the same true mass \(m=2\), gains \(\lambda=1\) and \(\gamma=2\), initial state \(x(0)=\dot x(0)=0\), and initial estimate \(\widehat m(0)=1\). Its outputs are the final tracking errors and mass estimates at \(t=60\). It asserts that tracking converges for all four references, that the estimate reaches the true mass for the two references whose acceleration persists, and that it remains more than \(0.2\) from the true mass for the constant and ramp references. The integration uses the ideal continuous-time model with tight solver tolerances, so the asserted outcomes are numerical evidence for these four fixtures; the parameter-convergence argument in the convergence section is the general statement.

import numpy as np
from scipy.integrate import solve_ivp

m, lam, gamma = 2., 1., 2.
references = {
    'constant': lambda t: (1., 0., 0.),
    'ramp': lambda t: (t, 1., 0.),
    'accelerating': lambda t: (t*t/2, t, 1.),
    'sinusoid': lambda t: (np.sin(t), np.cos(t), -np.sin(t)),
}
def closed_loop(reference):
    def dynamics(t, state):
        position, velocity, estimate = state
        x_m, v_m, a_m = reference(t)
        e, de = position-x_m, velocity-v_m
        s = de+lam*e
        nu = a_m-2*lam*de-lam*lam*e
        return [velocity, estimate*nu/m, -gamma*nu*s]
    return dynamics
final = {}
for name, reference in references.items():
    result = solve_ivp(closed_loop(reference), (0, 60), [0., 0., 1.],
                       rtol=1e-10, atol=1e-12)
    assert result.success
    position, velocity, estimate = result.y[:, -1]
    x_m, v_m, a_m = reference(60.)
    assert abs(position-x_m) < 1e-6 and abs(velocity-v_m) < 1e-6
    final[name] = float(estimate)
assert abs(final['accelerating']-m) < 1e-6
assert abs(final['sinusoid']-m) < 1e-6
assert abs(final['constant']-m) > .2 and abs(final['ramp']-m) > .2
print({name: round(value, 6) for name, value in final.items()})

Seven exercises with complete solutions

The next recording finishes the indirect adaptive-control comparison and then moves toward demonstrations and reinforcement learning. The central distinction carries forward: a successful behavior, a well-fitted predictor, and an identified physical model are different achievements.