Source map
Transcript spans locate lecture material. They do not measure the proportion of original exposition.
| Chapter | Section | Transcript span | Lecture material | Research extension | Teaching construction | Evidence class |
|---|---|---|---|---|---|---|
| lecture-01-models-constraints-and-control.html | A cart that reaches the mark but cannot stop | 435.12–1017.28 s | Feedback loop, reference, measurement, disturbance, tracking, robustness, and motivation for performance/planning/learning; official slides pp. 9-11 block diagram (summing junction, controller, system, output, sensor), home-heating instance and the entry points of disturbance and sensor noise, p. 12 internal versus bounded-input bounded-output stability, and p. 13 performance, planning and learning gaps. | Precise distinction between Lyapunov stability, asymptotic stability, and a finite-time arrival specification. | One-meter two-second bounded-acceleration cart task. | definitions and original formalization |
| lecture-01-models-constraints-and-control.html | A taxonomy organized by what is computed | 1017.28–1614.29 s | Open-loop, closed-loop, direct and indirect methods, dynamic programming, MPC, adaptive control, imitation learning, and model-based/model-free reinforcement learning; official slides p. 14 course map: DP and HJB/HJI as closed-loop branches with reachability, open-loop versus closed-loop computation trade-off, MPC as the bridge, adaptive optimal control with imitation learning and model-based and model-free RL defined. | DAgger's action-dependent observation distribution illustrates a specific imitation-learning limitation; direct/indirect definitions supplement the introductory overview. | Method comparison by computed object and unresolved failure question. | classification and research-supported limitation |
| lecture-01-models-constraints-and-control.html | A state must contain enough information to predict motion | 1615.06–2306.43 s | Three ingredients of optimal control, ODEs as predictors, Euler intuition, state/control histories, double integrator, and conversion to first-order state equations; official slides p. 18 three ingredients named, p. 19 component form of the model, and pp. 21-24 slide notation s, a, A, B translated. | ODE uniqueness qualifications; distinction between exact held-input dynamics and a numerical approximation. | Exact cart sampling derivation and half-meter one-step Euler discrepancy. | definition and derivation |
| lecture-01-models-constraints-and-control.html | Feasibility precedes performance | 2306.43–2452.08 s | Boundary, state, and control constraints; admissible controls and trajectories; official slides p. 25 general boundary conditions x(t0)=x0, x(tf)=xf and componentwise state and control bounds instantiated on the cart, plus a non-bound (disk-avoidance) constraint for 'and many more'. | None required for the independently derived ideal-model bound. | Minimum feasible rest-to-rest duration derived from two velocity bounds and an attaining pulse construction. | theorem under explicit ideal-model assumptions |
| lecture-01-models-constraints-and-control.html | The objective encodes a decision, including its compromises | 2452.34–3318.61 s | Terminal/running costs, fixed/free duration, functional optimization, additive structure, existence/uniqueness, and cost/reward conventions; official slides p. 27 notation h, g, t0, tf translated, specified or free final time with a time price, time-effort conflict and a tracking running cost, and pp. 28-29 optimal trajectory-control pair, global minimality, infinite-dimensional decision variable, and maximization as minimization of the negative with the reward vocabulary. | Finite optimization terminology from Boyd and Vandenberghe; explicit objective units and nonattainment. | Two-pulse hard versus soft endpoint example, exact scalar optimizer, and effort-only free-time nonexistence proof. | derivation and counterexample |
| lecture-01-models-constraints-and-control.html | A trajectory is not a feedback policy | 3319.82–3564.27 s | Open-loop history versus state-dependent policy and consequences of off-nominal states; official slides p. 30 optimal control law or policy and its important example, linear state feedback F x (written -K x), applied to the cart's proportional-derivative law. | Unsaturated linear feedback stability established separately from finite-time feasibility. | Midpoint velocity-kick calculation, loss of braking feasibility, and characteristic-root proof. | derivation with explicit guarantee boundary |
| lecture-01-models-constraints-and-control.html | Sampling the dynamics and closing the loop through optimization | 3564.27–3801.74 s | Discrete state recursion, summed stage costs, and receding-horizon execution of open-loop solves; official slides pp. 31-32 general discrete-time problem x_{k+1}=f(x_k,u_k,k), u_k in U, cost J(x_0;u_0,...,u_{N-1}) and optimal value J*(x_0), with a cart instance and the stochastic (Markov decision process) extension. | Rawlings, Mayne, and Diehl support the distinction between an MPC optimization and its closed-loop guarantees. | Exact two-step elimination, sampling-period cost scaling, and intermediate-constraint warning. | derivation and implementation distinction |
| lecture-01-models-constraints-and-control.html | Uncertainty requires a model and an information order | 3801.74–4013.44 s | Disturbance variables, stochastic and worst-case models, and differential-game motivation. | Explicit process versus measurement uncertainty and decision/disturbance quantifier order. | Two-disturbance cancellation counterexample. | definition and counterexample |
| lecture-01-models-constraints-and-control.html | Necessary conditions filter candidates; they do not certify success | 4013.44–4396.94 s | Finite unconstrained optimization, local/global minima, and introduction to gradient necessity; official slides p. 34 unconstrained nonlinear program with C1/C2 assumptions and notation f, x, n translated, p. 35 formal epsilon-ball definitions of local, global and strict minima, and p. 43 stationary points as a filter with its existence premise; pp. 36-42 are delivered in recording 2 and linked to the lecture-2 chapter. | Negative-gradient proof, boundary counterexample, and convex scalar sufficiency are developed beyond the recording's short ending. | Cubic/quartic counterexamples and justified one-dimensional clipping. | proof and counterexample |
| lecture-01-models-constraints-and-control.html | Executable model, exercises, and limits | Original teaching; no recording span | No additional recording span claimed; exercise 7 extends official slides pp. 29, 35 and 43 (non-unique minima, strict minima, and the stationary-point filter). | None. | Tested two-pulse numerical example, seven original fully worked exercises, and explicit scope of results. | original teaching construction and finite numerical checks |
| lecture-02-descent-and-equality-constraints.html | A stopped cart with a nonzero gradient | Original teaching; no recording span | No additional recording span claimed. | Equality-constrained quadratic programming. | Normalized two-input cart, exact terminal maps, and nonzero gradient at the feasible optimum. | original teaching construction |
| lecture-02-descent-and-equality-constraints.html | Necessary conditions ask what a minimum cannot do | 113–824 s | Taylor reasoning for first-order stationarity and positive-semidefinite Hessian necessity; official lecture-1 slides p. 37 coordinate-perturbation inequalities (routed from the lecture-1 cross-check; recording 6:38-10:46) and pp. 36 and 38 second-order expansion at a general point (routed from the lecture-1 review). | Explicit small-remainder justification. | Cubic and signed-quartic counterexamples. | source-grounded explanation with labeled derivations |
| lecture-02-descent-and-equality-constraints.html | When local information becomes a guarantee | 824–1490 s | Positive-definite Hessian sufficiency, convex sets/functions, local/global minima and strict convexity; official lecture-1 slides p. 42 single statement that zero gradient is necessary and sufficient for a global minimum of a C1 convex function on an open convex set (routed from the lecture-1 review; recording 22:48-23:48). | Uniform Taylor-remainder proof and supporting-plane derivation; existence distinguished from uniqueness. | Nonattainment example and application to the cart feasible line. | source-grounded explanation with labeled derivations |
| lecture-02-descent-and-equality-constraints.html | A downhill direction is not a safe step length | 1624–2379 s | Iterative descent and first-order descent direction condition; official slides pp. 4-8 monotone iterative-descent goal from an initial guess, gradient-method class, half-line first-order decrease, angle reading of a descent direction, zero-gradient stop, and strict-decrease meaning of gradient descent with its two tuning choices (recording 27:04-39:39). | Exact quadratic error dynamics and eigenmode analysis. | Cart step-size regimes and ill-conditioned two-coordinate quadratic. | source-grounded explanation with labeled derivations |
| lecture-02-descent-and-equality-constraints.html | Curvature chooses a direction; line search controls the distance | 2383–3090 s | Positive-definite scaled gradients, inverse-Hessian Newton direction, constant/exact/diminishing step choices and limits of stopping tests; official slides pp. 9-11 steepest-descent name, minimization rule, diminishing steps without per-iteration descent and with divergent sum, convergence-to-stationary-points, termination and rate questions left outside the course, and derivative-free coordinate descent and Nelder-Mead (recording 40:14-51:30). | Armijo backtracking and proof of one-search termination. | Four explicitly calculated cart trials; distinction between local model and global guarantee. | source-grounded explanation with labeled derivations |
| lecture-02-descent-and-equality-constraints.html | Equality constraints remove directions, not necessarily gradients | 3090–4497 s | Equality constraints, tangent directions, multiplier stationarity and constraint-gradient geometry; official slides pp. 12, 14 and 18-20 feasible-direction and penalty viewpoints, C1 assumption, subspace of first-order feasible variations, span and orthogonality readings of the multiplier equation, regular points, penalty and elimination proof routes (recording 52:34-54:35 and 1:07:45-1:16:57), and the second-order conditions to which p. 19 refers (official slides p. 19; not located in the recording). | Regular level-set interpretation and nullspace/row-space reasoning. | Degenerate squared constraint and cart multiplier. | source-grounded explanation with labeled derivations |
| lecture-02-descent-and-equality-constraints.html | The lecture's circle example has two answers to stationarity | 3367–4065 s | Linear objective on a circle, two multiplier candidates, and tangent level sets; official slides pp. 15-18 first-order feasible variations checked at (-1,-1) and both candidates classified by the second-order conditions to which p. 19 refers. | Compactness and Cauchy-Schwarz global classification. | Explicit nonconvex feasible-set distinction. | source-grounded explanation with labeled derivations |
| lecture-02-descent-and-equality-constraints.html | A saddle-point linear system solves the cart | 4497–4735.89 s | Constraint regularity, multiplier uniqueness at a given point, and Lagrangian stationary equations; official slides p. 21 Lagrangian on R^(n+m) and compact conditions grad_z L = 0, grad_lambda L = 0 as n+m equations in n+m unknowns (recording 1:16:57-1:18:55). | Equality quadratic-program block system and nonsingularity proof; correction of equation-count inference. | Exact cart system, residual interpretation and finite penalty distinction. | source-grounded explanation with labeled derivations |
| lecture-02-descent-and-equality-constraints.html | A multiplier measures the cost of changing the requirement | 1490–1555 s | Sensitivity mentioned as a broader topic, without this derivation. | Smooth value derivative from stationarity and equality differentiation. | Cart value slope, multiplier sign convention and requirement tradeoff. | source-grounded explanation with labeled derivations |
| lecture-02-descent-and-equality-constraints.html | Executable checks, exercises, and limits | Original teaching; no recording span | No additional recording span claimed; source note names the official lecture-2 slide additions, and exercise 7 applies the elimination route of official slides p. 20 to the circle example. | None beyond the theory identified in earlier sections. | Original executable equality solver, independent scalar checks, seven fully solved exercises and limits. | original teaching construction |
| lecture-03-inequalities-and-variations.html | An acceleration bound changes which improvement is possible | Original teaching; no recording span | No additional recording span claimed. | Finite inequality geometry. | Cumulative bounded two-pulse cart example. | original teaching construction |
| lecture-03-inequalities-and-variations.html | An active set is a set of indices, not a feasible region | 217.87–857.46 s | Inequality problem, active/inactive definitions, triangle illustration, and local removal of inactive constraints. | Continuity proof and distinction between local deletion and global solving. | Explicit active sets at three points and explanation of equality restriction. | lecture-derived with labeled clarification and extension |
| lecture-03-inequalities-and-variations.html | Four conditions keep the geometry and the signs consistent | 857.46–1415.28 s | Lagrangian, regularity, KKT stationarity, inactive multipliers, and nonnegative inequality multipliers; official lecture-2 slides pp. 25-26, delivered in this recording at 940.9-1031.5 s and routed from the lecture 2 review: stationarity from the equality-multiplier theorem applied to the problem that keeps only the active inequalities as equalities, extension to all inequalities with zero inactive multipliers, and the separate sign argument that nonnegativity requires. | Complementarity notation, sign example, empty-active-set clarification and nonconvex insufficiency. | Active but zero-multiplier cart threshold and scalar counterexamples. | lecture-derived with labeled clarification and extension |
| lecture-03-inequalities-and-variations.html | Work both cases, then certify what survives | 1545.15–2025.23 s | Source quadratic objective under one half-plane constraint; active case rejected and interior candidate retained. | Convex KKT sufficiency proof and sensitivity interpretation. | Cart active regimes and value function. | lecture-derived with labeled clarification and extension |
| lecture-03-inequalities-and-variations.html | The decision can be an entire function | 2029.87–2782.26 s | Indirect/direct method distinction, functionals, linearity and continuous signals; official slides p. 2 open-loop placement of indirect methods, p. 3 analytical advantage and Kirk reading, p. 4 general optimal-control problem with admissible control and trajectory and the finite-dimensional stationarity analogy, p. 5 calculus-of-variations definition, and p. 6 linear-functional definition on the domain Omega (transcript 2029.9-2753.6 s). | Domain and representation qualifications. | Continuous cart connects finite pulses with a richer admissible class. | lecture-derived with labeled clarification and extension |
| lecture-03-inequalities-and-variations.html | Closeness of trajectories depends on what the norm observes | 2782.29–3129.3 s | Function norms, supremum example, closeness and local extrema; official slides p. 8 local maxima and extremals (transcript 2996.9-3108.0 s). | C1 topology for derivative-dependent objectives and almost-everywhere norm qualification. | Oscillatory sequence proving uniform position closeness does not control velocity effort. | lecture-derived with labeled clarification and extension |
| lecture-03-inequalities-and-variations.html | A first variation is a linear map, not the full change | 3144.69–3931.66 s | Increment, linear first variation, small remainder, stationary functional, and admissible perturbations; official slides pp. 4 and 8 differential analogy, p. 9 increment notation and remainder-factor form of differentiability, and p. 10 fundamental theorem with its boundary hypothesis and proof by contradiction (transcript 3093.0-3894.8 s). | Bounded Fréchet derivative definition and distinction from directional derivatives; remainder need not be quadratic. | Exact quadratic functional increment and fractional-power remainder counterexample. | lecture-derived with labeled clarification and extension |
| lecture-03-inequalities-and-variations.html | Integration by parts turns stationarity into a differential equation | 3931.89–4970.77 s | Fixed endpoints, Taylor expansion, integration by parts, vanishing endpoint perturbations, fundamental lemma and Euler equation; official slides p. 11 simplest optimal-control reading, pp. 12-13 single-integral increment and Taylor remainder bound, and p. 15 fundamental lemma for the admissible test-function class (transcript 3894.8-4287.0 s and 4841.9-4935.5 s). | Bump-function proof, degeneracy qualification, solved quadratic functional and global certificate. | Hyperbolic-sine trajectory compared with the straight line. | lecture-derived with labeled clarification and extension |
| lecture-03-inequalities-and-variations.html | Original extension: the minimum-energy cart is a cubic trajectory | Original teaching; no recording span | No claim that this higher-derivative calculation appears in this recording. | Double integration by parts and global quadratic identity. | Rest-to-rest cubic, effort scaling, and actuator-limit counterexample. | original teaching construction |
| lecture-03-inequalities-and-variations.html | Checks, exercises, and boundary of guarantees | Original teaching; no recording span | No additional recording span claimed; exercise 7 extends official slides pp. 16 and 18 (the fixed-endpoint Euler equation as a necessary condition only). | Natural endpoint condition derived as an explicitly original exercise. | Analytic/quadrature cost-gap comparison, seven solved exercises and guarantee limitations. | original teaching construction |
| lecture-04-costates-and-free-final-time.html | Two reasonable arrival times cannot both be optimal | 3217.33–3451.06 s | Source rest-to-rest transfer from position ten to zero with free duration and time/effort tradeoff; official slides p. 2 placement of indirect methods on the open-loop branch beside direct methods, with closed-loop methods and MPC later (transcript 5.3-121.3 s). | Dimensional consistency and reduced fixed-duration value. | Two-versus-five-second quantitative comparison. | source-grounded derivation with labeled extensions |
| lecture-04-costates-and-free-final-time.html | The source's shortest path is a boundary-value problem | 407.63–1039.1 s | Arc-length objective, Euler equation, straight-line stationary solution and split boundary conditions; official lecture-3 slides p. 16 (routed from lecture 3, delivered in this recording at 283.2-375.9 s): the general Euler equation is a nonlinear, time-varying ordinary differential equation, second order only where g_xdotxdot is nonsingular, with boundary conditions split between x(t_0) and x(t_f). | Integral triangle-inequality global certificate and coordinate-unit qualification. | Explicit interpolation formula. | source-grounded derivation with labeled extensions |
| lecture-04-costates-and-free-final-time.html | A moved endpoint is not a perturbation at the old final time | 1080.37–1911.44 s | Generalized terminal conditions and shortest curve to the line t=5; official slides pp. 3-6 and 8 regularity assumptions and Kirk reading, Euler equations independent of the terminal specification, equation counts of Kirk's Section 4.3 table, coupled-target condition (fifth table row) with orthogonality check, and mixed fixed/free components (transcript 1080.4-1439.4 s and 1860.9-1876.3 s). | Chain-rule derivation, coupled terminal manifold, and limitation of a time-origin shift. | Four-case table with exact allowed variations. | source-grounded derivation with labeled extensions |
| lecture-04-costates-and-free-final-time.html | A costate is a multiplier function for a dynamical constraint | 1911.44–2410.42 s | Differential constraints, augmented integrand, multiplier functions, and path toward optimal control; official slides pp. 9-12 constrained problem in w with n differential constraints, m of n+m components independent, Euler equations of the augmented integrand plus the constraint, finite-dimensional analogy, and the three-stage variational roadmap with optimize-then-discretize (transcript 1876.3-2410.4 s). | Multiplier-versus-penalty distinction and consistent Hamiltonian bookkeeping. | Costate notation adapted to preserve cart position p. | source-grounded derivation with labeled extensions |
| lecture-04-costates-and-free-final-time.html | Derive the three equations before memorizing them | 2410.42–3118.24 s | State/costate/control equations, elimination of controls, boundary-value structure; source proof sketch at 4667.98–5083.18; official slides p. 13 full problem statement, admissibility, unbounded regions and notation (transcript 2410.4-2539.0 s), p. 11 multiplier rule specialized to w=(x,u), and pp. 20-23 proof-sketch order with the fixed-endpoint controllability qualification and cart Gramian (transcript 4759.6-5070.3 s). | Explicit variation with dimensions, normality qualification, algebraic invertibility and control-stationarity limitations. | Sign and transpose checks. | source-grounded derivation with labeled extensions |
| lecture-04-costates-and-free-final-time.html | Free arrival time adds a Hamiltonian boundary equation | 2567.92–2875.38 s | Hamiltonian terminal boundary formula and fixed/free cases; official slides pp. 15-16 Kirk Section 5.1 table, both-free case, constants-of-integration counts (2n, 2n+1, 2n+k) and the cart's five equations (transcript 2779.6-2817.5 s and 3088.2-3149.8 s), and the terminal-surface row with multipliers d_i (p. 15 row 3, not discussed in the recording). | Endpoint chain-rule derivation and autonomous Hamiltonian constancy. | Distinction between free time and independently free state. | source-grounded derivation with labeled extensions |
| lecture-04-costates-and-free-final-time.html | Solve the lecture's free-time cart in state–costate form | 3217.33–4644.27 s | Double-integrator dynamics, Hamiltonian, affine costates and control, five endpoint conditions, and T=(1800 beta/alpha)^(1/5). | Full constants, sign correction, and numerical values. | General signed-displacement solution and independent stationarity checks. | source-grounded derivation with labeled extensions |
| lecture-04-costates-and-free-final-time.html | Stationarity chooses the duration; a separate argument proves the minimum | 4595.78–4644.27 s | Source intuition for how effort/time weights affect duration. | Strict convexity and coercivity of the reduced value, boundary cases, and bound-related limitations. | Fifth-root scaling, cost ratio, and degenerate-parameter examples. | source-grounded derivation with labeled extensions |
| lecture-04-costates-and-free-final-time.html | Why a boundary-value solve can be numerically sensitive | Original teaching; no recording span | Motivated by the source's discussion of numerical boundary-value solvers; no claim that this example is in the recording. | Forward shooting sensitivity distinguished from inverse-map conditioning. | Exactly solved unstable scalar plant and endpoint-error amplification. | original teaching construction |
| lecture-04-costates-and-free-final-time.html | Verification, exercises, and limits | Original teaching; no recording span | No additional recording span claimed; Exercise 7 applies the terminal-surface row of official slides p. 15. | Global versus necessary-condition boundaries. | Independent endpoint, Hamiltonian, scalar finite-difference checks and seven fully worked exercises. | original teaching construction |
| lecture-05-bounded-controls-and-boundary-solvers.html | Clipping the previous answer misses the target | Original teaching; no recording span | No additional recording span claimed. Official slides p. 2 roadmap: indirect open-loop methods placed before direct methods, MPC and closed-loop methods (recording 0:05-3:24). | Dynamical endpoint constraints are not preserved by input clipping. | Bounded rest-to-rest cart with a measured clipping shortfall. | original teaching construction |
| lecture-05-bounded-controls-and-boundary-solvers.html | The Hamiltonian is minimized pointwise along a coupled trajectory | 655.18–1444.82 s | One-sided admissible variations and replacement of control stationarity by Hamiltonian minimization.; official slides pp. 4-8: unbounded problem statement with C^2 terminal cost and fixed initial time and state, full terminal boundary relation and its specializations, slide-to-chapter notation correspondence, actuator and safety constraint motivation with state constraints deferred, relative-minimum increment Delta J = delta J + higher-order terms, one-sided admissible variations and delta J >= 0 (recording 3:24-8:46 and 11:28-21:29), reduced to the pointwise variational inequality with a saturation-law check (recording 27:22-28:56). | Normal/abnormal multiplier boundary, almost-everywhere interpretation, needle-variation explanation and terminal-time caveat. | Separating pointwise input minimization from global trajectory optimality. | source-grounded structure with explicit completion and qualifications |
| lecture-05-bounded-controls-and-boundary-solvers.html | The source's bounded quadratic example produces a saturation law | 1444.82–2437.2 s | Damped second-order system, quadratic state/input cost, bounded scalar Hamiltonian minimizer, and incomplete trajectory solve. | Full costate equations and free-endpoint boundary stated explicitly. | Completing-square proof and diagonal-weight generalization. | source-grounded structure with explicit completion and qualifications |
| lecture-05-bounded-controls-and-boundary-solvers.html | Minimum time drives a linear Hamiltonian to an endpoint | 2572.38–3423.17 s | Control-affine minimum-time problem, endpoint input choices and singular-arc caveat.; official slides p. 15 Kirk Sec. 5.4 linear case: controllability of each (A,b_i) excludes singular intervals, real eigenvalues bound switches by n-1, complex-eigenvalue and uncontrollable boundaries (recording 52:18-57:03). | Isolated-zero versus singular-interval distinction. | Complete double-integrator bang-bang transfer and independent velocity-area global proof. | source-grounded structure with explicit completion and qualifications |
| lecture-05-bounded-controls-and-boundary-solvers.html | Minimum fuel creates an interval in which doing nothing is optimal | 3423.17–4114.35 s | Absolute-input Hamiltonian, positive/negative slope cases and bang-off-bang law. | Tie-set completion and fuel-model qualification. | Complete fixed-duration cart solution with rearrangement-based optimality proof. | source-grounded structure with explicit completion and qualifications |
| lecture-05-bounded-controls-and-boundary-solvers.html | Original extension: solve the bounded minimum-energy cart completely | Original teaching; no recording span | Uses source's clipped-affine structure but the complete cart solution is not in the recording. | Convex Hamiltonian certificate and limiting normality boundary. | Saturated-ramp slope, regime thresholds, exact effort and numerical comparison. | original teaching construction |
| lecture-05-bounded-controls-and-boundary-solvers.html | Clipping depends on the metric and on the constraints | 4114.35–4240.51 s | Diagonal positive-definite input weights enable separable saturation. | Coupled quadratic weighted projection. | Two-coordinate counterexample to naive clipping. | source-grounded structure with explicit completion and qualifications |
| lecture-05-bounded-controls-and-boundary-solvers.html | Necessary conditions presuppose an optimum that may not exist | Original teaching; no recording span | Official slides p. 22; not located in the recording (skipped at 1:10:06 and not delivered in the lecture-6 recording). The slide states that existence and uniqueness are not guaranteed and poses the x' = u nonuniqueness and x' = -x + u nonexistence minimum-fuel examples. | No external source; existence and uniqueness separated from the necessary conditions. | Lower bound |x_0| and an explicit optimal family for x' = u with the minimum-principle tie lambda = 1; lower bound e^{-T}|x_0| and a coast-then-burn sequence with cost ln(1 + e^{-tau_c}) tending to zero for x' = -x + u. | Official slides p. 22; not located in the recording; original complete solutions |
| lecture-05-bounded-controls-and-boundary-solvers.html | A boundary solver returns a solution of equations, not an optimality certificate | 4206.06–4678.13 s | Shooting/collocation overview, solve_bvp interface and z1'=z2,z2'=-abs(z1) example.; official slides p. 25 two-point standard form with unknown parameters and n + k residuals, three-stage Lobatto IIIA collocation, and the p. 26 Ascher-Russell reference with a pointer to chapter 6 for free-time rescaling (recording 1:12:39-1:13:37). | Solver residual semantics verified from official SciPy documentation. | Two independently derived analytic branches and monotone-derivative check. | source-grounded structure with explicit completion and qualifications |
| lecture-05-bounded-controls-and-boundary-solvers.html | Executable branch checks, exercises, and limits | Original teaching; no recording span | No additional recording span claimed. Exercises 7-8 extend official slides p. 22 (nonexistence) and p. 15 (linear switching count, using the p. 10 dynamics). | No new external theorem claimed. | Tested original two-branch solve_bvp implementation, bounded-profile checks, eight fully worked exercises. | original teaching construction |
| lecture-06-direct-transcription-and-convexification.html | Make the unknown duration a variable on a fixed interval | 5.33–934 s | Normalized time, constant duration state, cart boundary residual and solve_bvp demonstration. Official slides p. 3: solve_bvp setup (21-node mesh on [0,1], linear position guess, zero velocity and costate guesses, duration guess 1), with a reproduced convergence and tolerance check. Ascher and Russell (1981) SIAM Review reformulation reference, spoken at 112.4-140.6 s (official lecture 5 slides p. 26), cited in text without a link. | Primary-source context integrated where relevant; original calculations explicitly identified. | Positivity via a logarithmic unknown parameter; boundary-condition count. | source-grounded structure with explicit completion and qualifications |
| lecture-06-direct-transcription-and-convexification.html | Choose which variables the optimizer is allowed to move | 934–1435 s | Direct/indirect and control-only/state-and-control representations. Official slides pp. 2-3: open-loop placement and optimize-then-discretize versus discretize-then-optimize order (recording 934.4-1165.8 s), general problem with terminal set, control set, zero terminal cost and zero initial time; pp. 5 and 13: names state and control parameterization, collocation, simultaneous, direct transcription, control parameterization and direct shooting. | Primary-source context integrated where relevant; original calculations explicitly identified. | Sparse coupling versus condensed sensitivity. | source-grounded structure with explicit completion and qualifications |
| lecture-06-direct-transcription-and-convexification.html | A defect equation defines a finite approximation | 1435–1777 s | Euler dynamics and rectangle cost discretization. Official slides pp. 6 and 10: nonuniform-grid simultaneous and shooting programs with terminal set and control set; computed-state notation x(t_i) translated. | Primary-source context integrated where relevant; original calculations explicitly identified. | Exact ZOH comparison, rest-to-rest endpoint cancellation and intersample violation. | source-grounded structure with explicit completion and qualifications |
| lecture-06-direct-transcription-and-convexification.html | The river example separates a model from a solver anecdote | 1777–2582 s | Zermelo navigation and tighter heading-bound warm start. Official slides pp. 7-9: notation translation, displayed Euler program with fmincon and scipy.optimize.minimize, plotted results (bound 1 inactive with peak heading about 0.8 rad, saturated arc at 0.75). | Primary-source context integrated where relevant; original calculations explicitly identified. | Heading objective interpretation and explicit illustrative flow profile. | source-grounded structure with explicit completion and qualifications |
| lecture-06-direct-transcription-and-convexification.html | Eliminating states trades sparse constraints for long sensitivities | 2582–3568.8 s | Shooting, multiple shooting and higher-order integration alternatives. Official slides pp. 11-13: unrolled river shooting recursion with corrected summation limit and close agreement of the two transcriptions' plotted results on different grids. Span shortened to end at 3568.8 s, where the new higher-order section begins. | Primary-source context integrated where relevant; original calculations explicitly identified. | Chain-rule sensitivity product and qualified solver comparison. | source-grounded structure with explicit completion and qualifications |
| lecture-06-direct-transcription-and-convexification.html | Higher-order transcriptions change the trajectory between nodes | 3568.8–3638 s | Official slides p. 14: Euler piecewise-linear C0 state with zero-order hold, Hermite-Simpson collocation (piecewise-cubic C1 state, first-order hold, collocation points, implicit integration), pseudospectral global polynomials, and shooting with RK4 explicit integration; recording 3568.8-3594.7 s names Hermite-Simpson and pseudospectral methods. | Kelly 2017 for the Hermite-Simpson defect form; Rao 2009 survey and AA203 course notes section 5.2.1 cited in text without links. | Hermite cubic midpoint derivation, cart exactness under first-order hold, Euler/RK4/Hermite-Simpson order check on a scalar test equation, and pseudospectral smoothness boundary. | official-slide content with original derivations and numerical checks |
| lecture-06-direct-transcription-and-convexification.html | Original worked case: solve both finite representations exactly | Original teaching; no recording span | No additional recording span claimed. | Primary-source context integrated where relevant; original calculations explicitly identified. | Independent condensed and simultaneous cart QP, exact finite-grid energy gap. | original teaching construction |
| lecture-06-direct-transcription-and-convexification.html | Convex subproblems approximate a nonconvex trajectory problem | 3638–4478 s | Iterative dynamics linearization, convex subproblem and Jacobians. Official slides pp. 15-18: general time-varying linearization, infeasible nominal allowed, discretized convex subproblem and four-step loop; iteration-subscript notation translated. | Primary-source context integrated where relevant; original calculations explicitly identified. | Cubic-drag Taylor remainder and distinction between local and lossless convexification. | source-grounded structure with explicit completion and qualifications |
| lecture-06-direct-transcription-and-convexification.html | A trust region and a slack variable solve different problems | 4478–4822.99 s | Trust region, artificial infeasibility and bound slacks. Official slides p. 19: SCP flowchart with artificial unboundedness, trust-region update and stopping test (recording 4541.3-4691.6 s); p. 20: warm starts, penalty and augmented Lagrangian methods and the Tedrake remark (quote at 4778-4817 s; augmented Lagrangian not spoken). | Primary-source context integrated where relevant; original calculations explicitly identified. | Scalar linearization infeasibility and finite quadratic penalty counterexample. Artificial-unboundedness trust-region example and exact augmented Lagrangian iteration on the scalar penalty example. | source-grounded structure with explicit completion and qualifications |
| lecture-06-direct-transcription-and-convexification.html | Executable checks, exercises, and limitations | Original teaching; no recording span | No additional recording span claimed. Official slides p. 21: local-versus-global, compute and dimension comparison; recording 6 closes the open-loop part at 4788.6-4823.0 s, the open-/closed-loop and compute contrast is delivered at the start of recording 7 (5.2-460.0 s); the local/global bullet is slide-only; the dimension bullet is spoken in recording 7 only within the DP complexity span (2631.8-2814.6 s, 3211.6-3243.7 s). | Primary-source context integrated where relevant; original calculations explicitly identified. | Independent QP comparisons, path counterexample and eight full solutions. Decision-variable versus value-table size count. | original teaching construction |
| lecture-07-dynamic-programming-and-lqr.html | A sequence solves one initial condition; a policy solves a family | 204.77–830.38 s | Closed-loop versus open-loop policy and discrete finite-horizon model; official slides p. 2 roadmap placement (indirect and direct open-loop methods, MPC between the open- and closed-loop branches, DP as the root of LQR, iLQR, DDP and HJB/HJI) and pp. 3, 10 and 12-17 notation correspondence (h_N, J_k, stage argument, prime transpose). | Verified primary notes integrated beside relevant modeling and LQR claims. | Deterministic continuation qualification and optimizer existence. | source-grounded structure with explicit completion and qualifications |
| lecture-07-dynamic-programming-and-lqr.html | The tail replacement argument needs a sufficient state | 830.38–1485.58 s | Principle of optimality and contradiction proof; official slides pp. 5-6 strict-inequality proof hypothesis with ties and the formal tail-subproblem statement with its displayed cost-to-go. | Verified primary notes integrated beside relevant modeling and LQR claims. | Resource-budget state augmentation and fixed-endpoint interior-segment qualification. | source-grounded structure with explicit completion and qualifications |
| lecture-07-dynamic-programming-and-lqr.html | Backward computation prices the consequence of each action | 1485.74–1941.23 s | Concatenation of immediate decision and optimal tail; backward recursion; official slides p. 10 dynamic-programming theorem for every initial state, optimality of the minimizing policy, and the all-states computational price. | Verified primary notes integrated beside relevant modeling and LQR claims. | Two-sided Bellman proof and infimum/attainment distinction. | source-grounded structure with explicit completion and qualifications |
| lecture-07-dynamic-programming-and-lqr.html | A complete source example: reuse solved tails | 1941.23–2580 s | Eight-node shortest-path example with optimal cost 18. | Verified primary notes integrated beside relevant modeling and LQR claims. | Explicit absorbing-target finite-horizon construction and greedy comparison. | source-grounded structure with explicit completion and qualifications |
| lecture-07-dynamic-programming-and-lqr.html | Global finite-model optimization can still be unaffordable | 2580–3421.9 s | Quantization, curse of dimensionality and constraint remarks; official slides p. 8 reduction in computation, quantified here by the M^N brute-force count; p. 11 discretization (forward Euler as delivered at 584-653 s, applied here to the cart), control quantization and the named curse of dimensionality. | Verified primary notes integrated beside relevant modeling and LQR claims. | Exact memory estimate, finite-model optimality boundary and state/time discretization distinction. | source-grounded structure with explicit completion and qualifications |
| lecture-07-dynamic-programming-and-lqr.html | The linear-quadratic model makes the value function finite-dimensional | 3424.27–3955.44 s | LQR model, terminal value and quadratic Bellman expansion. | Verified primary notes integrated beside relevant modeling and LQR claims. | Units, terminal penalty versus equality and dimensions. | source-grounded structure with explicit completion and qualifications |
| lecture-07-dynamic-programming-and-lqr.html | Completing the square derives both the gain and the value | 3955.82–4508.94 s | Positive control curvature, linear gain and Riccati recursion; official slides pp. 16-17 substitution route to the closed-loop form of the Riccati update and the one-half scaling of P_k. | Verified primary notes integrated beside relevant modeling and LQR claims. | Detailed square completion, positive-semidefinite induction and global excess-cost certificate. | source-grounded structure with explicit completion and qualifications |
| lecture-07-dynamic-programming-and-lqr.html | The cart experiment, implementation, exercises, and limits | Original teaching; no recording span | No additional recording span claimed; exercise 7 extends official slides pp. 11 and 15 (forward Euler model and last-step gain). | Verified primary notes integrated beside relevant modeling and LQR claims. | Original cart impulse comparison, independent dense QP and seven fully solved exercises. | original teaching construction |
| lecture-08-tracking-ilqr-and-ddp.html | Affine LQR explains why a control law needs a feedforward term | 501.42–1147.63 s | Cross quadratic, linear and constant cost terms; affine dynamics and affine control; official slides p. 4 cross-term-only LQR with pure linear feedback and quadratic value (plus the change of input variable that removes the cross term), p. 5 augmented-state [x;1] form of the affine problem, pp. 4-5 notation correspondence, and p. 3 course-notes reading (Sections 3.1-3.2). | Primary implementation and constrained-variant research integrated in the prose. | Full generalized Riccati recursion, dimensions and scalar drift check. | source-grounded structure with explicit completion and qualifications |
| lecture-08-tracking-ilqr-and-ddp.html | A feasible reference turns tracking into regulation of the error | 1148.43–1968.37 s | Linear tracking, deviations, reference feasibility and feedforward plus feedback; official slides p. 6 explicit quadratic tracking objective with terminal weight, designer-chosen weights, and the penalty on u - u_bar rather than u. | Primary implementation and constrained-variant research integrated in the prose. | Infeasible reference defect and finite-horizon stability qualification. | source-grounded structure with explicit completion and qualifications |
| lecture-08-tracking-ilqr-and-ddp.html | A nonlinear tracker is locally accurate along a moving reference | 1968.37–2522.74 s | Linearization along a feasible nonlinear nominal trajectory and time-varying gains. | Primary implementation and constrained-variant research integrated in the prose. | Cubic-drag discrete cart Jacobians and exact Taylor remainder. | source-grounded structure with explicit completion and qualifications |
| lecture-08-tracking-ilqr-and-ddp.html | Trajectory optimization deliberately moves away from the nominal path | 2526.45–3424.83 s | Quadratic cost expansion to improve a feasible rather than desired trajectory; official slides pp. 8-10 identification of the Taylor coefficients with the generalized LQR step (q_k, r_k, Q_k, R_k, H_k), zero offset for a feasible nominal, and the affine reading in absolute coordinates. | Primary implementation and constrained-variant research integrated in the prose. | Gradient interpretation in the persistent cart example. | source-grounded structure with explicit completion and qualifications |
| lecture-08-tracking-ilqr-and-ddp.html | The local Bellman step produces both a correction and a feedback gain | 3424.83–3484.5 s | Backward and forward passes; nonlinear dynamics must be propagated; official slides pp. 10-11 outer iteration (re-linearize about the accepted rollout), feasibility of every iterate, and iLQR as a closed-loop recursion that produces an open-loop trajectory for later tracking. | Primary implementation and constrained-variant research integrated in the prose. | Full derivative and value recursion with explicit matrix dimensions and gain conventions. | source-grounded structure with explicit completion and qualifications |
| lecture-08-tracking-ilqr-and-ddp.html | Regularization and line search address different failure modes | 3484.5–3813.55 s | SCP comparison, initialization, damping, termination and indefinite curvature; official slides p. 12 SCP loop (LOCP, Euler-discretized DLOCP, convex solve) set against the iLQR loop, and p. 13 deviation penalty and spectrum shift, line search, local minima and initialization, and Tassa's thesis Section 2.2.3 (state-based regularization, mu schedule, expected-reduction line search). | Primary implementation and constrained-variant research integrated in the prose. | Consistent value update for modified gains and exact-gradient stopping check. | source-grounded structure with explicit completion and qualifications |
| lecture-08-tracking-ilqr-and-ddp.html | DDP retains curvature of the dynamics inside the Bellman expansion | 3815.02–4439.44 s | Direct quadratic Bellman approximation and second-order dynamics derivatives; official slides pp. 15-16 Q-function definition and its second-order Taylor expansion, and pp. 16-18 notation correspondence (Q_k, c_x, f_x, v_{k+1}, V_{k+1}, l_k, L_k). | Primary implementation and constrained-variant research integrated in the prose. | Chain-rule derivation, tensor contraction interpretation, cubic-drag specialization. | source-grounded structure with explicit completion and qualifications |
| lecture-08-tracking-ilqr-and-ddp.html | Original curvature counterexample, executable optimizer, and exercises | Original teaching; no recording span | No additional recording span claimed; exercise 7 extends official slides p. 13 (local minima and initialization). | Primary implementation and constrained-variant research integrated in the prose. | Exact scalar curvature versus oversized step, tested nonlinear iLQR, linear-limit check and seven full solutions. | original teaching construction |
| lecture-09-stochastic-control-and-markov-values.html | A Markov model specifies both uncertainty and available information | 35.54–845.14 s | State-action conditional disturbance law, observation order and expected objective; official slides p. 2 course placement on the closed-loop DP branch (5.4-35.5 s) and pp. 3 and 5 MDP name, state-dependent admissible set, slide notation and Powell's Rosetta Stone report for cross-community notation (35.5-451.8 s, 935.0-1020.6 s). | Primary stochastic DP and LQR references integrated beside relevant claims. | Information-order counterexample, state sufficiency and integrability conditions. | source-grounded structure with explicit completion and qualifications |
| lecture-09-stochastic-control-and-markov-values.html | Expected cost is a decision criterion, not a tail guarantee | 845.14–1049.49 s | Risk-neutral interpretation and limits of expectation-only optimization. | Primary stochastic DP and LQR references integrated beside relevant claims. | Rare-catastrophe example and qualification for other risk recursions. | source-grounded structure with explicit completion and qualifications |
| lecture-09-stochastic-control-and-markov-values.html | Conditional expectation turns a trajectory distribution into one-step recursion | 1049.49–1427.06 s | Stochastic principle of optimality and one-step Bellman recursion; official slides pp. 4, 6-8 tail subproblem and tail policy, all-tail-subproblems intuition, DP theorem J*(x_0) = J_0(x_0) with argmin policy, per-stage expectation over w_k only, and additive cost plus Markov law as the enabling assumptions (1020.6-1390.0 s). | Primary stochastic DP and LQR references integrated beside relevant claims. | Tower-property derivation and unreachable-state policy qualification. | source-grounded structure with explicit completion and qualifications |
| lecture-09-stochastic-control-and-markov-values.html | The inventory example: cost uses unmet demand even when the next state clips it away | 1427.06–2765.1 s | Capacity-two inventory, demand probabilities, three-stage objective and sample Bellman computation; official slides p. 10 inventory recursion in closed form with state-dependent admissible orders (1959.8-2179.0 s). | Primary stochastic DP and LQR references integrated beside relevant claims. | All nine values and independent exhaustive policy verification; advance-information calculation. | source-grounded structure with explicit completion and qualifications |
| lecture-09-stochastic-control-and-markov-values.html | Zero-mean additive noise changes the LQR value by a trace term | 2765.1–3275.57 s | Quadratic-plus-constant stochastic LQR; trace cost; unchanged gain; official slides pp. 12-14 constant-carrying induction hypothesis, P_{k+1} stage index and notation correspondence (2831.3-3194.5 s). | Primary stochastic DP and LQR references integrated beside relevant claims. | Trace proof, non-Gaussian moment sufficiency, initial-state expectation and exact noise enumeration. | source-grounded structure with explicit completion and qualifications |
| lecture-09-stochastic-control-and-markov-values.html | The unchanged gain fails when the control changes uncertainty | Original teaching; no recording span | Source explicitly notes covariance must not depend on control; no separate span claimed. | Primary stochastic DP and LQR references integrated beside relevant claims. | Multiplicative-noise one-step closed-form counterexample. | original teaching construction |
| lecture-09-stochastic-control-and-markov-values.html | An infinite stationary horizon replaces a terminal boundary by a fixed point | 3275.92–4198.38 s | Stationary infinite-horizon reward formulation, discount, optimal and fixed-policy equations; official slides pp. 15-16 MDP tuple, kernel and reward built from the disturbance form, slide and reinforcement-learning notation, stationarity rationale and optimal value as a function of the initial state (3361.2-3725.2 s, 4077.9-4168.9 s). | Primary stochastic DP and LQR references integrated beside relevant claims. | Bounded finite-model assumptions, tail bound, stationary-policy existence qualification and evaluation inverse proof. | source-grounded structure with explicit completion and qualifications |
| lecture-09-stochastic-control-and-markov-values.html | State-action values retain the action needed for model-free selection | 4198.38–4615.22 s | Optimal and fixed-policy Q equations; action selection without separate transition evaluation; official slides p. 19 known-model DP versus model-based and model-free reinforcement learning (4363.4-4540.7 s; fuller statement in recording 10, Fl5EjGhQjgs 318.6-443.6 s). | Primary stochastic DP and LQR references integrated beside relevant claims. | Action-gap bound and distinction from local DDP Q approximation. | source-grounded structure with explicit completion and qualifications |
| lecture-09-stochastic-control-and-markov-values.html | Executable checks, exercises, and limits | Original teaching; no recording span | No additional recording span claimed; Exercise 7 completes the stage-zero comparisons behind official slides pp. 10-11. | Primary stochastic DP and LQR references integrated beside relevant claims. | Exhaustive finite-policy/demand enumeration, stochastic LQR calculation and seven full solutions. | original teaching construction |
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Value iteration and residual certificates | 443.57–752.5 s | Discounted value iteration and convergence; routed from lecture_9.pdf p. 21, Q-function value iteration (recording 722.8-752.5 s), and from p. 19 a cross-reference to the model-based versus model-free distinction (recording 318.6-443.6 s) in lecture-09 #q-values. | Contraction context from Bertsekas. | Full sup-norm proof, returned-vector residual certificate, greedy policy error bound. | |
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Policy iteration | 752.75–1387.5 s | Exact policy evaluation and greedy improvement. | Discounted dynamic programming. | Monotonicity proof, strict-improvement qualification and tie handling. | |
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Continuous-time optimal control | 1387.63–1600.08 s | Continuous dynamics and objective; official slides pp. 2-4 place HJB and HJI on the closed-loop branch beside discrete DP, call them nonlinear partial differential equations, and restate the discrete problem beside the continuous one in h, g, t_0, t_f notation. | Units and regularity assumptions; concavity of the Bellman Hamiltonian in the gradient, the velocity-channel Isaacs Hamiltonian as neither convex nor concave in general, and a slide-to-chapter notation correspondence. | ||
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Adversarial strategies | 1600.08–2753.46 s | Nonanticipative strategies and negative starting-time convention; official slides pp. 5-6 and 21 add nature or another agent as the second player, the joint system state with the homicidal-chauffeur example (recording 1784-2035 s), and the open-loop information pattern (player one declares its plan, player two responds; conservative, unrealistic, computationally cheap). | Bansal and colleagues' differential-game framework. | Finite response-map game showing why information order matters; proof that the nonanticipative value is at least the open-loop value. | |
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Hamilton–Jacobi–Isaacs equation | 2753.71–3500.3 s | Short-time dynamic programming and local Hamiltonian; official slides pp. 3, 8, 13 and 17 add the displayed dynamic-programming principle for the game, the step from signals to constant vectors on the short interval, and the optimal-control feedback u* in argmax_u min_d of the Hamiltonian bracket. | Chen and Tomlin's 2018 review, named on official slides p. 3, cited for the formal treatment of the dynamic-programming principle. | Taylor expansion with remainder, velocity-channel explicit Hamiltonian; bang-bang velocity-channel feedback and its selection problem at zero gradient. | |
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Hamilton–Jacobi–Bellman equation | 3500.3–3964.14 s | No-adversary specialization and interpretation; official slides p. 18 add the HJB as the Isaacs equation without disturbance and with cost minimization, its terminal condition, and the feedback u* in argmin_u of the Hamiltonian bracket. | Smooth verification proof and nonsmooth absolute-target example. | ||
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Continuous-time LQR | 4000.08–4233.6 s | Quadratic value ansatz and differential Riccati equation; official slides p. 20 add the Riccati matrix notation V(t) and the constant infinite-horizon limit (the limit is not located in the recording). | Exact scalar hyperbolic solution, quadrature and discrete-time limiting derivation; algebraic Riccati equation under stabilizability and detectability, scalar limit tanh(T-t) to 1 with its rate and infinite-horizon cost. | ||
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Reachability quantifiers | 4263.5–4620.14 s | Avoidance and goal-reaching definitions; official slides pp. 21-24 add the applications of differential games (pursuit-evasion, reachability, economics), the inputs and outputs of a reachability computation, the name backward reachable set, and its planning use. | Explicit distinction between trajectory and strategy quantifiers; exact velocity-channel avoidance and goal sets for U greater than D. | ||
| lecture-10-bellman-solvers-and-hamilton-jacobi-equations.html | Executable checks and eight worked exercises | Original teaching; no recording span | No separate recording span claimed. | Bellman residual, Riccati integration, independent quadrature, finite game checks and worked solutions; two-stage enumeration separating the open-loop and nonanticipative information patterns (official slides p. 6) and a detectability counterexample for the infinite-horizon limit (official slides p. 20). | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | Signed terminal functions | 245.9–844.66 s | Level-set target encoding and reaching versus avoiding; lecture_10.pdf p. 30 (routed) adds the avoid and reach set definitions, the reach value sup over Gamma of inf over u of h(x(0)) with its min_u max_d HJI equation, and the feedback laws argmax_u min_d (avoid) and argmin_u max_d (reach), delivered 61.9-153.9 s, 720.9-809.8 s and 1156.3-1221.7 s. | Closed-set qualification and sign scaling. | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | Reachable tubes | 844.66–1356.4 s | Minimum-over-time outcome and tube equation. | Endpoint-success counterexample and value ordering. | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | Relative aircraft dynamics | 1356.85–2348.75 s | Pursuit/evasion geometry, plots and transient incursion; lecture_10.pdf p. 33 (routed) adds identical speeds v, turn-rate bounds u_a, u_b in [-u_max, u_max], the evader (control) and pursuer (disturbance) roles, and the ToolboxLS Section 2.6.1 source, delivered 1356.8-1489.4 s. | Explicit rotating-frame derivation; demonstration settling time not generalized. | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | Exact braking margin | Original teaching; no recording span | No separate source span claimed. | Original robust cart stopping-distance proof and finite-horizon comparison. | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | Grid complexity | 2348.75–2443.09 s | Dimensional limitations and approximate alternatives; lecture_10.pdf p. 34 (routed) adds the (n+1)-dimensional grid of J(x,t) and the alternatives of sacrificing global optimality, giving up guarantees, neural-network PDE solvers, sampling-based methods and reinforcement learning, delivered 2319.4-2443.0 s. | Memory arithmetic and finite robust predecessor. | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | MPC feedback construction | 2650.94–3221.97 s | Receding horizon, state measurement, prediction notation and terminal design; official slides pp. 2-15 add the roadmap placement between open-loop and closed-loop methods (delivered 2444.5-2504.4 s), Borrelli-Bemporad-Morari reading (p. 3), control and prediction horizons N_m and N_p and the sampling-interval hold (pp. 4-6), polyhedral X and U and the regulation task (p. 7), the terminal cost, terminal set and horizon as design choices (p. 9), reasons predictions differ (p. 10), time-invariant notation and closed-loop map (pp. 11-13), quadratic and one-/infinity-norm costs with full column rank (p. 14), and the online algorithm with its infeasibility branch (p. 15). | Convex-program qualifications and warm-start distinction. | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | Feasibility and regulation | 3221.97–3827.73 s | Persistent feasibility and stability are distinct goals; official slides pp. 16-17 add the named persistent-feasibility and stability issues and the analysis strategy through conditions on the terminal cost and terminal set. | Exact scalar dead-end and feasible nonregulating controller. | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | Invariance and lemma statement | 3860.27–4406.61 s | Feasible set, predecessors, invariance and truncated-set lemma; proof postponed; official slides pp. 18-22 add persistent feasibility as the design goal, set-builder X_0 and X_1, the one-step controllable set name, maximal control invariant and maximal positive invariant sets, the singleton-origin example, and MPT3 computation. | Quantifiers and geometric interpretation. | ||
| lecture-11-reachable-tubes-and-receding-horizons.html | Code and eight worked solutions | Original teaching; no recording span | No separate source span claimed. | Braking, finite predecessor and exact one-step MPC checks; a second exact-arithmetic program and Exercises 7-8 check the horizon-two lemma comparison and the terminal-equality horizon (official slides pp. 9, 19, 21-22). | ||
| lecture-12-terminal-design-and-mpc-stability.html | Truncated-set lemma | 200.64–694.18 s | Full deferred proof using predecessor inclusion. | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-12-terminal-design-and-mpc-stability.html | Terminal invariance | 694.18–1357.17 s | Recursive feasibility theorem and terminal-set tradeoff. Official slides pp. 5-7 add the backward induction X_f subset X_{N-1} subset ... subset X_1 subset X_0 with control invariance of every truncated set (pp. 5-6) and the terminal set as an artificial sufficient-condition device whose enlargement cannot shrink X_0 and, with the same terminal cost, cannot raise V_N (p. 7). | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-12-terminal-design-and-mpc-stability.html | Lyapunov theory | 1390.82–1961.65 s | Positivity and decrease; teaching completion gives local bounds. Official slides pp. 8-9 add the feasibility-versus-convergence distinction (original stalled scalar example) and the Lyapunov theorem on a closed, bounded, positively invariant set, stated here with a continuous positive-definite decrease margin because the slide's strict-decrease form needs further hypotheses (Exercise 7). | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-12-terminal-design-and-mpc-stability.html | MPC decrease proof | 1961.65–3484.08 s | Shifted-candidate cost cancellation; teaching completion supplies sum and local quadratic bounds. Official slides pp. 10-15 add the A0-A3 assumption labels and notation correspondence, the Borrelli-Bemporad-Morari reference for continuity, and the replacement of terminal invariance by a horizon condition (p. 15 note, slide-only and not spoken in the recording), derived here through the sublevel bound eta. | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-12-terminal-design-and-mpc-stability.html | Terminal design | 3485.04–3996.43 s | Lyapunov equation, Riccati terminal controller and invariant set. Official slides pp. 16-18 add the maximal positively invariant set O_inf with finite determination and a stopping rule, the discrete-time algebraic Riccati equation and its equivalence to the closed-loop identity, and the remark that both recipes are suboptimal choices, quantified by P >= P_ric and an original scalar comparison. | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-12-terminal-design-and-mpc-stability.html | Cart ellipsoid | Original teaching; no recording span | Original analytical support-function terminal design and numerical fixture. | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-12-terminal-design-and-mpc-stability.html | Explicit MPC | 3996.43–4210.4 s | Piecewise-affine control and computational tradeoff; original scalar solution and active-set derivation. Official slides pp. 19-20 add the constrained-LQR problem statement, its condensation into H and G, the indexed piecewise-affine law with offsets and region inequalities, and the partition of X_0 rather than X. | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-12-terminal-design-and-mpc-stability.html | A design recipe fixes the horizon, terminal set, weights, and terminal cost | 4210.4–4267.57 s | Official slides p. 21: design steps for squared 2-norm cost (horizon and control invariant X_f, X_f as large as possible, Q and R chosen freely, P adjusted per the stability theorem), the MPT3 toolbox, and a terminal cost without terminal constraint; the 'no other technique' generality claim and the terminal-cost-only bullet are slide-only. | MPT3 homepage (fetched 2026-10-06) for the toolbox scope: parametric optimization, computational geometry, MPC, invariant sets, explicit MPC. | Coupling of the weights with the terminal ingredients, horizon effect through the nested feasible-set chain, output constraints as state constraints, and horizon certificates for the scalar example (N >= 7 at x = 1) and the cart (eta = 0.169821). | Design procedure with proved guarantees; field-level generality assessment marked as not a theorem; toolbox scope from its homepage. |
| lecture-12-terminal-design-and-mpc-stability.html | Reference tracking | 4268.37–4520.02 s | Input-increment formulation; teaching completion gives augmented dynamics and steady-bias counterexample. Official slides pp. 22-23 add the full input-increment problem with u_{-1} = u(t-1), the applied input u(t) = u(t-1) + Delta u_0^*, and separate output-error and increment weights; the chapter adds that X_f must be built for the tracking target. | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-12-terminal-design-and-mpc-stability.html | Code and worked solutions | Original teaching; no recording span | Original independent numerical checks and eight solved exercises; a second check block verifies the terminal-cost comparisons, the O_inf example, the condensed horizon-seven problem, and the cart constant eta; Exercises 7-8 are motivated by official slides pp. 9 and 16. | Additional derivations and fixtures explicitly identified in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Learning settings | 221.49–961.78 s | Feedback, robust and learning responses; direct/model-based learning; episode categories; official slides p. 2 roadmap placement (transcript 5.4-167.9 s), pp. 5-6 feedback loop with an unknown plant (167.9-218.6 s), p. 7 quadrotor, Harrier and grasping examples with the max-min robust formulation, pp. 8-9 named methods in each family. | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Least squares | 962.35–1605.7 s | Regression and scalar dynamics identification; official slides p. 12 offline identification pipeline with an engineer in the loop. | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Estimator statistics | 1605.7–1980.02 s | Mean and covariance derivation; chapter states fixed-design assumptions and dependent-data limit; official slides pp. 16-19 covariance sandwich step, scalar-model information matrix, and convergence criterion (made precise by a trace bound and Markov's inequality). | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Excitation | 1980.02–2171.46 s | Informative operation and convergence; chapter supplies rank-one counterexample and streaming update; official slides pp. 18-19 information matrix applied to the feedback experiment, p. 14 continuous-time least squares (transcript 1540.2-1605.7 s). | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Control versus estimation performance | 2171.46–2407.22 s | Task sensitivity, data budgets and model-class mismatch. | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Adaptive control proves stability of a coupled loop | 2417.8–2567.47 s | Official slides p. 22 (online adaptation of the policy, direct versus model-update routes, classical focus on stability of the coupled controller, adaptive component and dynamics rather than optimal adaptive control) and p. 25 (MRAC and MIAC as families of techniques). | None. | Explicit coupled-state description for the cart and the reason it is nonlinear, which motivates the Lyapunov route. | Definition and qualitative design distinction. |
| lecture-13-identification-and-adaptive-tracking.html | Lyapunov detour | 2567.47–3052.53 s | Mass-spring-damper energy; chapter corrects semidefinite derivative and completes invariance reasoning; official slides p. 23 global Lyapunov theorem with radial unboundedness (transcript 2567.5-2788.0 s) and p. 24 energy-derivative substitution (2934.6-3052.5 s). | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Reference model and known mass | 3064.27–3712.34 s | MRAC ingredients and double-integrator tracking; official slides pp. 27-29 notation correspondence (x_tilde = e, v = nu, k = k_r) and the perfect-tracking qualification. | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Unknown-mass adaptation | 3712.34–4292.72 s | Filtered error and adaptation cancellation; chapter verifies parameter-energy factor; official slides p. 29 closed-loop identity and sliding-variable name (transcript 3742.0-3980.5 s), p. 30 Lyapunov candidate with the one-half outside the bracket, consistent with the update law. | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Tracking convergence | 4292.72–4798.8 s | Tracking implication; chapter adds boundedness/Barbalat steps and separates parameter convergence; official slides p. 31 role of the reference signal in parameter convergence (delivered at 235.0-314.8 s of recording 7YdOx-ih0II; chapter derives the one-parameter excitation condition) and the Slotine-Li Chapter 8 reference (4741.0-4798.8 s). | Qualifications and original derivations distinguished in chapter. | ||
| lecture-13-identification-and-adaptive-tracking.html | Code and exercises | Original teaching; no recording span | Original nonidentifiability and full adaptive dynamics fixtures; seven worked solutions; official slide p. 31 motivates the four-reference check and Exercise 7. | Qualifications and original derivations distinguished in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Indirect adaptive control | 235.02–533.82 s | Deferred MRAC parameter-convergence qualification and MIAC certainty-equivalent/cautious comparison; routed from official lecture_13.pdf pp. 32-33 (delivered in this recording at 349.1-533.8 s): MIAC also called self-tuning, MRAC tracking-error versus MIAC data-fitting updates, MIAC modularity (any estimator with any model-based controller), MIAC correctness harder to guarantee without rich signals (original closed-loop identifiability example with a destabilizing perfect-fit estimate), MRAC tracking guarantees regardless of signal richness with parameter convergence still needing excitation, and Slotine and Li, Applied Nonlinear Control, chapter 8 (recalled at 203.8-233.5 s; cited in text without a link). | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Three routes from sensing to control differ in the information they require | 533.82–1095.79 s | Official slides p. 2 course roadmap (imitation learning, model-free and model-based RL as branches of adaptive optimal control), pp. 3-4 end-to-end robot and driving systems and RLHF as motivation, p. 5 quadruped sensing stack with Option 1 (optimal control problem), Option 2 (learning from demonstrations) and Option 3 (trial and error), p. 7 resources; recording 888-1006 s on imitation being bounded by demonstration quality and on staged training. | Principles of Robot Autonomy chapters 17-19 verified in the fetched PDF (version dated April 11, 2026); Sutton and Barto cited in text. | Information count of N_u - 1 failures versus one demonstration with N_u = 3^12; observation-versus-state qualification; product-video evidence boundary. | source-grounded structure with explicit completion and qualifications |
| lecture-14-demonstrations-rewards-and-learned-models.html | Supervised learning | 1095.79–1388.82 s | Regression and likelihood; chapter supplies Gaussian derivation and density/log qualifications; official slides pp. 8-10 mapping f: X -> Y with regression and classification output sets and image inputs, averaged NLL and theta_mle, quadratic regression with normal equations and a four-point worked example, softmax cross-entropy as a categorical likelihood, and translation of the slides' D, n and N. | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Behavior cloning | 1390.64–1880.22 s | Expert pairs, direct policy imitation and ALVINN example; official slides pp. 11-14 imitation learning as skill transfer with a teleoperated two-arm example, BC and IRL as the two families, the two-step BC recipe with its NLL display and MSE/cross-entropy as Gaussian/categorical likelihoods, and ALVINN's architecture, simulated training data and NAVLAB test conditions from the paper, with its hill-shaped output read as a multimodal action representation. | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Sequential distribution shift | 1880.22–2158.99 s | Compounding error, physical constraints, embodiment and covariate shift; original Lipschitz recurrence; official slide p. 15 covariate shift named, output structure (physical constraints and temporal dependencies), the source/deployment embodiment gap for a humanoid learned from human videos, and demonstration cost with correlated samples. | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Inverse reinforcement learning | 2160.85–2294.45 s | Objective inference and warehouse comparison; original reward ambiguity and shaping derivation; official slides pp. 12 and 16 reward R: X x U -> R related to r(x,u,x'), IRL as an imitation-learning family complementary to BC, and the driving motivation (recording 1669-1727 s). | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Interaction and MDP | 2294.45–3582.61 s | Policy, environment, reward, model, trajectory distribution and MDP elements; official slides pp. 18-24 agent-environment loop, motivation (discovery beyond demonstrations with Move 37 and AlphaGo's supervised initialization; complex objectives with parallel-simulation locomotion), the three characteristics (no supervision, delayed feedback and credit assignment, non-i.i.d. data and exploration-exploitation), the MDP tuple and notation translation, trajectory index range, discount weighting, and the chess and quadruped MDP table with state, transition and reward qualifications. | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Values | 3582.8–3949.2 s | State/action values and optimal action selection; original delayed-reward three-state model; official slide p. 25 explicit V and Q sums with gamma^{t'-t}, optimal value definitions, and their attainment by one stationary policy. | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Algorithm tradeoffs | 3949.2–4415.76 s | Model-based/free, policy/value, sample efficiency, on/off-policy and combined pipelines; official slides pp. 26-30 model-based split into given and learned models, policy-optimization and argmax-Q formulas, the p. 27 tradeoff, assumption and representation axes (cart-pole and shower, recording 4131-4218 s), the on-policy definition and the CS 285 sample-efficiency ordering, wall-clock arithmetic, convergence questions with the Bellman contraction and a two-state divergence example, model-objective mismatch, the IL/RL strengths-and-weaknesses table, Alpamayo-R1 stages, and the p. 4 RLHF three-step procedure with its pairwise reward loss and KL penalty. | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-14-demonstrations-rewards-and-learned-models.html | Code and exercises | Original teaching; no recording span | Original uncertainty-aware action, exact Markov value and importance weighting checks; seven solutions; a second executable block checks the slide-derived regression, quadruped-reward, divergence, model-mismatch, preference and ALVINN-count constructions; Exercise 7 extends the official slide p. 24 quadruped reward. | Original examples and qualifications explicitly identified in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Compounding loss | 301.62–562.77 s | Covariate shift and horizon dependence; chapter separates probability and accumulated loss. Official slides pp. 4-5 behavior-cloning objective and its two failure modes (recording 190.5-301.6 s, before this section's start), covariate-shift name, and Ross and Bagnell's quadratic bound read as an expected count of mistakes. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Corrective data and DAgger | 839.82–1576.34 s | Learner rollout, expert relabeling, aggregation, querying variants and costs. Official slides pp. 10-11 indexed loop, return rule compared with Ross, Gordon, and Bagnell's Algorithm 3.1 and its mixture weights, data-efficiency comparison, and confidence-based and human-gated query rules. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Original corrective controller | Original teaching; no recording span | Two-label affine controller fixture explicitly original. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Data augmentation | 1576.34–2116.77 s | Side-camera and trail-camera corrective data examples. Official slides pp. 13, 15-17 steering-network architecture with shape and parameter check, viewpoint transformation and two-second label adjustment, and the Giusti et al. citation with its camera-to-label mapping. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Observation history | 2116.77–2351.26 s | Partial observations and history-dependent policies. Official slides pp. 20-21 non-Markovian demonstrations, finite observation window, sequence models, and empirical window choice. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Expressive distributions | 2351.26–3153.36 s | Categorical, Gaussian, mixture and autoregressive action models. Official slides pp. 6, 20, 22-24 categorical and Gaussian heads, multiple-expert multimodality, exact single-Gaussian fit to two modes, choice of the component count, and conditional VAE. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Flow matching | 3154.19–3585.2 s | Diffusion overview and linear-path vector-field training/integration; original Gaussian marginal-flow check. Official slides pp. 24-25 diffusion forward process and noise prediction, training distribution and notation correspondence, and one-dimensional monotone transport to a multimodal target. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Action chunking | 3585.2–3926.6 s | Multi-action prediction, latency/coherence, Diffusion Policy and model examples. Official slides pp. 26-28 chunk loop indexing, mode-switch count, Diffusion Policy horizons and inference latency, and RT-1 architecture and action discretization. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Behavior cloning trades reward design for dependence on the demonstrations | 3926.6–4084.05 s | Official slides p. 30: behavior-cloning strengths (simplicity, no trial-and-error learning, no explicit reward, with the driving example) and weaknesses (dependence on demonstration quality, compounding errors under covariate shift, no exploration). | None; cross-references chapter 14 on held-out loss and the next chapter on reinforcement learning. | Population-optimum argument for why the likelihood objective reproduces rather than improves on the demonstrator. | Conceptual comparison of method properties; no empirical claim. |
| lecture-15-corrective-data-and-generative-policies.html | Outcome-conditioned supervision | 4084.05–4542.1 s | Return filtering, weighting, goals and reward conditioning. Official slides pp. 32-34 discounted return and filter set, per-transition reward weights with the exact categorical fit and sign requirement, conditioned objective, reward-to-go, hindsight-relabeling count, and stated advantages. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | IRL overview | 4542.1–4766.22 s | Alternating reward/policy updates and ambiguity; source provides pointers rather than full algorithms. Official slides p. 37 selection principles of apprenticeship learning, maximum margin planning, and maximum entropy IRL with in-text citations; the p. 36 warehouse example is covered in chapter 14. | Added derivations and qualifications labeled in chapter. | ||
| lecture-15-corrective-data-and-generative-policies.html | Code and exercises | Original teaching; no recording span | Original exact mistake process, aggregated controller and Gaussian transport checks; eight worked solutions. A second listing checks the arithmetic added from official slides pp. 6, 13, 23, 25-26, 28, and 33-34; exercises 7 and 8 extend official slides pp. 33 and 25. | Added derivations and qualifications labeled in chapter. | ||
| lecture-16-sampling-and-temporal-differences.html | A value prediction belongs to a policy and a state model | 174–490 s | MDP,values,Bellman equations,tabular V and Q; official slides p. 2 course placement, p. 4 tuple and control objective, pp. 5-6 Bellman optimality equations for V* and Q*, the Q expectation equation and uniqueness, and the notation correspondence for pp. 4-6, 16-17, 21, 24-29 and 37. | No separate external empirical claim. | Discount and reward-index conventions; persistent cart formalism. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Exact policy iteration reveals two different obstacles | 490–1247 s | Policy iteration,gridworld,model and storage limitations; official slides pp. 7, 8 and 14 prediction-control table with iterative policy evaluation, policy iteration and value iteration updates, separate sweep and policy indices, uniqueness, policy improvement and the GPI fixed point; pp. 9-13 gridworld sweeps 3 and 10, converged values, greedy policies after one and two sweeps and optimality from the third sweep, cited to Sutton and Barto Example 4.1. | No separate external empirical claim. | Grid update arithmetic,proper-policy qualification and contraction derivation. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Monte Carlo evaluation replaces an expectation with completed experience | 1247–1844 s | Completed episodes,first/every visit,model-free simulator distinction; official slides p. 18 counter-and-sum form with first-visit and every-visit convergence arguments and the finite-sample every-visit bias. | No separate external empirical claim. | Repeated-cart returns,correlation qualification and truncation bound. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Blackjack illustrates both interpretable values and state sufficiency | 1844–2213 s | State representation,policy evaluation and value surfaces; official slides pp. 19-20 reward rules, automatic hit below twelve, value as win minus loss probability, no recurrence of states within a game, and the 10,000 versus 500,000 episode comparison. | No separate external empirical claim. | Usable-ace definition and finite-deck Markov qualification. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | An incremental mean separates estimation from tracking | 2213–2356 s | Incremental means and variable step size. | No separate external empirical claim. | Mean derivation,exponential weights and policy-drift boundary. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Temporal differences move a prediction toward an informed successor prediction | 2356–2974 s | TD targets,error,bootstrapping and continuing tasks. | Primary TD mechanism and stochastic-approximation context integrated where relevant. | Terminal mask,numerical update,one-state discount correction. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Bias and variance refer to a specified target distribution | 2974–3273 s | Bias and variance comparison of MC and TD. | No separate external empirical claim. | Frozen-table bias and conditional-variance derivation with explicit assumptions. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Sampling width and bootstrap depth are independent choices | 3273–3609 s | Backup diagrams,sampling and bootstrap axes,multi-step continuum; official slides pp. 27-29 and 31 backup diagrams as the expectation versus samples of the one-step target, and the exhaustive-search corner with its leaf count. | No separate external empirical claim. | n-step return and error-delay tradeoff. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Policy improvement needs action values and deliberate exploration | 3609–4218 s | GPI,Q instead of V,two doors,epsilon-greedy,blackjack control; official slides pp. 35-37 two-door running means and state-action coverage, the epsilon-soft improvement guarantee and its limit, and the optimal blackjack rule of Sutton and Barto Figure 5.2. | Primary TD mechanism and stochastic-approximation context integrated where relevant. | Exploration probabilities,reachability and convergence boundaries. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Three processes organize the algorithm without solving representation | 4218–4424.98 s | Generate samples,estimate returns or model,improve policy,taxonomy; official slides pp. 40-41 taxonomy objective with trajectory distribution, value-based and model-based forms, parameterized fits, argmax and gradient improvement with named algorithms, and the CS 185/285 attribution. | No separate external empirical claim. | Experiment questions and representation boundary. | source-grounded with original formal teaching |
| lecture-16-sampling-and-temporal-differences.html | Executable checks isolate the target before testing a learner | Original teaching; no recording span | No additional recording span claimed; the second program checks the gridworld numbers of official slides pp. 9-13. | No separate external empirical claim. | Deterministic executable mechanism checks; exact gridworld sweep, converged-value and greedy-policy checks. | original teaching construction |
| lecture-16-sampling-and-temporal-differences.html | Eight exercises with fully worked solutions | Original teaching; no recording span | No additional recording span claimed; exercises 7 and 8 extend official slides p. 18 and pp. 35-36. | No separate external empirical claim. | Eight original exercises and full solutions. | original teaching construction |
| lecture-16-sampling-and-temporal-differences.html | What the evidence establishes | Original teaching; no recording span | No additional recording span claimed. | No separate external empirical claim. | Evidence and applicability boundaries. | original teaching construction |
| lecture-17-value-based-rl.html | Temporal-difference control learns before an episode ends | 496.9–1006.8 s | TD control, SARSA and windy gridworld; official slides p. 8 three TD advantages and TD inside GPI with epsilon-greedy improvement, p. 9 quintuple and (s,a) notation, p. 10 SARSA algorithm order; intro bridge cites the recap and placement of pp. 2-7, 28 and 32 (recording 5.3-496.9 s), developed in the lecture 16 chapter. | Convergence conditions stated. | Terminal-masked formalism. | source-grounded with formal completion |
| lecture-17-value-based-rl.html | A windy gridworld defeats the obvious route and rewards online updates | 1006.8–1583.6 s | Official slides pp. 11-12: windy gridworld layout, wind strengths, reward, epsilon=0.1, alpha=0.5, gamma=1, completed-episodes curve, and whether Monte Carlo applies; this recording span also delivers the on-policy and off-policy definitions of official slides p. 13, written in #offpolicy. | Fixed-exploration and constant-step qualifications of the learning curve. | Breadth-first search verifying the 15-move minimum and the failed straight route; looping-policy termination argument. | source-grounded with tested code |
| lecture-17-value-based-rl.html | Off-policy learning separates behavior from the target | 1583.6–1857.8 s | Behavior/target policies and reuse motivations; official slides p. 13 on-policy and off-policy definitions with SARSA classified as on policy (recording 1444.6-1583.6 s) and p. 15 alternative-successor target with the no-importance-ratio argument. | Coverage boundary. | Three-policy experiment ledger. | source-grounded with qualification |
| lecture-17-value-based-rl.html | Q-learning bootstraps toward the greedy successor value | 1857.8–2264.0 s | Q-learning target and algorithm; official slides p. 16 greedy substitution into the alternative-successor target, p. 17 algorithm order and convergence theorem with its hypotheses and the gamma=1 termination requirement. | Tabular theorem assumptions. | Tie-handling note. | source-grounded with formal completion |
| lecture-17-value-based-rl.html | The cliff exposes the difference between learned and executed risk | 2264.0–2774.4 s | SARSA and Q-learning cliff comparison; official slides p. 18 grid dimensions, reset-to-start fall, epsilon=0.1, alpha=0.5, gamma=1, learning-curve levels, and the limits of SARSA (optimal epsilon-greedy policy) and Q-learning (Q*). | Training/evaluation distinction. | Executable expected-target calculation; epsilon-perturbed Bellman equation and executable exact evaluation of three routes and the best epsilon-greedy controller under both fall conventions. | source-grounded with tested code |
| lecture-17-value-based-rl.html | Function approximation trades storage for coupled generalization | 2774.4–3514.1 s | Value approximation and stochastic gradient descent; official slides p. 20 backgammon and Go state counts and continuous states (recording 2774.4-2858.6 s), p. 21 memory and visitation limits of tables, p. 22 three approximator interfaces and model families, p. 23 mean-squared value error and its gradient step, p. 24 Monte Carlo and TD targets, p. 25 tabular versus regression-pair use of the targets. | Discontinuity qualification. | Discrete/continuous interface. | source-grounded with derivation |
| lecture-17-value-based-rl.html | Fitted Q iteration turns Bellman updates into repeated regression | 3514.1–3968.5 s | TD regression and fitted Q-learning skeleton; official slides p. 26 control with approximate policy evaluation, p. 27 per-transition fitted Q update, pp. 28-29 three-process skeleton of fitted Q-learning. | Semi-gradient boundary. | Frozen-target formulation. | source-grounded with formal completion |
| lecture-17-value-based-rl.html | DQN stabilizes correlated data and moving targets | 3968.5–4654.35 s | Atari DQN, replay and fixed targets; official slides p. 30 image input, convolutional and fully connected layers with one output per joystick action, p. 31 correlated samples and moving target as the two failures, and the replay minibatch loss with fixed target parameters. | Primary DQN evidence integrated. | Polyak convention and buffer audit. | source-grounded with primary research |
| lecture-17-value-based-rl.html | Instability, workflow, exercises and limits | Original teaching; no recording span | No additional recording span claimed; Exercises 7 and 8 rest on official slides pp. 11-12 and p. 25. | Deadly-triad context. | Diagnostics and eight worked exercises. | original teaching construction |
| lecture-18-policy-optimization.html | An explicit stochastic policy induces a trajectory distribution | 124.18–589.66 s | MDP trajectory distribution and parametric policy objective.; official slides pp. 2-3 roadmap placement and RL taxonomy (model-based with given or fitted model, value-based implicit argmax policy, policy optimization), pp. 3, 5-6, 8 notation correspondence (reversed r/R, gamma = 1 convention, t = 1 indexing and the product-index erratum). | Policy-gradient assumptions stated. | Persistent Bernoulli cart controller. | source-grounded with formal model |
| lecture-18-policy-optimization.html | The log-derivative removes unknown dynamics from the gradient | 589.66–1100.9 s | Likelihood-ratio derivation and sampled policy gradient.; official slides p. 8 likelihood-ratio identity as the step that turns the gradient into an expectation. | REINFORCE primary result integrated. | Bernoulli score derivation. | source-grounded with derivation |
| lecture-18-policy-optimization.html | REINFORCE alternates fresh rollouts and ascent steps | 1100.9–1652.3 s | REINFORCE algorithm and MLE comparison.; official slides p. 11 three-stage sample, estimate and improve loop, pp. 12-13 displayed maximum-likelihood gradient, return-weighted interpretation, categorical and Gaussian scores. | On-policy boundary. | Reward-translation interpretation. | source-grounded with explicit qualification |
| lecture-18-policy-optimization.html | Reward-to-go removes rewards an action could not have caused | 1652.3–2120.0 s | High variance and causality reduction.; official slides p. 15 variance consequence for learning, pp. 15 and 17 reward-shift effect quantified on the bandit, p. 16 conditioning proof of the causality reduction. | Discount convention clarified. | Cart credit assignment. | source-grounded with formal completion |
| lecture-18-policy-optimization.html | An action-independent baseline preserves expectation and can cut variance | 2120.0–2700.0 s | Baseline intuition, proof and training effect.; official slide p. 19 training comparison with and without an average-return baseline. | Finite-batch and weighted-optimum qualification. | Executable exact variance example. | source-grounded with tested code |
| lecture-18-policy-optimization.html | Policy gradients trade direct optimization for on-policy sampling | 2700.0–3077.2 s | Policy-gradient properties and on-policy limitation. | Importance-weight boundary. | Efficiency accounting. | source-grounded synthesis |
| lecture-18-policy-optimization.html | A critic replaces a noisy realized future with a conditional expectation | 3077.2–4012.9 s | Actor-critic, state baseline and advantage. | Bias-variance boundary. | Cart relative-action example. | source-grounded with derivation |
| lecture-18-policy-optimization.html | Advantage actor-critic can learn one value network | 4012.9–4430.99 s | A2C, TD advantage and value fitting.; official slides p. 24 fitted Q-learning, REINFORCE and actor-critic loop comparison with squared regression loss, p. 25 regression-size argument and one-step Bellman derivation, pp. 26-27 Monte Carlo and temporal-difference regression labels with successor-state bootstrap. | Implementation diagnostics. | Explicit terminal-masked estimator. | source-grounded with formal completion |
| lecture-18-policy-optimization.html | AlphaGo combined policy learning, value learning, self-play, and search | 4430.99–4811.86 s | AlphaGo policy, value, self-play and MCTS. | Primary Nature paper integrated. | AlphaGo versus AlphaGo Zero distinction. | source-grounded with primary research |
| lecture-18-policy-optimization.html | Large-scale applications wrap the same estimators in new architectures and data pipelines | Original teaching; no recording span | Official slides pp. 34-36; not located in the recording (recording a1g9U_5zO54 ends with AlphaGo at 4431-4811 s, and recording ZXMThMHFD_w moves from PPO to the model-free summary at 1066 s). The slides name Mnih et al. 2016 asynchronous advantage actor-critic for maze navigation with CNN and RNN policy and value networks, a reward-model and PPO pipeline for language-model alignment, and PPO trained entirely in simulation for robot manipulation. | Mnih et al. 2016, Ouyang et al. 2022 and OpenAI et al. 2019 verified for the stated claims. | State-sufficiency reading of recurrent maze policies, token-level reading of the alignment episode, reward-model over-optimization and simulation-gap boundaries. | Official slides pp. 34-36; not located in the recording; primary research |
| lecture-18-policy-optimization.html | Exercises and limitations | Original teaching; no recording span | No additional recording span claimed. Official slides p. 28 architecture design (two networks versus shared body, synchronous versus asynchronous workers) added to the implementation section; not located in the recording. Exercises 7-8 extend official slides pp. 15 and 17 (reward shift) and p. 27 (TD label). | Primary results synthesized. | Eight original fully worked exercises. | original teaching construction |
| lecture-19-model-based-rl.html | A surrogate objective permits more than one policy update | 25.3–450.2 s | Policy-gradient surrogate and importance ratio. Plus official slides pp. 4-5: reduction of RL to optimization, L^PG loss and its gradient identity with L^IS at theta_old, sampling-distribution boundary. | TRPO and PPO primary papers integrated. | Exact gradient-equivalence boundary. | source-grounded structure with explicit qualification |
| lecture-19-model-based-rl.html | TRPO constrains policy distributions rather than raw parameters | 450.2–702.8 s | KL trust region and implementation limitations. Plus official slide p. 6: architectures with multiple outputs, deep CNN/RNN performance, conjugate-gradient complexity. | Primary TRPO result integrated. | Sampled-average guarantee boundary. | source-grounded structure with explicit qualification |
| lecture-19-model-based-rl.html | PPO clips incentives for destructive likelihood-ratio changes | 702.8–1066.0 s | Penalty and clipped PPO objectives. Plus official slide p. 7: PPO v1 KL-penalty Lagrangian with corrected sign, dual descent on the multiplier, adaptive-penalty rule. | Primary PPO result integrated. | Sign-dependent clipping derivation. | source-grounded structure with explicit qualification |
| lecture-19-model-based-rl.html | Model-free methods share one loop but differ in their targets | 1066.0–1500.0 s | Official slides pp. 8-10: summary of model-free RL (DP, Monte Carlo and TD backups, function approximation, GPI, SARSA and Q-learning, DQN, policy gradient and maximum-likelihood gradient, critic) and the three-stage loop of RL algorithms; recording 1066.9-1500 s. | Backups stated for V and Q; semi-gradient reading of the TD update. | Variance ratio 1/(1-gamma^2) for Monte Carlo against one-step TD (about 50.3 at gamma 0.99). | source-grounded structure with original derivation |
| lecture-19-model-based-rl.html | Model-based RL learns dynamics and then plans through them | 1500.0–2098.5 s | Basic recipe, distribution shift, MPC data aggregation. Plus official slides pp. 11-12 and 14-17: trade-off axes, sample-efficiency ordering, three-step recipe with base policy pi_0, system-identification regime, covariate-shift planar example, MPC plus adaptive-control loop. | System-identification interpretation. | Persistent uncertain cart regression. | source-grounded structure with original worked model |
| lecture-19-model-based-rl.html | Aleatoric and epistemic uncertainty answer different questions | 2098.5–3001.6 s | Predictive distributions, uncertainty types, Bayesian posterior. Plus official slides pp. 19-22: capacity tension and planner exploitation, output entropy versus epistemic uncertainty, posterior contraction with data, posterior predictive. | Risk interpretation derived from total expectation. | Cliff and quadratic-loss reasoning. | source-grounded structure with explicit completion |
| lecture-19-model-based-rl.html | Gaussian processes and ensembles trade analytic structure for scale | 3001.6–3945.0 s | GP posterior intuition and bootstrap ensembles. Plus official slides pp. 23-24: GP distribution over functions, strengths and weaknesses, Dirac-mixture view of ensembles, Deep Ensembles versus VI and MultiSWAG, role of SGD and random initialization. | Calibration limitation made explicit. | Three-gain ensemble calculation. | source-grounded structure with numerical extension |
| lecture-19-model-based-rl.html | PETS propagates model uncertainty through sampled trajectories | 3945.0–4462.9 s | PETS, trajectory sampling, CEM and sample efficiency. Plus official slides pp. 25-27: Monte Carlo score for planning under model uncertainty, PETS experiments and model-accuracy plots. | Primary PETS paper integrated. | Executable exact-grid analogue. | source-grounded structure with tested code |
| lecture-19-model-based-rl.html | The course methods form a stack rather than a tournament | 4463.0–4898.0 s | Layered autonomy stack and method selection. Plus official slides pp. 2, 28-29: course roadmap and layered autonomy stack with goal G and safety concepts. | Safety-envelope interpretation. | End-to-end cart architecture. | source-grounded synthesis |
| lecture-19-model-based-rl.html | Exercises with worked solutions and limitations | Original teaching; no recording span | No additional recording span claimed. | Primary-source limitations synthesized. | Six original fully worked exercises. | original teaching construction |