Module 0 of 14 · one week
Four words get fixed here: function, operator, discretization, learned operator. Three goals that the podcasts run together get pulled apart, because each one is refuted by a different experiment. Then three facts from functional analysis, each proved, because the rest of the course leans on all three.
The first five minutes of the seed video (Latent Space, 4 September 2026, KS_IpnX7n9I) make three promises in quick succession. Neural operators will replace simulators. Physics laws give a network dense feedback where labels are scarce. One model will learn many physical systems at once. Spoken in a row they sound like one claim about one technology, and that is how they land on a listener.
They are three claims about three different things, and each one is refuted by a different experiment. A model can be an excellent stand-in for a solver and be worthless as a shared representation. A model can drive a residual to zero on one problem instance and tell you nothing about the next instance. Evidence for one does not transfer to the others, so the three have to be graded separately or not at all.
Neither episode separates them. That is not a complaint about the speakers; the earlier episode (79mIutht1f4, 26 August 2026) is the better explanation of the two, and it handles the first promise carefully between 00:16:12 and 00:19:07 and gives an honest account of why PINNs disappoint between 00:11:58 and 00:14:04. The separation just is not something a conversation is built to do. A course is. So the split comes first, before any mathematics, because every later page has to say which of the three it is testing.
Surrogate. A model that reproduces a known simulator faster than the simulator runs. Its test is error against that simulator on held-out inputs, together with a wall-clock number.
PINN. A model that fits one solution by penalising the residual of the equation. Its test is the residual and the error on that one instance.
Foundation model. A model that reuses one representation across several physical systems. Its test is a transfer measurement: does pretraining on systems A and B reduce the data needed for system C, compared against an equal-size model trained on C alone?
Success at one proves nothing about the other two. Module 10 comes back to this with the published transfer numbers, and Module 12 grades the company's claims against it.
A function maps a point to a value, \(x \mapsto u(x)\). An operator maps a whole function to a whole function, \(f \mapsto u\). "Initial condition in, solution at time one out" is an operator; so is "heat source map in, temperature field out", which is the operator Lab 02 learns. A function on an interval carries infinitely many degrees of freedom. A computer stores finitely many numbers. A discretization is that finite record: a set of points \(D_L = \{x_1, \dots, x_L\}\) and the values \(a(x_\ell)\) there. A learned operator is a parametric map \(G_\theta\) that is defined on functions but that we only ever evaluate through discretizations.
Those four sentences are cheap. The gap between the third and the fourth is the whole subject, so it gets a page now, with numbers, before Module 5 meets it as an abstraction.
Take the operator this course returns to most often: send \(f\) to the \(u\) that solves
\[-u''(x) = f(x) \ \text{ on } (0,1), \qquad u(0) = u(1) = 0.\]
Feed it \(f(x) = \sin(\pi x)\). Because \(\frac{d^2}{dx^2}\sin(\pi x) = -\pi^2 \sin(\pi x)\), the solution is \(u(x) = \sin(\pi x)/\pi^2\): differentiate twice and negate to get \(\pi^2 \sin(\pi x)/\pi^2 = \sin(\pi x) = f\), and both endpoint values vanish. Its peak sits at \(x = 1/2\) and equals \(1/\pi^2 = 0.10132\).
Now discretize. Record \(f\) at the five points \(x_\ell = \ell/4\) for \(\ell = 0,1,2,3,4\). The record is
\[\big(0,\ \sin(\pi/4),\ \sin(\pi/2),\ \sin(3\pi/4),\ \sin(\pi)\big) = (0,\ 0.70711,\ 1,\ 0.70711,\ 0).\]
Five numbers, and they are all the model will ever see. Now for what they do not determine. Let \(g(x) = \sin(\pi x) + \sin(4\pi x)\). At every one of the five sample points \(\sin(4\pi x_\ell) = \sin(\pi \ell) = 0\), so \(g\) produces the identical record. The two inputs are indistinguishable to the discretization, yet their solutions are not equal. Solving with \(g\) adds the term \(\sin(4\pi x)/(16\pi^2)\), because differentiating \(\sin(4\pi x)\) twice brings down \(-16\pi^2\). At \(x = 1/8\), a point the record does not sample, that extra term equals
\[\frac{\sin(\pi/2)}{16\pi^2} = \frac{1}{157.91} = 0.00633,\]
which is \(1/16\), or 6.25 percent, of the peak of the true solution. So a five-point record of the input leaves the output undetermined by six percent of its own size, and no amount of training fixes that, because the information was destroyed before the model saw anything.
Three things follow, and they organise Modules 1 through 5. First, representation: does a good approximating model exist at all? Second, learning: what can a finite sample of input-output pairs determine? Third, discretization: how does the finite model on a grid relate to the continuum object it is supposed to approximate? Optimization sits on top of all three as a fourth problem. The JMLR paper lists exactly these four sources of error on pages 50 to 51 and proves results for only two of them, which is the ledger Module 5 makes precise (research_notes/pdf-fno-jmlr.md:705-711).
The example above is the third problem in miniature. Notice which way the failure runs: the culprit is the mode \(\sin(4\pi x)\), whose zeros land exactly on the sample points. Module 1 gives that phenomenon its name, aliasing, and Module 2 shows an FNO producing it by accident at grids it never trained on.
The company episode sells resolution invariance by analogy with a language model's context length. The underlying claim, from the earlier episode between 00:16:12 and 00:17:51, is narrower and defensible: standard networks have fixed input and output sizes, neural operators model inputs and outputs as continuous functions, and so they accept and produce data at any resolution.
What that guarantees and what it does not is the entire content of Module 1, so here is only the summary. A fixed set of weights can be evaluated on any grid, and as the grid refines the output converges to one continuum operator. It does not follow that the model is accurate on a grid it never saw. Accuracy at a new resolution needs assumptions about the content the training grid could not carry, and Lab 01 measures what happens when those assumptions are not checked: the error rises 56 percent at the first unseen grid for one model and 18 percent for another.
A physics-informed network solves one problem instance by optimization, so its difficulty is an optimization difficulty, not a shortage of capacity. Krishnapriyan et al. made this concrete on the convection equation \(u_t + \beta u_x = 0\): a standard PINN reaches roughly 100 percent relative error once \(\beta\) passes about 10, with a condition number growing like \((\beta N)^2\) (research_notes/web-papers.md:600-616). The same network, handed the right answer as an initialisation, holds it. The capacity was there; the descent could not find it. Anandkumar makes the same point in the earlier episode between 00:11:58 and 00:14:04. Module 4 gives the numbers and the two repairs the literature actually uses.
Three facts from functional analysis carry the rest of the course. Each is proved here in the smallest setting where the mechanism is visible, and each ends with the place in this repository where it decides something. If you already know all three, read only the consequences; if you do not, these are the three to read in full.
The precise statement is this: there is no rule that takes an element of \(L^2(0,1)\) and a point \(x_0\) and returns "the value there". Every dataset in this course is a table of point values, so the fact needs its argument, not an assertion. This subsection does four things. It says what the space is and why its elements are equivalence classes and not functions. It proves that evaluation is not defined on a class, and that it is not even continuous on the functions where it is defined. It proves the rescue: enough smoothness gives a class exactly one continuous representative, and in \(d\) dimensions the threshold is \(s > d/2\) derivatives. It ends with the three routes around the problem, and where this course uses each one.
Start with the raw material: measurable \(u : (0,1) \to \mathbb R\) with \(\int_0^1 |u|^2\,dx < \infty\). The candidate norm is \[\|u\|_2 = \Big(\int_0^1 |u|^2\,dx\Big)^{1/2}.\] It is homogeneous and it obeys the triangle inequality. It fails one axiom. The function that is 1 at \(x = 1/2\) and 0 elsewhere has \(\|u\|_2 = 0\) and is not the zero function. So \(\|\cdot\|_2\) is a seminorm, and \(d(u,v) = \|u - v\|_2\) is a pseudometric: two different objects sit at distance zero from each other.
That is fatal rather than untidy. In a pseudometric space limits are not unique, so "the" limit of a Cauchy sequence has no meaning, and neither does "the" orthogonal projection that Fact 3 needs. The repair is forced: collapse everything at distance zero into one object. Declare \(u \sim v\) when \(\|u - v\|_2 = 0\), and let \(L^2(0,1)\) be the set of classes. Only then is \(\|\cdot\|_2\) a norm; only then is the space complete (Riesz-Fischer) and a Hilbert space with an orthonormal basis. Facts 2 and 3 spend that structure. Fact 1 is its bill.
Null sets. \(N \subset \mathbb R\) is null if for every \(\varepsilon > 0\) there are countably many intervals \(I_1, I_2, \dots\) with \(N \subset \bigcup_n I_n\) and \(\sum_n |I_n| < \varepsilon\). One point \(x_0\) is null: cover it by \((x_0 - \varepsilon/4,\, x_0 + \varepsilon/4)\), of length \(\varepsilon/2\). The rationals in \((0,1)\) are null: enumerate them \(q_1, q_2, \dots\) and cover \(q_n\) by an interval of length \(\varepsilon 2^{-n-1}\), for a total of \(\varepsilon/2\). A null set can therefore be dense. It need not be countable either; the Cantor set is uncountable and null.
Claim. \(\|u\|_2 = 0\) if and only if \(u = 0\) outside a null set. That condition is written "\(u = 0\) almost everywhere", abbreviated a.e.
Proof. If \(u = 0\) off a null set, then \(|u|^2 = 0\) off that set, and the Lebesgue integral of a function that vanishes off a null set is zero. Conversely, let \(A_n = \{x : |u(x)| > 1/n\}\). On \(A_n\) the integrand exceeds \(1/n^2\), so \(0 = \int_0^1 |u|^2 \ge n^{-2}\,|A_n|\), which forces \(|A_n| = 0\) for every \(n\). The set where \(u \ne 0\) is \(\bigcup_n A_n\), a countable union of null sets, which is null. \(\square\)
Consequence. \(u \sim v\) means exactly \(u = v\) a.e. In \(L^2(0,1)\) the zero element and the indicator function of the rationals are the same element, although they differ at every rational, a dense set of points.
Claim 1 (evaluation carries no information). Fix any \(x_0 \in (0,1)\) and any number \(c\). Every element of \(L^2(0,1)\) has a representative whose value at \(x_0\) is \(c\).
Proof. Take any representative \(u\) and set \(\tilde u(x) = u(x)\) for \(x \ne x_0\), \(\tilde u(x_0) = c\). Then \(\tilde u - u\) vanishes off \(\{x_0\}\), a null set, so \(\tilde u \sim u\). \(\square\)
So \(u(x_0)\) is not merely ambiguous for awkward functions. Any value at all can be arranged, for every element, at every point. A model that reads an \(L^2\) input by sampling reads a property of the representative that the modeller happened to write down, not a property of the element.
Claim 2 (evaluation is unbounded). Restrict to continuous functions, where \(u(x_0)\) is unambiguous. There is still no constant \(C\) with \(|u(x_0)| \le C\|u\|_2\) for all of them.
Proof. Take the tent \(t_n(x) = \max(0,\, 1 - n|x - x_0|)\), which is continuous, peaks at \(t_n(x_0) = 1\), and is supported in an interval of width \(2/n\). Then \[\|t_n\|_2^2 = \int_{-1/n}^{1/n} (1 - n|s|)^2\,ds = 2\int_0^{1/n}(1 - ns)^2\,ds = \frac{2}{3n},\] so \(\|t_n\|_2 = (2/3n)^{1/2} \to 0\) while \(t_n(x_0) = 1\) for every \(n\). No \(C\) survives. \(\square\)
Consequence. \(t_n \to 0\) in \(L^2\). A learner that reads its input by point sampling can be handed a sequence of inputs that converges to zero in the very norm its error is measured in, while one of the numbers it reads never moves. Evaluation is not only undefined on classes; it is discontinuous on the functions where it is defined. Module 1 measures the quantitative form of this on a grid, where it is called aliasing.
One honest caveat, because the fact as stated is slightly too strong. The Lebesgue differentiation theorem does give every class a canonical value at almost every point: the limit of its averages over shrinking intervals around that point. "Almost every" is the catch. The exceptional null set depends on which element you took, so no fixed grid of sample points is safe for every input, and a fixed grid is exactly what a trained model has.
Claim 3 (a continuous representative is unique). If two continuous functions on \([0,1]\) agree a.e., they agree everywhere. So a class that has a continuous representative has exactly one, and evaluating that representative is a well defined operation on the class.
Proof. Let \(w\) be continuous and zero off a null set \(N\). Suppose \(w(x_0) \ne 0\). By continuity there is an interval \(I\) around \(x_0\) on which \(|w| > |w(x_0)|/2 > 0\), so \(I \subset N\). An interval of positive length is not null, and no subset of a null set has positive length. Contradiction. \(\square\)
Claim 4 (which spaces have that property). Work on the torus \(\mathbb T^d\), with Module 1's norm \(\|u\|_{H^s}^2 = \sum_k (1 + |k|^2)^s |\hat u(k)|^2\). If \(s > d/2\), then every \(u \in H^s(\mathbb T^d)\) has a continuous representative, and \[\sup_x |u(x)| \le C_s\,\|u\|_{H^s}, \qquad C_s = \Big(\sum_k (1 + |k|^2)^{-s}\Big)^{1/2} < \infty .\]
Proof. Split each Fourier coefficient and apply Cauchy-Schwarz: \[\sum_k |\hat u(k)| = \sum_k (1+|k|^2)^{s/2}|\hat u(k)| \cdot (1+|k|^2)^{-s/2} \le \|u\|_{H^s}\Big(\sum_k (1+|k|^2)^{-s}\Big)^{1/2}.\] The last sum converges exactly when \(2s > d\): compare it with \(\int_{\mathbb R^d}(1+|\xi|^2)^{-s}\,d\xi\), whose integrand decays like \(|\xi|^{-2s}\) while the shell volume grows like \(|\xi|^{d-1}\). So \(\sum_k |\hat u(k)| < \infty\), the Fourier series converges uniformly by the Weierstrass M-test, and a uniform limit of continuous functions is continuous. That sum agrees with \(u\) in \(L^2\), hence a.e., so it is the continuous representative, and the displayed inequality bounds its supremum. \(\square\)
Consequence. In one dimension the threshold is \(s > 1/2\). Evaluation is unbounded on \(H^0 = L^2\) and bounded on every \(H^s\) with \(s > 1/2\). The price of a point value is half a derivative. In two dimensions, as in Lab 02, the price is one derivative.
research_notes/pdf-fno-jmlr.md:467-468). The proof samples on a grid and passes to Riemann sums, which is legitimate only when point values exist. Discretization invariance is bought with that assumption.research_notes/pdf-fno-jmlr.md:477-478). The mechanism is Lemma 28 (p.81): every bounded linear functional on \(W^{m,p}(D)\), which is \(L^p\) when \(m = 0\), is approximated uniformly on a bounded set by \(u \mapsto \int_D \kappa u\,dx\) with a smooth compactly supported \(\kappa\). Lemma 30 (p.86) proves the same on \(C(\bar D)\), and it goes through Lemma 29 (p.85), which first writes the functional with point evaluations. It may do that because on \(C(\bar D)\) point evaluation is bounded. Put the two lemmas side by side and you are reading Fact 1 in the paper's own hands.../labs/02_thermal/run_thermal.py:92-95). A finite volume solver hands you route 3 whether you asked for it or not, because its unknowns are cell averages.Where this decides something in this course. Lab 01's Burgers initial conditions are drawn from a law whose coefficients satisfy \(\mathbb E|\hat u_0(k)|^2 = 625/(k^2+25)^2\), which Fact 2 below reads off the covariance operator, so \(|\hat u_k|\) falls like \(k^{-2}\) once \(k\) is well above 5. Then \(\sum_k (1+k^2)^s |\hat u_k|^2\) behaves like \(\sum_k k^{2s-4}\), which is finite for \(s < 3/2\). Since \(3/2 > 1/2\), those inputs do have continuous representatives, so reading them off a grid is legitimate and route 1 applies. What remains is not Fact 1 but aliasing, and Module 1 puts a number on it: 0.15 percent of the initial condition's energy sits above \(k = 32\), the highest mode a 64 point grid can carry.
The FNO paper's Darcy benchmark is the other case. Its coefficient \(a\) takes two values, 12 and 3, on a random partition, and the reference solution is computed on a 421 by 421 grid (research_notes/pdf-fno-jmlr.md:108-112). A two valued field has no continuous representative, so its natural space does not embed in \(C(\bar D)\), and Theorem 8 as stated does not cover that input space. The theorems that do cover it are the universal approximation family, whose Assumption 9 admits \(L^p\), and they read the input by route 2. Fact 1 is what separates the paper's two hypothesis sets.
The experiment then does what the theorem does not license: it point samples. The paper builds every coarser Darcy grid by subsampling the 421 by 421 data (p.32), and the FNO error stays flat, 0.0108 at 85 points per side and 0.0098 at 421 (research_notes/pdf-fno-jmlr.md:121). The reason is Fact 1 used in the other direction. A two valued field does have a point value at every location off the interfaces between the phases, and those interfaces are curves, a set of zero area. A sample is therefore exact unless its cell straddles an edge, and the area that is misread shrinks as the grid refines. The failure Fact 1 warns about is real but is confined to a set whose measure goes to zero. A field with no such structure, one that oscillates at every scale, gives no such reprieve.
Module 4 turns the same fact on DeepONet, whose branch network reads the input at fixed sensor locations. That is point evaluation, so the sensor reading means something only under the smoothness assumption of route 1.
Fact 1 was about one input. Fact 2 is about the set of inputs a theorem is allowed to quantify over, and it explains a hypothesis that otherwise looks like decoration. Every approximation theorem in this course reads "for every compact set \(K\) of inputs", never "for every input of norm at most one". The restriction is not caution. Without it the theorems are false, and the proof technique they all share has nothing to hold on to.
A set \(S\) in a metric space is compact if every sequence in \(S\) has a subsequence converging to a limit that lies in \(S\). An \(\varepsilon\)-net for \(S\) is a finite list of points \(u_1, \dots, u_n\) such that every \(u \in S\) lies within \(\varepsilon\) of some \(u_i\). The two notions are linked: a compact set has a finite \(\varepsilon\)-net for every \(\varepsilon > 0\), and that finite net is the object every proof below actually uses. A sequence \((e_j)\) is orthonormal if \(\langle e_j, e_j\rangle = 1\) and \(\langle e_j, e_k\rangle = 0\) for \(j \ne k\); on the torus the Fourier modes \(e_k(x) = e^{2\pi i k x}\) are the standard example.
Claim 5. Let \(H\) be an infinite dimensional Hilbert space and \(B = \{u \in H : \|u\| \le 1\}\) its closed unit ball. Then \(B\) is closed and bounded, and \(B\) is not compact. Moreover \(B\) has no finite \(\varepsilon\)-net for any \(\varepsilon < \sqrt2/2\).
Proof. Because \(H\) is infinite dimensional it contains a countable linearly independent set, and Gram-Schmidt turns that into an orthonormal sequence \((e_j)\), every term of which lies in \(B\) since \(\|e_j\| = 1\). Expand the distance between two of them: \[\|e_j - e_k\|^2 = \|e_j\|^2 + \|e_k\|^2 - 2\langle e_j, e_k\rangle = 1 + 1 - 0 = 2, \qquad j \ne k.\] Every pair sits at distance \(\sqrt2\). Any subsequence therefore has all its pairwise distances equal to \(\sqrt 2\), so it is not Cauchy and cannot converge. That kills compactness.
For the net, suppose \(u_1, \dots, u_n\) were an \(\varepsilon\)-net with \(\varepsilon < \sqrt2/2\). The sequence \((e_j)\) is infinite and the net is finite, so by the pigeonhole principle two distinct terms \(e_j, e_k\) lie within \(\varepsilon\) of the same centre \(u_i\). Then \(\|e_j - e_k\| \le \|e_j - u_i\| + \|u_i - e_k\| < 2\varepsilon < \sqrt2\), contradicting the displayed identity. \(\square\)
Compare this with finite dimensions, where Heine-Borel says closed and bounded is enough, which is why finite dimensional approximation results carry no compactness hypothesis and nobody thinks about it. The unit ball of \(\mathbb R^n\) is compact for every \(n\); the constant that quantifies its \(\varepsilon\)-net just grows like \(\varepsilon^{-n}\). Passing to infinite dimensions does not degrade that constant. It removes the net altogether.
The negative result would be crippling if the only alternative to "all inputs of norm at most one" were "a finite list of inputs". It is not. Shrinking the coefficients of an input faster than square-summably restores compactness, and that condition is exactly what a smoothness assumption says.
Claim 6. Fix an orthonormal basis \((e_j)_{j \ge 1}\) of \(H\) and positive weights \(a_j\) with \(a_j \to \infty\). The ellipsoid \[E = \Big\{ u \in H : \sum_j a_j |\langle u, e_j\rangle|^2 \le R^2 \Big\}\] is compact in \(H\).
Proof. Write \(c_j(u) = \langle u, e_j\rangle\). First, \(E\) is bounded. Only finitely many \(a_j\) fall below any threshold, since \(a_j \to \infty\) and every \(a_j\) is positive, so \(\alpha := \inf_j a_j > 0\), and \(\alpha\|u\|^2 \le \sum_j a_j|c_j(u)|^2 \le R^2\) gives \(\|u\| \le R/\sqrt\alpha\).
Now take a sequence \(u^{(n)} \in E\). For each fixed \(j\), \(|c_j(u^{(n)})| \le R/\sqrt{a_j}\) is a bounded sequence of scalars, so a diagonal argument produces one subsequence, still written \(u^{(n)}\), along which \(c_j(u^{(n)}) \to c_j\) for every \(j\) at once. The limit candidate is \(u = \sum_j c_j e_j\), and it lies in \(E\): for each fixed \(m\), \(\sum_{j \le m} a_j |c_j|^2 = \lim_n \sum_{j \le m} a_j |c_j(u^{(n)})|^2 \le R^2\), and letting \(m \to \infty\) keeps the bound.
Convergence in norm is the only step that uses \(a_j \to \infty\). Split at any \(m\): \[\|u^{(n)} - u\|^2 = \sum_{j \le m} |c_j(u^{(n)}) - c_j|^2 + \sum_{j > m} |c_j(u^{(n)}) - c_j|^2 .\] For the tail, use \(|p - q|^2 \le 2|p|^2 + 2|q|^2\) and the fact that both \(u^{(n)}\) and \(u\) lie in \(E\): \[\sum_{j > m}|c_j(u^{(n)}) - c_j|^2 \le \frac{1}{\alpha_m}\sum_{j > m} a_j\big(2|c_j(u^{(n)})|^2 + 2|c_j|^2\big) \le \frac{4R^2}{\alpha_m}, \qquad \alpha_m := \inf_{j > m} a_j .\] Since \(a_j \to \infty\), \(\alpha_m \to \infty\). Given \(\varepsilon > 0\), pick \(m\) with \(4R^2/\alpha_m < \varepsilon/2\), then pick \(n\) large enough that the first sum, which has only \(m\) terms each going to zero, is below \(\varepsilon/2\). \(\square\)
The instance that matters. Take \(H = L^2(\mathbb T^d)\), the Fourier basis, and \(a_k = (1 + |k|^2)^s\). Then \(E\) is the ball of radius \(R\) in \(H^s\), and Claim 6 says a bounded set in \(H^s\) is compact in \(L^2\). This is Rellich's theorem, and the proof shows exactly where the hypothesis bites: \(a_k \to \infty\) holds if and only if \(s > 0\). At \(s = 0\) every weight is 1, the ellipsoid is the unit ball, and Claim 5 says compactness fails. The threshold is sharp, and it is a different threshold from Fact 1's \(s > d/2\). Any positive amount of smoothness buys compactness; buying a point value costs \(d/2\) derivatives.
The theorems assume a compact \(K\). Lab 01 draws its inputs at random, so the question is fair: does a random draw land in one? It does, with probability as close to one as you ask for, and the calculation is short enough to do here.
Lab 01's initial conditions are Gaussian with covariance \(625(-d^2/dx^2 + 25I)^{-2}\) on \((0, 2\pi)\) (../labs/01_burgers/results_b_m16.json, field u0_law). The operator \(-d^2/dx^2\) has eigenvalue \(k^2\) on the mode \(e^{ikx}\), so the covariance has eigenvalue \(625/(k^2 + 25)^2\) there, which is exactly the expected energy in mode \(k\):
\[\mathbb E\,|\hat u_0(k)|^2 = \frac{625}{(k^2+25)^2} \approx 625\,k^{-4} \ \text{ for large } k .\]
Feed that into the \(H^s\) norm and swap expectation with the sum, which is legitimate because every term is non-negative: \[\mathbb E\,\|u_0\|_{H^s}^2 = \sum_k (1+k^2)^s\,\frac{625}{(k^2+25)^2} \ \sim \ 625\sum_k k^{2s-4},\] finite exactly when \(2s - 4 < -1\), that is \(s < 3/2\).
Take \(s = 1\). The expectation is finite, so \(\|u_0\|_{H^1}\) is finite for almost every draw, and Markov's inequality turns that into the statement the theorems want: \(\mathbb P(\|u_0\|_{H^1} > R) \le \mathbb E\|u_0\|_{H^1}^2 / R^2\), so for any tolerance \(\delta\) there is an \(R\) with at least \(1 - \delta\) of all draws inside the \(H^1\) ball of radius \(R\). By Claim 6 that ball is compact in \(L^2\). The 1000 training inputs and 200 test inputs of Lab 01 therefore sit inside one compact set, with a probability you can name, and the approximation theorems apply to them.
Two things about that argument matter later. It is a statement about the input distribution, not about \(L^2\), and it would fail for a distribution with a flatter spectrum: white noise has \(\mathbb E|\hat u(k)|^2\) constant, so \(\mathbb E\|u\|_{H^s}^2\) diverges for every \(s \ge 0\) and no such \(R\) exists. And the same exponent, \(|\hat u_k| \sim k^{-2}\), does double duty: at \(s < 3/2\) it puts the inputs in a compact set for Fact 2, and because \(3/2 > 1/2\) it also gives them continuous representatives for Fact 1. One decay rate, two hypotheses, both satisfied. Module 5 shows what breaks in a theorem when neither is checked.
The third fact is the one that decides how many Fourier modes an FNO layer has to keep, so it is the closest of the three to a design choice. A Hilbert space \(H\) is separable if it has a countable dense subset, which is equivalent to having a countable orthonormal basis; every space in this course is separable. Given such a basis \((e_j)\), the truncation \(P_m u = \sum_{j \le m}\langle u, e_j\rangle e_j\) is the orthogonal projection onto the first \(m\) coordinates. The operator norm \(\|T\| = \sup_{\|u\| \le 1}\|Tu\|\) is the worst error \(T\) makes on the unit ball.
Claim 7. \(P_m u \to u\) for every \(u \in H\), and yet \(\|I - P_m\| = 1\) for every \(m\).
Proof. Parseval gives \(\|u - P_m u\|^2 = \sum_{j > m} |\langle u, e_j\rangle|^2\), the tail of a convergent series, so it goes to zero for each fixed \(u\). For the norm, \(\|I - P_m\| \le 1\) because \(I - P_m\) is an orthogonal projection. Now test it on \(u = e_{m+1}\), which has norm 1: \((I - P_m)e_{m+1} = e_{m+1}\), so the norm is at least 1, hence exactly 1. \(\square\)
Consequence. The two halves of Claim 7 do not contradict each other; they differ in the order of the quantifiers. Pointwise convergence says: for each \(u\), for each \(\varepsilon\), there is an \(m\). Norm convergence would say: for each \(\varepsilon\), there is an \(m\) that works for every \(u\) at once. The second is false, and the witness moves: whatever \(m\) you pick, the basis vector \(e_{m+1}\) is missed entirely.
The rescue. Uniformity returns on any compact \(K\), and Fact 2 is what makes the argument run. Cover \(K\) by finitely many balls of radius \(\varepsilon\) around centres \(u_1, \dots, u_n\), which Fact 2 guarantees is possible. Choose \(m\) large enough that \(\|u_i - P_m u_i\| < \varepsilon\) for all \(n\) centres at once, which is possible because there are finitely many of them. Then for any \(u \in K\), picking a centre with \(\|u - u_i\| < \varepsilon\), \[\|u - P_m u\| \le \|u - u_i\| + \|u_i - P_m u_i\| + \|P_m(u_i - u)\| < \varepsilon + \varepsilon + \varepsilon = 3\varepsilon,\] using \(\|P_m\| \le 1\) on the last term. This is the shape of every discretization invariance proof in the course: a finite net on a compact set, plus a uniform bound on the operator. Fact 2 supplies the net, Fact 3 supplies the reason you need one.
An FNO layer keeps \(k_{\max}\) Fourier modes and discards the rest, so \(P_m\) with \(m = k_{\max}\) is inside the architecture. Fact 3 says no choice of \(k_{\max}\) carries a uniform accuracy guarantee over all inputs of bounded \(L^2\) norm, and Fact 2 says the guarantee returns as soon as the inputs are confined to a compact set of the kind the previous subsection exhibited. Lab 01 puts a size on the effect. Two identical models, one with \(k_{\max} = 8\) and one with \(k_{\max} = 16\), both trained on 64 points, report relative \(L^2\) errors of 0.0224 and 0.0226 on that grid and 0.0349 and 0.0264 at 1024 points (../labs/01_burgers/results_b_m16.json, results_b_m32.json). Doubling the retained modes is worth nothing at the training grid and 24 percent at 1024 points.
How much of that 24 percent is truncation as Fact 3 describes it, and how much is the separate grid-scale artefact Module 2 measures, is not settled by these two numbers. The ablation that would separate them is listed in Module 2 as not yet run. What Fact 3 does settle is that the question is not silly: there is no theorem that makes the gap zero, and the size of the gap is an empirical quantity.
One operator runs through the whole course, and it is the one from the worked example above: send \(f\) to the solution of \(-u'' = f\) on \((0,1)\) with \(u(0) = u(1) = 0\). It earns its place by being the simplest object that still has every feature the hard problems have. It is linear, so its Green's function exists in closed form, and Module 2 writes it down and checks it. It smooths, taking two derivatives off the input, so it maps \(L^2(0,1)\) into \(H^2(0,1) \cap H^1_0(0,1)\) and is a compact operator, which is Fact 2 seen from the output side. And its spectrum is explicit, which is what lets Module 2 derive the Fourier layer from the exact solution instead of presenting the layer as a diagram, and Module 3 prove that truncating it has a uniform error bound where truncating the identity does not.
It is also a fair benchmark rather than a rigged one, and the JMLR paper reports the number: a neural operator with zero hidden layers learns this map to relative error \(10^{-7}\) (research_notes/pdf-fno-jmlr.md:537-545). That figure is the sanity floor. Any architecture that cannot solve this problem to near machine precision is broken, and no architecture that solves it has proved anything about Navier-Stokes.
This module has no lab. The five questions below take its place, because the thing that needs measuring here is not a model but your own starting point, and the answer changes how you pace Modules 1 to 5. Answer without references. An incomplete answer tells you what to review; it says little about whether you can learn the subject.
What a satisfactory answer contains. (1) Functions equal almost everywhere are the same element; changing one value changes no norm; and the rescue, that a class with a continuous representative has exactly one, which \(H^s\) with \(s > d/2\) guarantees. (2) An orthonormal sequence has pairwise distance \(\sqrt 2\); and the rescue, that an ellipsoid with weights going to infinity is compact, so a bounded set in \(H^s\) with \(s > 0\) is compact in \(L^2\). (3) On the unit torus the multiplier is \(\exp(-4\pi^2\kappa t|k|^2)\) for \(\kappa > 0\); it damps high \(k\) exponentially, so inverting it amplifies noise at high \(k\) exponentially. (4) A bounded Lipschitz domain, \(a \ge \alpha > 0\) bounded, \(f \in H^{-1}\), trial and test space \(H^1_0\), a bounded coercive bilinear form, Lax-Milgram. (5) The order of the quantifiers and the set the supremum runs over. Module 1 proves (4). Module 2 derives (3).
Pacing rule. If 1, 2, and 5 come easily, take Module 1 in four days. If 4 is unfamiliar, give Module 1 a second week. Fluency with the Fourier transform does not imply comfort with Sobolev spaces or with spectral approximation estimates, so do not skip Module 1 on that basis.
research_notes/pdf-fno-jmlr.md:300-304, :705-711.21-1524.pdf; notes cover Theorem 8 and Assumption 9 at research_notes/pdf-fno-jmlr.md:467-468, :477-478.research_notes/pdf-fno-jmlr.md:537-545.research_notes/pdf-fno-jmlr.md:108-112, :121.research_notes/web-papers.md:600-616.../labs/01_burgers/results_b_m16.json (field u0_law; test errors 0.0224 at 64 points and 0.0349 at 1024), results_b_m32.json (0.0226 and 0.0264). Block-average down-sampler: ../labs/02_thermal/run_thermal.py:92-95.research_notes/company-episode.md:21-31; transcript in research_notes/earlier_episode_transcript.txt.You can now do four things you could not do before this page: name which of the three objectives any given claim is about, say why a five-point record of an input leaves the output undetermined and by how much, prove that an approximation theorem needs a compact input set rather than a bounded one, and produce a compact set that a real input distribution actually lands in. The questions below test the last three; the first is tested by every later module.