Module 10 of 14 · one week, plus the capstone
This is the course's research module, and the one claim it exists to test: that a model pretrained on several physical systems needs less data on a new system than an equal-size model trained on that system alone. The public record supports a narrow version of that claim with exact numbers, and complicates the broad version with documented negative transfer. Both go here, followed by the experiment that would settle it for your own systems.
The company's stated core finding is that "the shared model wins": a multi-physics model beats equal-parameter single-domain specialists (research_notes/company-episode.md:153). It has published no benchmark, and a Tech Times headline on 26 August 2026 says as much in its title (:123).
That is not a dead end, because the claim is not proprietary. Anandkumar's own co-authored paper on the subject, DPOT, is public, and so is the work of five other groups. The claim is testable from the published record, and this module reads those papers for the claim, not for the architecture. Module 5 supplied the reason to be careful: empirical risk minimization on a measure \(\mu\) leaves predictions under a different measure \(\nu\) entirely free, so "transfer works" is never a fact about a model on its own, only about a model and a pair of distributions.
Hypothesis. A model pretrained on systems \(S_1,\dots,S_k\) and fine-tuned on a fraction \(f\) of the data of a held-out system \(S_{k+1}\) reaches lower test error on \(S_{k+1}\) than an equal-parameter, equal-compute model trained from scratch on the same fraction, across several seeds, with the gap largest at small \(f\).
Alternative outcomes, all of which have been observed somewhere. The gap is zero, so no transfer. The gap is negative, so interference. The gap depends on which systems are in the pretraining set, so transfer is structure-dependent. Or the gap vanishes as \(f\to1\), so transfer is a small-data phenomenon only.
Three words in that hypothesis carry the whole burden. "Equal-parameter", because a larger pretrained model beating a smaller scratch model proves nothing about pretraining. "Held-out", because a system that appeared in pretraining is not a transfer target, it is a training set. And "fraction", because the claim is about data efficiency, so the result is the whole curve of error against \(f\) and never a single point on it. A paper reporting one number at one data fraction has not tested this hypothesis. The project kit's experiment matrix (../05_physics_foundation_model_study_kit.html) is this hypothesis with four arms, and the lab below is that matrix instantiated on this course's own data.
| Paper | Pretraining set | Transfer result | Grade |
|---|---|---|---|
| Poseidon (Herde et al., ETH, arXiv 2405.19101) | 4 compressible Euler and 2 incompressible Navier-Stokes operators, 77,840 trajectories of 11 snapshots each, giving about 5.11 million pairs by all-to-all pairing | Across the median of 15 downstream tasks, 20 fine-tuning samples reach the error FNO needs 1024 samples for, a factor of about 50. Per-task efficiency gains run to 890.6× on NS-PwC, 552.5× on NS-BB, 502.9× on NS-SVS, 78.3× on Helmholtz and 62.1× on Wave-Gauss. At 128 samples the average accuracy gain is "an entire order of magnitude". Poseidon-L needs an order of magnitude fewer samples than FNO on 13 of the 15 tasks, and 9 of the 15 involve physics absent from pretraining. | A foundation.md:29-41 |
| Subramanian et al. (NeurIPS 2023, arXiv 2306.00258) | Three 2D steady systems: Poisson, advection-diffusion, Helmholtz, with FNO throughout | On Poisson, reaching a target error of \(10^{-2}\) takes about 64 fine-tuning examples with pretraining against about 8,000 from scratch, a factor of about 100. Advection-diffusion shows similar \(10^3\) to \(10^4\) savings in the few-shot regime. Across model sizes from 64K to 256M parameters, fine-tuned error falls monotonically with size and gains more from added parameters than scratch training does. | A foundation.md:97-102 |
| DPOT (Hao et al., with Anandkumar, arXiv 2403.03542) | 12 datasets drawn from FNO Navier-Stokes, PDEBench, PDEArena and CFDBench, over 100k trajectories, models up to 0.5B parameters | High-resolution 2D turbulence 16.7 to 13.5 percent, about 19 percent relative. 3D Navier-Stokes transfer 41 to 22.6 percent, about 45 percent relative. Long Kolmogorov rollout 82.2 to 33.5 percent. Claimed up to 52 percent reduction against baselines. | B foundation.md:45-56 |
| Aurora (Microsoft, arXiv 2405.13063) | "Over a million hours" of ERA5, HRES, IFS ensembles, GFS, GEFS, CMIP6, MERRA-2 and CAMS; 150k steps on 32 A100s | Air quality: matches or beats CAMS on 74 percent of targets and 89 percent at 3 days, with pretraining beating scratch by 54 percent on average. High-resolution weather at 0.1°: lower RMSE than IFS HRES on 92 percent of targets, with pretrained beating scratch by 25 percent. Its flexible encoder ingests sources at different native resolutions. | A foundation.md:129-138 |
| UPT (Alkin et al., NeurIPS 2024, arXiv 2402.12365) | ShapeNet-Car, 10K transient pipe flows, Lagrangian Taylor-Green | Not a multi-PDE-family result, and should not be cited as one. The relevant finding is about resolution: trained with 2048 supernodes and 16K queries, it "generalizes across a wide range of different number of input or output positions, with even slight performance increases when using more input points", plus a 400× speedup over pisoFoam in latent rollout. | A foundation.md:60-67 |
Two numbers from that table should be memorised, because they are the ones any argument about physics foundation models has to get past: Poseidon's 20 samples against FNO's 1024, and Subramanian's 64 examples against 8,000. They come from two different groups using two different architectures on two different problem families, and they agree on the order of magnitude. That is as close to independent replication as this literature currently gets.
The same literature contains its own counterexamples, and the papers report them instead of burying them, which should be said before the list.
foundation.md:16-24). Compressible Navier-Stokes at Mach 0.1 and 1.0 and diffusion-reaction do improve, so the effect is per-task and not uniform.:103-107). Note which system fails: Helmholtz is the non-smoothing, oscillatory one, which is also where Module 5's rate arguments do not apply.:56). If the shared representation were doing the work, the larger and more broadly pretrained model would not lose.:166).:163). Module 7's wall does not move because the model was pretrained.:80-86). An impressive number without a control arm is not evidence for the hypothesis in this module.The narrow claim holds. Pretraining on a family of related PDEs reduces downstream data requirements by one to two orders of magnitude in the low-data, near-distribution regime, and that is supported by multiple independent results with reproducible numbers.
The broad claim does not. One model across genuinely different areas of physics, beating equal-size specialists everywhere, has one partial result at 6 of 12 domains and several documented counterexamples, of which MPP's own shallow-water number is the clearest.
Anandkumar's own statement in the earlier episode is the narrow claim, not the broad one: a model cannot transfer to a regime with zero related data, but coupled-physics fine-tuning, such as heat plus stretching, needs far fewer samples than training from scratch (01:14:20 to 01:15:48, company-episode.md:82). The gap between that and the company's marketing is a gap in the marketing, not in her reasoning.
Weather is the one domain where a multi-source, multi-resolution model already beats the operational alternative on held-out data, and by a wide margin: GraphCast beats ECMWF HRES on 90 percent of 1,380 targets, and GenCast beats ENS on 97.2 percent of 1,320 targets (foundation.md:140-146). Weather is also the domain with the most data, the most uniform geometry, and the longest history of standardized evaluation, so it is the best case, not a typical one.
LAB Not yet run. This is the capstone's core, specified here so that Module 13 can refer to it.
The sources are Lab 01's Burgers generator at ν = 0.1, plus a second viscosity of 0.02 as a shifted regime, and a 1D advection-diffusion generator built the same way: \(u_t + cu_x = \nu u_{xx}\) is exact in Fourier space and takes about ten lines. The target is either Lab 02's die thermal operator, which is elliptic and two-dimensional and therefore a genuine change of physics, or, for a first pass, Burgers at the unseen viscosity, which is only a parameter shift. Running the parameter shift first is the right order: if transfer fails there, it will not succeed on the harder target and you have saved four hours.
| Arm | Training | What it isolates |
|---|---|---|
| A | Scratch FNO on the target only, at each data fraction | the baseline curve, without which nothing else means anything |
| B | Multi-task FNO on sources and target jointly, with an equation identifier channel | interference against sharing |
| C | Pretrain on the sources, then fine-tune on the target fraction | the transfer claim itself |
| D | Arm C plus a normalized PDE residual penalty during fine-tuning | Module 11's question |
Hold these constant across every cell: parameter count, optimizer budget in steps, normalization, solver fidelity, and evaluation grid. Use fractions \(f\in\{0.01,0.05,0.10,0.25,1.0\}\) of 1000 target samples, with at least three seeds per cell, and split by initial-condition family and coefficient regime and never by random frames, for the reason Module 8 gives. The deliverable is the plot of error against \(f\) for all four arms with seed bands, plus a one-page transfer map saying where C beat A and where it did not. A negative result with these controls in place is a result, and a more useful one than a positive result without them.
The project kit lists five failure modes to watch for. Aliasing mistaken for super-resolution, which Module 2's spectrum check catches. Target leakage through the split. An equation token that lets the model memorize which dataset it is looking at instead of sharing structure. A residual loss that quietly dominates the data loss. And reporting a single final-time image instead of rollout and invariant errors.
The cost on this machine is manageable. Each FNO run in Lab 01 took about 5 minutes, so 4 arms by 5 fractions by 3 seeds is 60 runs, roughly 5 hours, and less with early stopping on the small fractions. The thermal target at 64² costs more per run; Module 6 has its timing.
research_notes/foundation.md:29-41.:97-107.:10-25.:45-56. UPS 2403.07187: :71-76. ICON 2304.07993: :80-86. The Well dataset 2412.00568: :111-116. The Mamba-backbone model 2511.21861 and the rollout survey 2509.02846: :163-166.:60-67. Aurora 2405.13063: :129-138. GraphCast, GenCast and NeuralGCM: :140-150.research_notes/company-episode.md:102-124, :146-156. Anandkumar on coupled-physics fine-tuning, 01:14:20 to 01:15:48: :82.../05_physics_foundation_model_study_kit.html.You can now state the transfer hypothesis with the three controls that make it falsifiable, cite the strongest published result on each side with its numbers, tell two different failure modes apart, and design the arm that separates memorisation from sharing.