Weight-Space
Programming

What happened—and what to learn from it

Weight-Space Programming (WSP) is the authors’ name for using a general-purpose large language model (LLM) to inspect, repair, or construct a neural network’s numerical parameters directly, without an optimizer writing the numbers. Takehi, Teoh, Zhang and Khattab’s November 2 research post reports eight experiments, ranging from tiny game policies to a pretrained language model. [1]

The editor in every experiment is Astra: OpenAI’s gpt-6-astra, called through the Codex command-line tool with every built-in tool switched off. The harness audited all 1,125 released turns as free of built-in tool calls. Where code was allowed, Astra wrote a Python block and the authors’ sandbox ran it. [27] The headline target is Qwen3.5-0.8B, a 0.8-billion-weight open model that can write out its reasoning before it answers (“thinking mode”). MATH-500 is a 500-problem mathematics benchmark.

The video’s first practical rule is about method: before you read a benchmark jump as new intelligence, identify which failures disappeared. Here, 576 of the 579 rescued practice answers (99.5%) had been cut off at the output limit before the edit, after repetitive reasoning. The gain holds under the measured protocol. “Six times smarter” claims more than that measurement supports, because the protocol measures finished, graded answers. [22] [2] [3]

Three published responses to test against the data

Simon Willison separates two questions the post combines: what a model can do with raw weights and no tools, and what a model armed with code can do to another model. Willison then showed that a laptop model, Qwen 3.8 27B, named 3 of 4 games from the same prompts. [19] Matthew Honnibal calls the post misleading on this point: raw weights in the prompt appear only in Experiments 1, 2 and 4, on networks of 17,253 to 171,664 weights, and every experiment on a real language model uses Python. [20] Sean Goedecke argues that a competent human researcher could do the same tasks, and that the game test reduces to a three-line norm calculation. [21] Section 02 reruns that calculation on the released weights, and sections 04 to 08 check the other claims against the logs.

Evidence labels throughout: “reported” refers to the authors’ measurements; “real data” means this page computed the values from a released file; “derivation” is mathematical explanation; “illustration” denotes a teaching example, not a reconstruction of their model.

Weights are a program’s numerical machinery

A weight is a numerical parameter used in a network’s computation. A bias is an additive parameter; both belong to the parameter collection. An activation is an intermediate value produced for a particular input. Weights usually persist across inputs; activations change with the input. A multilayer perceptron (MLP) is a stack of layers of the kind defined next. A policy network maps a game observation to one score per action. PPO (proximal policy optimization) is a reinforcement-learning algorithm that trains such a policy from experience of playing.

Let x be an input vector, W a weight matrix, b a bias vector, h a hidden activation, and φ a nonlinear function such as ReLU, which replaces negative values with zero. One layer computes:

A forward pass evaluates this computation through the network. A gradient tells us how a loss—a numerical error measure—changes with each parameter. Let θ denote all model parameters, L the loss, η the step size, and ∇ the gradient operator. A basic training update is:

For a WSP edit, let Δ be the change designed by the LLM. Then the update is simply θ′ = θ + Δ. The difficult work is choosing a meaningful Δ: for instance, implementing a paddle controller or changing a particular direction written into a language model’s internal state.

No gradients does not mean no optimization. The Qwen experiment iterates over candidate edits using practice results. At the system level, it is a feedback-driven search over interventions. “No training” means no target-model training procedure was allowed in this experiment. The editor’s own prior training, the target’s pretraining, evaluation compute, and human-designed tools still matter.

How can numbers reveal behavior?

Experiment 1. The authors trained PPO policies on their own 10 × 10 grid versions of four games. All four networks have the same shape (500 → 32 → 32 → 5, tanh, 17,253 weights), the same 5 input channels per cell (agent, moving objects, reward targets, obstacles, previous frame), and the same 5 actions. Astra received one network per conversation: the channel and action meanings, and all the weights as text rounded to 3 decimals. No game was named and no candidate list was given. It had no tools and one reply. It named a correct game for 4 of 4 networks. [4]

Three limits apply to that 4 of 4. The Freeway answer was “Frogger”, which the scorer maps to Freeway by keywords. Each network is the best of 3 seeds for its game. Four networks do not estimate an accuracy rate.

The prompt gives the numbers a coordinate system. Input i = (row × 10 + column) × 5 + channel. So input 405 is row 8, column 1, channel 0. Astra quoted weights of 4.066 and −4.013 for that input, and the released file holds those two values. If many strong connections read the player channel along the bottom row and the outputs separate left from right, a paddle controller is a plausible explanation. This is inference from structure, not a general decoder that translates arbitrary weights into English.

A symmetry that makes this difficult

Let W₁ and W₂ be the input and output matrices of a hidden layer, with biases omitted. Let P be a permutation matrix that reorders hidden neurons. For an elementwise activation φ, define new matrices W₁′ = PW₁ and W₂′ = W₂Pᵀ, where superscript T denotes transpose. Then:

The network’s function is unchanged even though many weight positions move. Raw coordinates are not a unique description of a learned function. Reliable interpretation must track relationships rather than attach permanent meanings to neuron numbers.

What survives the symmetry when inputs are labeled

Let e_j be the vector with a one in position j and zeros elsewhere. Then W₁e_j is the column of W₁ that holds every weight leaving input j. A permutation matrix only reorders entries, so it keeps Euclidean length: PᵀP = I. Therefore:

The total weight leaving a labeled input has the same size under every reordering of the hidden neurons. This is why Experiment 1 is tractable: each input already names a cell and a channel, so one number per input gives a map of the board. The argument requires labeled inputs. Experiment 3 removes the input layer, so this map does not exist there. The released files store inputs as rows (h1 = tanh(x @ W1 + b1)), so the code below computes the same quantity as a row norm.

Shares are of squared norm. An even spread over the 8 playable rows would give 12.5% per row.

import numpy as np

d = np.load("a9f535cc.npz")  # exp1_which_game/networks/ in the logs
W1 = d["actor/dense1/kernel"]  # 500 x 32, inputs as rows
norms = np.linalg.norm(W1, axis=1).reshape(10, 10, 5)
agent = norms[:, :, 0] ** 2
print("share per row:", np.round(agent.sum(axis=1) / agent.sum(), 3))
print("share per col:", np.round(agent.sum(axis=0) / agent.sum(), 3))

What this shows and does not show. Goedecke published this same norm calculation and its heatmaps as the case that the task is within a person’s reach. [21] The table supports part of that case: a script with no game knowledge finds a one-row agent, a one-column agent, and a band of targets. The script does not name a game. That final step needs knowledge of what small grid games look like, and Astra supplied it in 833 to 1,034 reasoning tokens. The logs do not establish how: the released reasoning summaries quote individual weights and patterns, and they do not show a norm calculation.

The same prompts on other models

Willison ran the released prompts unchanged on other models, one run per prompt. [19]

On Pac-Man, Qwen 3.8 used 43,506 reasoning tokens against Astra’s 1,034, about 42 times as many. Elapsed times are not comparable, because the hardware differs. So the ability to name these games from weights is not limited to one hosted frontier model, on this evidence of 3 correct answers in 4 runs. Two details of the Qwen answers need a check against the files. They describe an “8×8” board, which matches the playable interior of the 10 × 10 grid in the prompt. The Pong answer places the paddle on the right side, while column 1, at the left edge, holds 94.5% of the agent channel: the game name is right and that detail is wrong. Willison’s judgment is that a sufficiently experienced person with a notepad and a pencil could get there with enough effort.

Experiment 2: a diffusion model’s output layer

A diffusion model generates an image by removing noise step by step. The authors trained three such models (DDPM, MLP 160 → 64 → 64 → 144, ReLU, 23,824 weights) on filled circles, filled squares, and filled triangles at 12 × 12 pixels. Astra received all the weights as text and an instruction not to compute any forward pass. It named the right shape for 3 of 3 models at 84%, 77% and 88% confidence. It also described every shape as a hollow outline. A shape classifier sorted 99.4%, 98.4% and 100% of 1,000 generated samples into the intended shape, and those samples are filled. [5]

The post says each hidden unit’s output weights draw one piece of the shape’s edge. Fig. 2 supports the edge and adds a detail: a pixel value of −1 is black and +1 is white, so an orange edge writes dark pixels, and in several of these units a teal patch beside it writes bright ones. A patch next to an edge is compatible with a solid shape, which the outline drawings omit. This is a visual reading of 9 of the 192 output units, so treat it as a lead and not a measurement. The authors read the outline error as evidence that Astra reads weight geometry and does not simulate the network. That is their inference from one kind of mistake on three models.

Experiment 3: no labeled inputs, Python allowed

The third test removes the labels. Three character-level language models (4 layers, 6 attention heads per layer, 7.16 million weights) were trained on Python code, Japanese in Latin letters (romaji), or English Wikipedia. Astra received only the four middle blocks, 7,084,032 numbers, loaded in a Python sandbox rather than pasted into the prompt. It could compute linear algebra on the matrices, used 6 to 11 of 100 allowed runs, and could not run the network or any layer on any input. It was told the three candidate texts and asked which applied, and what each head does. [6]

A copying (induction) head is an attention head that finds an earlier occurrence of the current text and attends to what followed it. The authors measured each head by running the network on a random 40-character string repeated twice. Astra never saw this measurement.

Move the cutoff in Fig. 3 and read the match. At the post’s cutoff of 0.2, Astra named exactly the 5 measured copying heads, in both conversations for each network. The result does not depend on the exact cutoff: measured scores fall into two groups, at most 0.019 and at least 0.220, so any cutoff between them returns the same 5 heads. The logs’ own scoring field true_copy_heads uses 0.3 instead, which keeps only 2 of the 5 heads; the experiment’s README documents this difference. [6]

The language answer is weaker evidence than the heads. Astra was correct for all 3 networks, but this is a choice among 3 stated candidates, and its stated probabilities were 0.48 to 0.62. A uniform random guess would get all 3 right 3.7% of the time ((1/3)³), and the two conversations per network are not independent draws. The authors note that Astra’s method matches Elhage and colleagues’ framework, which the prompt did not mention. [25]

Experiment 7: the case where weights alone failed

The authors fine-tuned pairs of Gemma 3 270M, a 270-million-weight model, in the same way. One copy learned to give the correct answer. The other learned sycophancy: following the user’s suggested answer even when it is wrong. In four attempts with weights and Python, Astra could not tell them apart. Attempts 1–3 answered P = 0.5 for every pair, with “A” as a forced tie-break. The authors stopped attempt 4 after 29–30 runs per pair, with one pair finished. [10]

Astra then received a tool that runs both models on up to 16 texts of its choice and returns only the attention patterns, meaning how strongly each position attends to each other position. It picked the sycophant in all 3 pairs, at 90%, 97% and 55% confidence. In one measurement, from pair 2 over 8 texts, head 10.2 of the honest model puts 94.3% of its attention on the question’s numbers and 0.75% on the user’s answer; the sycophant puts 3.5% and 85.7%. One fact changes how to read this test: the unedited starting model already follows every wrong guess. So the sycophant keeps the starting behavior, and the honest fine-tune is the copy that learned something new (it follows 0.7% to 2.8% of wrong guesses). [10]

All eight experiments, sorted by what Astra could access

The access columns follow Honnibal’s table [20], checked here against each experiment’s README. Raw floats in the prompt with no tools occur in Experiments 1, 2 and 4 only. The larger the network, the more the result depends on code and on running the target. So the small-network results do not show that Astra read a 0.8-billion-weight model the same way.

Think “compile an algorithm into neurons”

In the writing experiment, Astra received MinAtar game source code and the input/output layout, then specified edits to an all-zero network. MinAtar is a set of small Atari-style games on a 10 × 10 grid. It had no opportunity to try the policy first. The network architecture was 600 → 32 → 32 → 6, with tanh hidden activations, totaling 20,486 parameters. The final reply used a compact weight-edit language, so Astra did not write out every entry: the Breakout reply has 92 statements and sets 815 of the 20,486 parameters (4.0%) to nonzero values, in 501 seconds. [8]

A tiny network you can understand completely

The following is our teaching example, not the authors’ policy. Let b be ball position, v horizontal velocity, p paddle position, and τ = 2 the prediction horizon. Ignore walls and collisions. Define the predicted target t = b + τv and error e = t − p. Two ReLU neurons encode leftward and rightward error:

For the input column vector x = (b, v, p)ᵀ, the first matrix has rows (−1, −2, 1) and (1, 2, −1). Its biases are zero. The output logits—unnormalized action scores—are 8h_L for left, 0.2 for stay, and 8h_R for right. Choose the largest logit.

import numpy as np

# x = [ball position, horizontal velocity, paddle position]
W1 = np.array([[-1., -2., 1.], [1., 2., -1.]])
b1 = np.zeros(2)
W2 = np.array([[8., 0.], [0., 0.], [0., 8.]])
b2 = np.array([0., 0.2, 0.])


def policy(ball, velocity, paddle):
    x = np.array([ball, velocity, paddle])
    h = np.maximum(W1 @ x + b1, 0)
    logits = W2 @ h + b2
    return ["left", "stay", "right"][int(np.argmax(logits))]


assert policy(5, 1, 3) == "right"
assert policy(3, -1, 5) == "left"
assert policy(5, 0, 5) == "stay"
print("All three toy checks passed.")

What the actual game scores establish

Reported mean scores over 1,000 episodes per policy, with sampled actions and sticky actions at 0.1. The table uses the authors’ best trained seed for each baseline. [8]

Against 500M-step PPO, Astra’s network scores about 20 times as much in Breakout (517.4 against 25.3), 96% as much in Asterix, and 60% as much in Seaquest. So the video’s point stands that knowing the rules does not guarantee the better player. The result demonstrates useful neural program construction. It is not an equal-information contest: the editor has explicit source code and extensive prior knowledge, while PPO learns through interactions. An additional baseline would separate those effects: a human-written controller compiled into the same architecture under the same information access. Goedecke notes that people have already written small networks by hand, such as the AdderBoard challenge. [21] [28]

Experiment 4: a repair that adds a small program

Kinetix is a physics-control environment. The task is a car that must back up, then drive fast enough to jump a red strip. The trained policy (171,664 weights, PPO, about 1 billion steps) never succeeded in 256 episodes: it drives onto the ramp and stalls. Astra received all the weights as text, the scene, and one recorded episode, with no tools and one reply. It wrote 51 edit statements. The edited policy succeeded in 1,024 of 1,024 test episodes. [7]

The 3.54% figure describes a structural change. Astra took over a few units per layer, set their rows and columns to zero, and wired a separate path that reads position and speed, reverses from rest, switches to full drive at body x ≈ 3.8, and turns off once the green wheel passes x = 0.95. Most of the 6,073 changed entries are those zeroed rows and columns. The trained network keeps control after that point. So the repair is a short hand-written controller placed inside the weights. It does not retune what the policy learned.

Experiment 6: writing past the size of the prompt

A 9 × 9 Go network (243 → 512 × 6 → 82, ReLU, 1,480,274 weights) does not fit in a prompt. Astra wrote it through Python from all-zero tensors, with no training, no fitting to data, and no search. A pass let it play up to 128 games against practice opponents or inspect up to 256 positions. Each budget was run once. [9]

Win rates over 2,000 games per opponent. The post’s chart shows only seed 0 of the same-size baseline; all three seeds are listed here.

The 50-pass run used 47 game passes, so it saw at most 47 × 128 = 6,016 games. The same-size PPO baseline trained on 20.5 million games, about 3,400 times as many. That ratio compares different things: Astra starts with general Go knowledge and writes rule code, and PPO starts from nothing. The zero-pass run shows what prior knowledge alone reached: a single 336-line Python run that won 54.1% and 54.9% against small PPO networks and lost to all three same-size PPO networks.

How a loop repair became a math-score jump

The target was the text component of Qwen3.5-0.8B, with 24 layers. Astra could inspect and change weight tensors in a persistent CPU Python session, with at most 300 code runs. A controlled tool executed generations or returned internal measurements. Gradient descent, any fitting of weights to a loss, and running the model in the kernel were forbidden. The editor did not read 0.8 billion decimal numbers in its prompt. It wrote code to inspect selected tensors and propose changes. [2]

1. Diagnose the failure. Inspect representative generations. The initial batch had 85 of 96 answers reaching the 32,768-token output cap and 11 of 96 correct, and about 95% of the phrases at the end of cut-off answers were duplicates.
2. Construct a contrast. Compare productive and repetitive continuations of matched contexts, from four matched pairs. A teacher-forced read evaluates supplied text instead of sampling a new continuation.
3. Locate candidate writers. Find internal directions associated with repetition and MLP components that strongly write along them. Here the MLP is the feed-forward block within a model layer.
4. Intervene and test. Try different strengths and sites. The records include layer-isolation tests and a norm-matched random-direction control, which test causation more directly than a correlation does.
5. Freeze the candidate. Retain a sparse edit in the MLP output matrices of layers 15 and 19. Confirm it on practice problems excluded from edit selection, then evaluate separate benchmarks.

The sequence above summarizes the released p50c records; it is not a complete transcript of the search. Layer numbers follow the code’s zero-based indexing. [3] [12]

The editor’s kernel had no GPU, but target-model execution occurred through the forward tool. This does not establish that the whole experiment was CPU-only or computationally cheap. The recorded p50c wall time was 20.3 hours, including a 4.7-hour service interruption. That is historical run information, not an estimate for your machine.

Dose and control

A nat is a unit of log-probability. Doubling the strength about doubled the effect (2.0 times from strength 1 to 2, 4.3 times from 1 to 4). The final edit lowers repetitive text about 11 times as much as productive text. The random-direction control gained 1.56 points with a standard error of 3.52, so it cannot be distinguished from zero. [12] These measurements come from a small set of matched contexts inside the run, and Astra chose them.

Three budgets, three different edits

Each budget is one run, and the three runs reached three different mechanisms. [29] [30] [12] So the table cannot separate the effect of more passes from run-to-run variation in the search, which matches the video’s point that one run reached 48% and each ran once. The results table in the logs adds other Qwen rows that the post does not report: two runs that submitted no edit under v2 and v3 prompts (0 and 5 passes), a v4 prompt at 10 passes (no edit, 7.9%) and at 25 passes (14.1%), and two candidates that were evaluated but not submitted (8.6% and 10.1%). Four v5 rows have no scores. No row other than the 50-pass run exceeds 14.1%, and the table holds no second 50-pass score. [26]

A sparse, rank-one intervention at each site

The residual stream is the running hidden-state vector to which model blocks add their outputs. For one edited MLP, let a be its 3,584-dimensional hidden activation and W its 1,024 × 3,584 output matrix. Its contribution is y = Wa. Column j of W says what direction hidden neuron j writes into that 1,024-dimensional state. [13]

What is a “repetition direction”?

It is a vector in internal activation space, not an English instruction and not necessarily one neuron. As a pedagogical construction, let μ_rep and μ_prod be the mean hidden states for repetitive and productive contexts. A normalized contrast direction u can be defined by:

Here ‖·‖₂ is Euclidean length. This equation explains a standard contrast idea; it is not a claim that this single line reconstructs the exact released edit. The final report says the run used four matched contexts and layer-specific unit directions measured at residual outputs 16 and 20. The intermediate activation files and direction files are not included. [12]

The actual final formula

Let m be a vector with 64 entries equal to one and all other entries zero, selecting MLP columns. Let ⊙ denote entrywise multiplication and α the edit strength. With a layer-specific u and m at each site, the logged final edit is:

The outer product of a column vector and a row vector has rank at most one: its changes all lie along one output direction. The mask makes the change sparse across columns. “Rank-one” describes the structure of the update, not the number of entries it touches. The formula applies before the evaluated checkpoint is rounded to bfloat16, a 16-bit numerical format. [12] [14]

Strength 6 reverses and amplifies; it does not erase

For a selected output column w, write w = w_⊥ + cu, where w_⊥ is perpendicular to the unit vector u and c = uᵀw. The edited column is:

At α = 1, the selected column’s component along u disappears. At α = 2, it reverses with equal magnitude. At the actual α = 6, it becomes −5cu: reversed and five times as large. The video says “remove” and the post says “subtracted”; both describe the operation, and neither states this end result. This is a derivation from the logged formula. It does not imply the entire model state is projected or globally reversed; unselected neurons, other blocks, and nonlinear dynamics remain.

How small is “0.016%”?

Two layers × 64 columns × 1,024 rows = 131,072 targeted scalar entries. Dividing by the model’s rounded 0.8-billion parameter label gives approximately 0.0164%. The exact released bfloat16 diff has 130,188 entries with changed bits; 884 targeted values round back to their originals. Thus “targeted entries” and “numerically changed entries” are slightly different counts. [14]

Count is also different from magnitude. The two edited matrices change by about 10.64% and 8.11% in relative Frobenius norm—the square-root-of-sum-of-squares measure of matrix size—while the edit touches 0.0164% of the parameters. The headline percentage uses an approximate denominator; it should not be treated as an exact universal parameter fraction. [13]

A real gain with a specific causal story

MATH-500 is a 500-problem mathematics evaluation; GSM8K contains grade-school math word problems. IFEval tests compliance with verifiable instructions. MATH-500 values average four generations per problem; GSM8K and IFEval average two. These are mean accuracies, not “at least one successful answer out of four.” [2]

The MATH-500 increase is 40.2 percentage points, or approximately 6.09 times the baseline score. The ratio is sensitive to how badly the baseline fails: on MMLU-Pro, where the base model hits the cap on 28.9% of answers, the score moves from 40.3% to 45.5%, a ratio of 1.13. Both models used thinking mode, temperature 0.6, top-p 0.95, top-k 20, presence penalty 1.5, and a 32,768-new-token cap.

From the merged.json files. [31] The two AIME rows are not in the post. The post’s figure shows GPQA Diamond’s base score as 24.5, an interim value at 149 of 198 questions; the finished score is 22.7. [2]

The AIME rows test the claim directly. These harder competition problems saw about a third of answers finish that had not finished before, and accuracy reached only 0.6% and 2.3%. Finishing does not supply the ability to solve them. Across the table, the cap rate falls on all ten benchmarks, and the accuracy gain varies from 0.6 to 53.3 points.

An independent re-run of the frozen checkpoint. Ahsen Khaliq rebuilt the edit from the logs, verified its hashes, and ran the full MATH-500 protocol (500 problems × 4 samples, seed 0) on the public checkpoint. The edited model scored 51.0% with 31.6% of answers hitting the cap, against the post’s 48.1% and 33.1%: 2.9 points higher on accuracy. A 25-problem smoke test of the base model gave 5.0% with 95% hitting the cap. [23] This re-run checks the evaluation of one frozen edit. It does not repeat Astra’s search.

Completion and correctness are different quantities

Let F be the event that an answer finishes before the cap, and C the event that it is graded correct. The exact identity is:

The symbol ¬F means “did not finish before the cap,” and P denotes probability. A cut-off response can still contain an extractable correct answer, so its contribution must not automatically be set to zero. In particular, 7.9% accuracy with only 7.0% finishing proves that “all truncated answers are wrong” is inconsistent with these aggregate measurements.

When the second term is small, improving the completion rate can substantially raise measured accuracy even without a comparable improvement in mathematical knowledge. But finished answers form a selected subset: easy questions may finish disproportionately. Comparing correctness conditional on finishing is diagnostic, not a clean experiment isolating “pure reasoning.”

The strongest supporting practice result

The practice evidence. The 1,152 confirmation problems were excluded from edit selection, and no problem overlaps the 352 used earlier in generation-based selection. The records report 579 rescues (wrong before, correct after) and five regressions (correct before, wrong after). Of the rescues, 576 were originally truncated. On the 114 originally finished outputs, accuracy was 93.86% before and after, with three rescues and three regressions. The pooled gain is 49.83 points with a standard error of 1.50 and a 95% normal interval of 46.89 to 52.76 points; the five independent splits gain between 46.9 and 55.1 points. These are held-out practice results, not unseen benchmark scores. [3]

How far “already knew the answer” goes. The video summarizes the edit as letting the model finish answers it already knew how to solve. Among the 1,038 originally truncated problems, 589 became correct (56.7%) and 449 did not (43.3%). Accuracy among finished answers falls from 93.86% to 73.94%. That fall is not damage, because the problems that finished before keep their 93.86%. The likely cause is selection: easier problems finished first, and the originally truncated problems reach 56.7% correct after the edit. The logs do not measure difficulty directly. A footnote in Goedecke’s post, that all problems under the limit were solved, comes close but is not exact: 107 of 114 finished answers were correct. [21] The logs also cannot show that the 589 solutions were present before the edit, because the unedited model never wrote them. The edit can change the whole subsequent reasoning trajectory, and median tokens in correct answers rose from 5,310.5 to 9,171.5. “No new capability was demonstrated” is better supported than the universal claim “weight editing cannot ever add capabilities.”

A bound for the benchmark. The second term of the identity is not negative, so P(C | F) ≤ P(C) / P(F). On MATH-500 after the edit, this gives at most 48.1 / 66.95 ≈ 72%. So at least about 28% of finished MATH-500 answers are not correct. The benchmark files store the cut share per job, not per sample, so this page reports the bound and not the exact value.

What the model card changes

Qwen’s card states that this model runs in non-thinking mode by default, and warns that in thinking mode, “when using the recommended sampling parameters”, it is more prone to thinking loops than other Qwen3.5 models. The text-thinking recommendation is temperature 1.0. Temperature controls the sharpness of token sampling; lower values concentrate probability on the most likely tokens. The experiment used 0.6, and separate research reports greater looping at low temperatures, while noting that higher temperature does not solve the underlying learning errors. [15] [16]

Two more lines of the card narrow that point. The card’s own thinking-mode benchmark table uses the same settings as the experiment, including temperature 0.6 and presence penalty 1.5, so 0.6 is not an unusual choice. And the warning concerns the recommended parameters, so temperature 1.0 is not reported to remove the loops. The card’s example also allows 81,920 output tokens, 2.5 times the experiment’s 32,768 cap. None of these settings was tested against the edit in the released runs. [15]

A larger token cap could help a long but productive solution. An endless loop may simply consume the larger budget. Changing mode, sampling, stopping, or weights can each alter the outcome; measure accuracy, completion, and compute together.

Gemma: a useful negative result with caveats

The Gemma 3 270M follow-up started at 6.8% MATH-500 and reported 7.0%, 7.6%, 7.7%, and 7.0% after budgets of 5, 10, 25 and 50 passes. Astra’s final answers in three of the four runs reported the gain as not established. It had a much lower cap rate, 8.5%, consistent with fewer looping failures to repair. Astra’s notes describe ordinary reasoning errors, such as wrong arithmetic and invalid algebraic steps. [11]

Four other Gemma runs in the results table, which the post does not report, scored 6.7%, 6.7%, 6.1% and 6.2% on the same scale. [26] So across eight runs the range is 6.1% to 7.7% around a 6.8% base. Gemma is not a controlled clone of Qwen: size, architecture, generation settings (no thinking mode, temperature 1.0, 8,192 tokens), and prompt version differ. Its headline scores use a format-tolerant extractor, and under the strict boxed-answer rule that Qwen’s scores use the base is 2.8%, with reported runs at 3.5%, 3.1%, 3.8% and 2.6%. The authors conclude that editing “could not add an ability the model lacked”. The evidence for that is this one model. It limits the demonstrated claim; it does not prove a theorem about what WSP can never do.

Learn the mechanism, then test the claim

Path A — Five minutes, no model download

1. Set the direction slider to 0, 1, 2, and 6. Explain the change in the selected column’s parallel component.
2. Run the toy NumPy controller above. Replace all weights and biases with zero and observe that all actions tie. Restore the designed parameters: the rule reappears without training.
3. Open the final report and exact diff manifest. Check the formula and two changed tensors yourself.
4. Use the recipe in section 02 on a network from exp1_which_game/networks. Confirm the 95.9% in row 8.

Path B — Repeat the game test on your own model

Each prompt is a self-contained text file, so any model with a long enough context can take it. The commands below are the ones Willison used, with the LLM command-line tool and LM Studio. [19]

llm install llm-lmstudio
LMSTUDIO_TIMEOUT=900 llm \
  -f https://raw.githubusercontent.com/RikiyaT/lm_understand-logs/refs/heads/main/exp1_which_game/runs/pacman/prompt.txt \
  -m lmstudio/qwen/qwen3.8-27b

Set the context window above the prompt size: Astra’s input count was about 73,700 tokens, and Willison reports that an 8,192 context failed with errors. The Qwen runs in that post took 46 minutes to 1 hour 5 minutes. Replace pacman with breakout, pong, or freeway. The URL names the game; the prompt text does not.

Path C — Reconstruct the published parameter edit

This replays the final artifact; it does not rerun Astra’s discovery process. To skip the rebuild, use the published edited checkpoint or the hosted demo, which shows the base and edited models side by side. On Windows, use Git Bash. First clone the logs, create an isolated Python environment, and install NumPy:

git clone https://github.com/RikiyaT/lm_understand-logs.git
cd lm_understand-logs
git rev-parse HEAD
python -m venv .venv
source .venv/Scripts/activate
python -m pip install numpy

Record the printed repository commit. Download the original model.safetensors from the pinned Qwen revision 2fc06364715b967f1860aea9cf38778875588b17 using the model’s file browser. Put it in a new base-qwen folder. Then run the authors’ reconstruction script:

python exp8_math/code/apply_edit.py \
  base-qwen/model.safetensors \
  exp8_math/runs/p50c/edit \
  edited-qwen.safetensors

The script checks the base tensor hashes and rebuilt tensor hashes. Expect two “hash OK” messages. It supports the public single-file release and needs no PyTorch for reconstruction. A hash mismatch means the input is not the expected revision; do not bypass the check. [17]

Reconstruction works on CPU and needs the base weights, output storage, and RAM for the arrays. Allow several GB of headroom. On macOS/Linux, activation is source .venv/bin/activate. The command above follows the authors’ script interface; this tutorial did not download or execute the full Qwen checkpoint.

A weights file alone is not a complete runnable model folder. For inference you also need compatible configuration, tokenizer, chat template, and a runtime supporting this Qwen architecture. Do not overwrite a configuration with the wrong architecture. Reconstructing exact tensors verifies the patch; generating answers verifies behavior; rerunning the editor tests discovery reproducibility. These are three distinct tasks.

Path D — Test whether the edit earns its complexity

Use a fixed set of math problems that neither the editor nor your tuning process sees. Keep a development set for choosing settings. Evaluate both the base and edited checkpoint under every configuration below:

Log each problem’s correctness, extracted final answer, whether thinking closes, whether the cap is hit, repetitive-tail rate, tokens, and elapsed time. Use several sampling seeds and report uncertainty over problems. Several samples of one problem are related observations, not several new benchmark questions.

For a paired comparison, let dᵢ equal edited correctness minus base correctness for problem i, averaging samples within a problem first. Let N be the number of problems, g the mean paired gain, and s_d the sample standard deviation of the problem-level differences. A simple standard error is:

For stronger validation, resample whole problems in a paired bootstrap, and repeat independent editing searches. The first estimates uncertainty for a frozen edit on a problem population; the second measures whether the editing process reliably finds good patches.

Where the research frontier actually lies

The connection to earlier interpretability work: Arditi and colleagues showed that a behavior can sometimes be mediated by a low-dimensional internal direction. WSP’s contribution here is that a general LLM organized the investigation, chose an intervention, and wrote the change. The underlying idea of directional intervention predates this result. A low-dimensional explanation for one behavior is not evidence that every behavior has a single removable direction. [18]

The human-baseline argument. Goedecke lays out the same steps as a playbook a researcher would try, comparing it to the Heretic “abliteration” tool: write looping and non-looping texts, take the difference of their internal states, find the weights that contribute most, and trim them. The post’s analogy is that editing weights resembles tweaking a compiled binary, and that the better route stays training. [21] Honnibal reaches the same place from the access table: the Python experiments show tasks a human engineer could do. [20] No released experiment measures a human against Astra, so both claims are arguments and not results. What the logs do show is the time and the budget: 20.3 hours (4.7 of them an outage), 158 turns, and 50 passes at maximum reasoning effort, with Astra choosing the steps.

What the post projects, against what it measured

A research interpretation: the use with direct support in these records is an automated neural debugger: form a failure hypothesis, inspect internals, propose a reversible patch, test controls, and retain only changes that survive independent checks. Whether such methods can efficiently install new skills in large pretrained models remains open. The authors say they are building a benchmark and harness for this question.

What this page does not establish

This page reruns no model. The labs recompute statistics from the released files of Experiments 1–3, and the numbers for Experiments 4–8 are copied from the released records. The logs release Astra’s reasoning only as summaries or as explanations the harness asked for, and some runs release neither, so claims about how it reasoned stop at what those records state. The logs were read as released, and nothing here audits whether a released file matches what ran.

Three checks for understanding

What’s next

A high-value solo experiment: build a small model-failure dashboard that compares base versus patched weights and separates accuracy, completion, repetition, and token cost. Start by asking whether tuned decoding recovers most of the apparent advantage. That makes a falsifiable contribution without first recreating the entire autonomous editor.

Sources and evidence trail

The research post and logs supply the experimental facts. The video supplies the critical framing and the two practical rules. Willison, Honnibal, and Goedecke supply the replication and the published critiques. Details were checked against the primary records below. Diagrams labeled “illustration” or “derivation” are explanatory constructions; labs labeled “real data” are computed from the named files. None is a source screenshot. This page does not claim its own rerun of the Qwen benchmark or editor.

1. Takehi, Teoh, Zhang & Khattab — Sparks of Weight-Space Programming. MIT CSAIL research post, November 2, 2026. Definition and experimental overview.
2. Experiment 8 README. Protocol, budget meaning, score provenance, run accounting, released and omitted material.
3. p50c submission evidence. Paired practice confirmation, rescues, regressions, and descriptive outcome strata.
4. Experiment 1: game identification.
5. Experiment 2: diffusion-model shapes.
6. Experiment 3: language and copying heads.
7. Experiment 4: car-policy repair.
8. Experiment 5: MinAtar policy construction. Baselines, source-code access, edit language, and scores.
9. Experiment 6: Go policy construction.
10. Experiment 7: sycophancy identification.
11. Gemma follow-up README. Negative result and grading differences.
12. p50c final report. Actual α = 6 formula, direction origin, controls, final confirmation.
13. Frozen weights manifest. Shapes, selected neurons, relative matrix-change sizes.
14. Exact evaluated diff manifest. 130,188 numerically changed bfloat16 entries.
15. Qwen3.5-0.8B official model card. Default mode, thinking-loop warning, recommended decoding.
16. Pipis et al. — Wait, Wait, Wait… Why Do Reasoning Models Loop? Temperature and looping; separate research, not a direct test of this edit.
17. Authors’ apply_edit.py. Pinned base revision and bit-exact reconstruction interface.
18. Arditi et al. — Refusal in Language Models Is Mediated by a Single Direction. Prior directional-intervention work.
19. Simon Willison — Qwen 3.8 27B on my laptop can interpret model weights, too. November 3, 2026. Local replication of Experiment 1, commands, and timings.
20. Matthew Honnibal — The ‘Astra can read and write weights to improve Qwen’ post is misleading. November 3, 2026. Access table across experiments.
21. Sean Goedecke — LLMs are not superhuman at reading and writing neural network weights. November 3, 2026. Norm heatmaps and the human-baseline argument.
22. Prompt Engineering — GPT-6 Astra Made Qwen 6x Better at Math Without Training. YouTube video and transcript. Key chapters: definition, network construction, Qwen, headline interpretation.
23. Edited Qwen3.5-0.8B checkpoint on Hugging Face. Hash-verified rebuild and independent MATH-500 re-run by Ahsen Khaliq.
24. Hosted demo. Base and edited models on a problem you type.
25. Elhage et al. — A Mathematical Framework for Transformer Circuits. The matrix-composition method that matches Astra’s route in Experiment 3.
26. Results table excerpt. Includes runs that the post does not report.
27. lm_understand-logs repository. Harness description, the tool-free audit, placeholders, licenses, and all experiment folders.
28. Papailiopoulos — AdderBoard. Community challenge in setting the weights of tiny arithmetic transformers.
29. 10-pass final answer.
30. 25-pass final answer.
31. 50-pass and unedited evaluation scores. Accuracy, cap share, and mean tokens per benchmark.

Attribution: the repository states that its logs, data, and networks are CC BY 4.0 and its scripts are MIT-licensed; third-party material retains its own terms. The arrays behind the real-data labs in section 02 are derived from exp1_which_game/networks/*.npz, exp1_which_game/eval/scores.json, exp2_diffusion/networks/*.npz, exp2_diffusion/eval/samples/, exp2_diffusion/runs/*/answer.md, exp3_language/eval/heads.json, and exp3_language/eval/scores.json, rounded for size. This tutorial links to the originals and adds independent mathematical derivations, diagrams, a toy controller, and an evaluation plan.