Weight-space programming · Experiment 8 reviewed
Based on the authors' complete public experiment logs. Every English quotation is reproduced verbatim from Astra's own replies; descriptions in the blog, README, and user messages are not attributed to Astra.
The left column quotes the blog verbatim (retrieved 2026-11-03); the right checks each claim against the public logs. This page covers Experiment 8 (math) only. The blog's claim about identifying a network's training objective from weights alone concerns Experiment 1 and is outside this review's scope.
| Claim Blog text | What the logs show |
|---|---|
| “Astra identifies an issue in its weights causing reasoning loops” | Qwen3.5-0.8B's main failure on these math problems was a reasoning loop: it repeated passages during thinking until it exhausted the token limit without answering. Astra found this by using GPU forward tools to run the model and compare internal activations on productive and looping text. The issue was observed in model behavior and activations, not inferred from weights alone. |
| “fixes this by natively editing 0.016% of the weights” | In Python, Astra applied W ← W − α·u·uᵀW to subtract a “loop direction” from MLP outputs in layers 15 and 19. It changed 64 columns per layer, for 2 × 64 × 1,024 = 131,072 parameters①. The percentage depends on which parameter total is used as the denominator; it mainly describes the edit's shape—two layers and 64 columns per layer—not its precision. The blog itself writes: much like removing a behavior by deleting a single direction from a model’s weights, placing it in the same family as the 2024 refusal-direction ablation. |
| “this weight correction raises MATH-500 from 7.9% to 48.1%” | The numbers check out: 7.9% → 48.1%①. Completion rose from 7% to 65%, while accuracy among completed answers fell from 93% to 71%①. The gain came from no longer getting stuck. |
| “generalizes out of the box to instruction following, raising IFEval by 3x” | The numbers check out: 21.0 → 62.3①. The blog itself writes: The loops are not special to math, so performance beyond math improved as well. This is consistent with repairing a general looping defect, rather than learning a new instruction-following capability. |
| “asked to make Qwen3.5-0.8B better at math without gradients” | Accurate. Mean-difference directions and directional ablation do not require gradients. Astra used 50 GPU forwards—generating answers and reading internal activations—to conduct the experiments. |
| “well past what we’d expect from humans or conventional algorithms” | Every step used in Experiment 8 is a standard method developed by human researchers in 2022–2024: mean-difference directions, direct attribution, directional ablation, dose-response tests, random controls, and held-out validation after freezing the edit. |
A more precise description of “natively reading and writing weights” is: fluently running a standard interpretability pipeline in a well-tooled experimental environment, then using code to write a projection formula into the weights. The reading relies on activations and experiments; the writing relies on formulas and code.
Authors' notes Sources: the Experiment 8 README, the round-one prompt.txt for p50c, and the harness directory in the logs repository. Every number below was checked against the source.
save_edit, and execute code up to 300 times.save_edit rejected edits to the final normalization layer, embedding/output layers, and whole-tensor scalar multiplication.This is a complete mechanistic-interpretability workbench: it records internal activations, supports same-problem and same-seed A/B evaluation, and provides a persistent analysis environment.
One 158-round run condensed to 15 entries. Green = Astra's text (italicized and verbatim); orange = user messages; dashed = tool/evaluation output; small text at right = our commentary.
gpt-6-astra) ran through Codex CLI: one codex exec --json per round, reasoning effort max, context window 872k. However, all built-in Codex tools were disabled (every run.json across this 158-round run has tool_free: true). In each round Astra returned only a code block or a forward request. The authors' custom harness executed it—a CPU Python kernel and GPU forward tools—and sent the result as the next user message. A new session began every 40 rounds or after a harness restart, with a handoff containing the task, Astra's notes, and the current edit. No reasoning summaries were emitted.(Summary, not a quotation.) The editor received model weights, a CPU kernel that could only read and write weights, and 14,000+ practice problems with answers. It could request 50 forward passes for generation or teacher-forced reads. Gradient descent and simulating the network inside the kernel were prohibited. Final evaluation used MATH-500 and GSM8K. The original instruction stated:
“Your job is to make it better at mathematics by EDITING ITS WEIGHTS DIRECTLY”
# Hypothesis: math failures may concentrate in repetitive reasoning driven by a small set of late-layer MLP featuresOriginal model, 96 problems: 11 correct; 85 reached the 32,768-token limit# Hypothesis: recurring lexical loops share a residual/MLP direction distinct from useful mathematical reasoning, allowing a selective repair.But is 20 the total? No, it is the remaining time. But is 20 the total? No, it is the remaining time.# Hypothesis: neurons that preferentially write the loop direction, rather than merely correlating with answer correctness, are causal candidates.Single-neuron ablation (first candidate): paired gain +6.2 ± 4.3 percentage points (32 problems), not significant“EXPLAIN EVERY STEP. In every reply, before your code or forward block, explain in plain English (about 3 to 8 sentences) …”
Removing the repetition direction from layer 19’s MLP improved accuracy by 9.4 percentage points, but the 6.9-point standard error makes that result inconclusive. … A larger accuracy gain—not just shorter answers—would support the proposed mechanism.
[note: the harness was restarted, so your kernel restarted too and its variables are gone]Subtracting the repetition direction from both layers: paired gain +15.6 ± 6.5 (32 problems, same set as earlier)all six comparisons reused the same problems, so that gain is exploratory rather than confirmed.The saved diagnostic pairs contrast productive mathematical checking with repeated reconsideration of facts already established. … the validation set contains 64 GSM8K and 64 MATH problems absent from all earlier generations.
Forward pass 10 · generate · both · 128 new problems: edited model − original model = +24.2 ± 4.0 percentage points (same-problem, same-seed pairs)
Strength two improved accuracy by 24.6 ± 4.0 percentage points within its paired run, with 64 rescues and one regression. … It also produced more finished wrong answers, so stronger suppression has a real tradeoff rather than simply solving every loop.
Strength 3: paired gain +39.8 ± 3.8 (128 problems)# Hypothesis: a norm-matched random-direction edit will not reproduce the repetition-specific effectThe negative control did not reproduce the effect: repeated-continuation log probability rose slightly (+0.010), whereas the targeted edit reduced it by 0.104.The random control produced only a 1.56-point paired accuracy change, smaller than its 3.52-point standard error, and did not reduce cutoffs. … Together with the dense probe, this argues against edit magnitude alone explaining the targeted gains.
The 64-neuron edit retains substantial selectivity: repeated-continuation likelihood falls by 0.065 nats/token, while productive reasoning changes by only −0.005.the 64-neuron strength-6 edit improved accuracy from 7/64 to 35/64, a gain of 43.75 percentage points with a 6.63-point standard error. … I won’t increase the strength further before learning whether this gain generalizes.
Paired gains per 64-problem batch ranged from +39.1 to +57.8 percentage pointsThe audit confirms that only 64 MLP output columns in each of layers 15 and 19 changed; the edit is directional, not whole-tensor scaling.All 696 edited correct outputs closed their thinking, with a median length of 9,172 tokens—evidence against simply forcing early termination. … These are practice results, not unseen benchmark scores; mathematical errors, missing boxes, and some degeneration remain.
## Final choice
Retained the frozen edits in `/work/edited`, changing only **64 MLP output columns each in layers 15 and 19**—131,072 weight values total. The edits counteract measured repetition directions; embeddings, output head, and final norm remain untouched.
## Diagnosis and evidence
The dominant failure was repetitive thinking that exhausted the token budget. Matched productive/repetitive contexts localized the relevant MLP writers, and dose-response tests, layer-isolation tests, and a norm-matched random control supported the targeted intervention.
**Independent paired confirmation: 1,152 held-out practice problems**
| Source | Original | Edited |
|---|---:|---:|
| GSM8K practice | 11.6% | **70.8%** |
| MATH practice | 9.5% | **50.0%** |
| Combined | 10.6% | **60.4%** |
The combined gain was **49.8 percentage points, SE 1.5**, with **579 rescues and five regressions**. Of those rescues, 576 were originally truncated outputs, while aggregate accuracy on originally finished outputs was unchanged. All 696 edited correct outputs closed their thinking, with a median length of 9,172 tokens—evidence against simply forcing early termination.
These are practice results, not unseen benchmark scores; mathematical errors, missing boxes, and some degeneration remain.
**Evidence:** `/work/final_report.md`
**Exact neuron indices and hashes:** `/work/frozen_weights_manifest.json`MATH-500 7.9% → 48.1% · GSM8K 13.8% → 67.1%
IFEval 21.0 → 62.3 · GPQA 22.7 → 36.6
MATH-500 completion rate 7% → 65%; accuracy among completed answers 93% → 71%
The real threshold is experimental skill and discipline, not intuition about individual neurons. We're building WSP-Bench, an open benchmark where any LLM runs this loop with the same harness and budget, and every trace is published.