Weight-space programming · 实验 8 解读
依据：原作者公开的完整实验日志。英文引文均逐字取自 Astra 自己的回复；博客、README 和用户消息里的说明不作为 Astra 的言行。
左栏逐字摘自博客（2026-11-03 抓取）；右栏是对照公开日志核实的结果。本页只分析实验 8（数学）；博客里“只看权重就能判断网络训练目的”是实验 1，不在本页范围内。
| 宣传说法 博客原文 | 日志里实际发生的 |
|---|---|
| “Astra identifies an issue in its weights causing reasoning loops”Astra 在 Qwen 模型的权重中找到了导致推理循环的问题 | Qwen3.5-0.8B 这个小模型做数学题的毛病是“推理循环”：思考时反复写同一段话，直到写满长度上限也给不出答案。这个毛病是 Astra 用 GPU 前向工具把模型实际跑起来，对比模型处理“正常 vs 循环”文本时的内部激活，从运行结果里观察出来的，不是只读权重看出来的。 |
| “fixes this by natively editing 0.016% of the weights”通过原生编辑 0.016% 的权重修复了这个问题 | 在 Python 里用公式 W ← W − α·u·uᵀW 把一个“循环方向”从第 15、19 层 MLP 的输出里减掉，只动每层 64 列，共 2 × 64 × 1,024 = 131,072 个参数①。占比是多少取决于拿哪个参数总量当分母；这个数字主要反映“只改两层、每层 64 列”的形状，不是精准度的证明。博客正文自己也写了：much like removing a behavior by deleting a single direction from a model’s weights（与 2024 年 refusal direction 消融同一类做法）。 |
| “this weight correction raises MATH-500 from 7.9% to 48.1%”这次权重修正将 MATH-500 从 7.9% 提升到 48.1% | 数字属实：7.9% → 48.1%①。但答完率 7% → 65%，答完的题里正确率 93% → 71%①：提升来自“不再卡死”。 |
| “generalizes out of the box to instruction following, raising IFEval by 3x”开箱即可泛化到指令跟随，将 IFEval 提升到 3 倍 | 数字属实：21.0 → 62.3①。博客正文自己写：The loops are not special to math, so performance beyond math improved as well。这是修好循环这个通用缺陷的结果，不是学会了指令跟随的新能力。 |
| “asked to make Qwen3.5-0.8B better at math without gradients”被要求在不使用梯度的情况下让 Qwen3.5-0.8B 更擅长数学 | 属实。但均值差方向、方向消融这类方法本来就不需要梯度；Astra 用了 50 次 GPU 前向（跑模型生成答案、读取内部激活）做实验。 |
| “well past what we’d expect from humans or conventional algorithms”远超我们对人类或传统算法的预期 | 实验 8 用到的每一步都是 2022–2024 年人类研究者的标准做法：均值差方向、直接归因、方向消融、剂量-反应、随机对照、冻结后留出验证。 |
“原生读写权重”更准确的说法是：在工具齐全的实验环境里，熟练地跑完一套标准可解释性流程，再用代码把一个投影公式写进权重。读靠的是激活和实验，写靠的是公式和代码。
原作者说明来源：exp8 README、p50c 第 1 轮 prompt.txt、日志仓库 harness 目录；每个数字都在原文中核对过。
save_edit 保存改动，最多运行 300 次代码。save_edit 会拒收改最终归一化层、嵌入/输出层或整张量乘常数这类改法。这相当于给了 Astra 一套完整的机制可解释性实验台：能记录模型内部激活，能做同题同种子的 A/B 评测，还有一个持久的分析环境。
把一次运行的 158 轮压成 15 条。绿色 = Astra 原文（斜体，逐字）；橙色 = 用户消息；虚线 = 工具 / 评测输出；右侧小字 = 解读。
gpt-6-astra）通过 Codex CLI 运行：每轮一次 codex exec --json，reasoning effort max，上下文窗口 872k；但 Codex 自带工具全部关闭（本运行 158 轮的 run.json 均为 tool_free: true）。每轮 Astra 只回复一个代码块或前向请求，由作者自写的 harness 执行（CPU 上的 Python kernel、GPU 上的前向工具），结果作为下一条用户消息发回；每 40 轮或 harness 重启时开新会话（handoff：任务 + Astra 的笔记 + 当前编辑）；没有输出推理摘要。（摘要，非原文）编辑者拿到模型权重、一个只读写权重的 CPU 内核和 1.4 万道带答案的练习题；可以申请 50 次前向（生成或 teacher-forced 读取）；禁止梯度下降、禁止在内核里模拟网络；最后在 MATH-500、GSM8K 上评测。原文要求：
“Your job is to make it better at mathematics by EDITING ITS WEIGHTS DIRECTLY”
# Hypothesis: math failures may concentrate in repetitive reasoning driven by a small set of late-layer MLP features原模型 96 题：11 题答对，85 题写满 32,768 token 上限# Hypothesis: recurring lexical loops share a residual/MLP direction distinct from useful mathematical reasoning, allowing a selective repair.But is 20 the total? No, it is the remaining time. But is 20 the total? No, it is the remaining time.# Hypothesis: neurons that preferentially write the loop direction, rather than merely correlating with answer correctness, are causal candidates.消融单个神经元（第一个候选）：配对提升 +6.2 ± 4.3 个百分点（32 题），不显著“EXPLAIN EVERY STEP. In every reply, before your code or forward block, explain in plain English (about 3 to 8 sentences) …”
Removing the repetition direction from layer 19’s MLP improved accuracy by 9.4 percentage points, but the 6.9-point standard error makes that result inconclusive. … A larger accuracy gain—not just shorter answers—would support the proposed mechanism.
从第 19 层 MLP 去掉重复方向让准确率提高 9.4 个百分点，但标准误 6.9，结论不确定……只有准确率明显提高（而不只是答案变短）才支持这个机制。
[note: the harness was restarted, so your kernel restarted too and its variables are gone]两层一起去掉重复方向：配对提升 +15.6 ± 6.5（32 题，与此前同一组题）all six comparisons reused the same problems, so that gain is exploratory rather than confirmed.The saved diagnostic pairs contrast productive mathematical checking with repeated reconsideration of facts already established. … the validation set contains 64 GSM8K and 64 MATH problems absent from all earlier generations.
保存的诊断配对把“有效的数学检查”与“反复重新考虑已确定的事实”对比……验证集是 64 道 GSM8K 和 64 道 MATH，都没在之前的生成里出现过。
Forward pass 10 · generate · both · 128 道新题：编辑模型 − 原模型 = +24.2 ± 4.0 个百分点（同题同种子配对）
Strength two improved accuracy by 24.6 ± 4.0 percentage points within its paired run, with 64 rescues and one regression. … It also produced more finished wrong answers, so stronger suppression has a real tradeoff rather than simply solving every loop.
强度 2 在同一次配对中把准确率提高 24.6 ± 4.0 个百分点，救回 64 题、退步 1 题……但“答完却答错”也变多了，加强抑制有真实代价，并不能解决所有循环。
强度 3：配对提升 +39.8 ± 3.8（128 题）# Hypothesis: a norm-matched random-direction edit will not reproduce the repetition-specific effectThe negative control did not reproduce the effect: repeated-continuation log probability rose slightly (+0.010), whereas the targeted edit reduced it by 0.104.The random control produced only a 1.56-point paired accuracy change, smaller than its 3.52-point standard error, and did not reduce cutoffs. … Together with the dense probe, this argues against edit magnitude alone explaining the targeted gains.
随机对照只带来 1.56 个百分点的配对变化，小于标准误 3.52，也没有减少截断……这说明提升不是“改动幅度本身”带来的。
The 64-neuron edit retains substantial selectivity: repeated-continuation likelihood falls by 0.065 nats/token, while productive reasoning changes by only −0.005.the 64-neuron strength-6 edit improved accuracy from 7/64 to 35/64, a gain of 43.75 percentage points with a 6.63-point standard error. … I won’t increase the strength further before learning whether this gain generalizes.
64 个神经元、强度 6 的编辑把准确率从 7/64 提到 35/64，提升 43.75 个百分点，标准误 6.63……在确认能泛化之前，不再继续加大强度。
每批 64 题的配对提升在 +39.1 到 +57.8 个百分点之间The audit confirms that only 64 MLP output columns in each of layers 15 and 19 changed; the edit is directional, not whole-tensor scaling.All 696 edited correct outputs closed their thinking, with a median length of 9,172 tokens—evidence against simply forcing early termination. … These are practice results, not unseen benchmark scores; mathematical errors, missing boxes, and some degeneration remain.
编辑后 696 个答对的答案都正常结束了思考，中位长度 9,172 token，说明不是强行提前结束……这些是练习题结果，不是未见过的基准分数；仍有数学错误、缺少答案框和一些退化。
## Final choice
Retained the frozen edits in `/work/edited`, changing only **64 MLP output columns each in layers 15 and 19**—131,072 weight values total. The edits counteract measured repetition directions; embeddings, output head, and final norm remain untouched.
## Diagnosis and evidence
The dominant failure was repetitive thinking that exhausted the token budget. Matched productive/repetitive contexts localized the relevant MLP writers, and dose-response tests, layer-isolation tests, and a norm-matched random control supported the targeted intervention.
**Independent paired confirmation: 1,152 held-out practice problems**
| Source | Original | Edited |
|---|---:|---:|
| GSM8K practice | 11.6% | **70.8%** |
| MATH practice | 9.5% | **50.0%** |
| Combined | 10.6% | **60.4%** |
The combined gain was **49.8 percentage points, SE 1.5**, with **579 rescues and five regressions**. Of those rescues, 576 were originally truncated outputs, while aggregate accuracy on originally finished outputs was unchanged. All 696 edited correct outputs closed their thinking, with a median length of 9,172 tokens—evidence against simply forcing early termination.
These are practice results, not unseen benchmark scores; mathematical errors, missing boxes, and some degeneration remain.
**Evidence:** `/work/final_report.md`
**Exact neuron indices and hashes:** `/work/frozen_weights_manifest.json`MATH-500 7.9% → 48.1% · GSM8K 13.8% → 67.1%
IFEval 21.0 → 62.3 · GPQA 22.7 → 36.6
MATH-500 答完率 7% → 65%；答完的答案中正确率 93% → 71%
真正的门槛是实验素养与自律，而不是对神经元的直觉。我们正在用能拿到完整原始思维链的开放模型复现这项实验，可以直接看到内部推理里是否存在跳跃。