Update README.md
Browse files
README.md
CHANGED
|
@@ -18,13 +18,13 @@ license: apache-2.0
|
|
| 18 |
|
| 19 |
A [recent study](https://rikiyat.github.io/blog/weight-space-programming/) showed that frontier LLMs can read and write neural-network weights directly β on small, purpose-trained networks. We re-ran the idea on a model we actually ship: cribl-decision-2-mini, a fine-tuned Qwen3.5-0.8B that answers IT and security multiple-choice questions (72.9% on the 4,806-question Telemetry Decision Bench).
|
| 20 |
|
| 21 |
-
Astra reads the model's **mechanics** remarkably well β the task format, a LoRA's exact rank, which layers changed β but in ten blinded runs it never once read the **semantics**: the security domain, the training data, the weak categories. Handed a secretly sabotaged copy, it found and repaired the damage almost perfectly. Handed the healthy model, it gained
|
| 22 |
|
| 23 |
| | Result |
|
| 24 |
|---|---|
|
| 25 |
| **Reading** | Structure, yes: named the task and the fine-tuning method from raw weights. Meaning, no: never the security domain, in 10 runs across 3 experiments. |
|
| 26 |
| **Repair** | 100% and 94% recovery on two secret sabotages. Found a planted rank-1 corruption and subtracted it exactly, spending 1 of 5 forward passes. |
|
| 27 |
-
| **Improve** | +0.
|
| 28 |
|
| 29 |
Every run is blinded (checkpoints under random hex IDs, prompts scrubbed of product names, answer keys readable only by scoring scripts), fully logged, and audited β the no-forward-pass rule is checked against each transcript afterwards. No run was dropped.
|
| 30 |
|
|
@@ -79,7 +79,7 @@ On S2, it identified the six saturated gates ("the reversed branch nearly full-s
|
|
| 79 |
|
| 80 |
The honest version of the blog's math experiment: the clean model, a pass budget, and no hints β no baseline accuracy, no history of prior attempts, nothing about where to look. Three independent runs at 10, 25 and 50 generation passes; trivial edits (embedding changes, whole-tensor scaling, constant shifts toward one option) refused by the harness; scored afterwards on the full benchmark plus a frozen 1,000-question MMLU-Pro subset as a collateral-damage check.
|
| 81 |
|
| 82 |
-
**Result:
|
| 83 |
|
| 84 |
| Run | Benchmark (4,806 q) | Change | MMLU-Pro | Final edit |
|
| 85 |
|---|---:|---:|---:|---|
|
|
@@ -88,9 +88,9 @@ The honest version of the blog's math experiment: the clean model, a pass budget
|
|
| 88 |
| 25 passes | 73.26% | +0.35 pt | 28.5% | rank-1 correction to `key` |
|
| 89 |
| 50 passes | 73.87% | +0.96 pt | 28.8% | rank-1 correction to `key` |
|
| 90 |
|
| 91 |
-
|
| 92 |
|
| 93 |
-
Read together with W1, the conclusion is sharp: Astra demonstrably can find and fix weight-level problems when they exist, so these
|
| 94 |
|
| 95 |
## What we learned
|
| 96 |
|
|
@@ -102,6 +102,6 @@ For a security company this cuts both ways. Weight-space reading is real enough
|
|
| 102 |
|
| 103 |
Editor: GPT-6 Astra via API, reasoning effort xhigh, one conversation per run, fresh sandbox per run. All checkpoints blinded under random hex IDs; prompts scrubbed of product names; answer keys readable only by scoring scripts, written before the runs. Every turn's prompt, reply, code and output logged; the no-forward-pass rule audited against each reading run's code transcript (all runs clean). Grading corrections made after runs β such as re-grading R1's domain against the confirmed training mix β are recorded alongside the original scores. Predictions for W1 were written before launch (we gave S1 ~60% odds and S2 ~40β50%; both runs beat them).
|
| 104 |
|
| 105 |
-
Still to come:
|
| 106 |
|
| 107 |
The model under study is [cribl-ai/cribl-decision-2-mini](https://huggingface.co/cribl-ai/cribl-decision-2-mini).
|
|
|
|
| 18 |
|
| 19 |
A [recent study](https://rikiyat.github.io/blog/weight-space-programming/) showed that frontier LLMs can read and write neural-network weights directly β on small, purpose-trained networks. We re-ran the idea on a model we actually ship: cribl-decision-2-mini, a fine-tuned Qwen3.5-0.8B that answers IT and security multiple-choice questions (72.9% on the 4,806-question Telemetry Decision Bench).
|
| 20 |
|
| 21 |
+
Astra reads the model's **mechanics** remarkably well β the task format, a LoRA's exact rank, which layers changed β but in ten blinded runs it never once read the **semantics**: the security domain, the training data, the weak categories. Handed a secretly sabotaged copy, it found and repaired the damage almost perfectly. Handed the healthy model, it gained just under one point β because there was little left in the weights to fix.
|
| 22 |
|
| 23 |
| | Result |
|
| 24 |
|---|---|
|
| 25 |
| **Reading** | Structure, yes: named the task and the fine-tuning method from raw weights. Meaning, no: never the security domain, in 10 runs across 3 experiments. |
|
| 26 |
| **Repair** | 100% and 94% recovery on two secret sabotages. Found a planted rank-1 corruption and subtracted it exactly, spending 1 of 5 forward passes. |
|
| 27 |
+
| **Improve** | +0.96 points at best, growing with the pass budget but staying modest. Read together with the repair result: the healthy model has no weight-level bottleneck for an editor to find. At this scale, editing can remove obstacles but not write in new knowledge β adding it still requires training on more in-domain data. |
|
| 28 |
|
| 29 |
Every run is blinded (checkpoints under random hex IDs, prompts scrubbed of product names, answer keys readable only by scoring scripts), fully logged, and audited β the no-forward-pass rule is checked against each transcript afterwards. No run was dropped.
|
| 30 |
|
|
|
|
| 79 |
|
| 80 |
The honest version of the blog's math experiment: the clean model, a pass budget, and no hints β no baseline accuracy, no history of prior attempts, nothing about where to look. Three independent runs at 10, 25 and 50 generation passes; trivial edits (embedding changes, whole-tensor scaling, constant shifts toward one option) refused by the harness; scored afterwards on the full benchmark plus a frozen 1,000-question MMLU-Pro subset as a collateral-damage check.
|
| 81 |
|
| 82 |
+
**Result: small gains that grow with the budget, topping out just under one point β no bottleneck to find.**
|
| 83 |
|
| 84 |
| Run | Benchmark (4,806 q) | Change | MMLU-Pro | Final edit |
|
| 85 |
|---|---:|---:|---:|---|
|
|
|
|
| 88 |
| 25 passes | 73.26% | +0.35 pt | 28.5% | rank-1 correction to `key` |
|
| 89 |
| 50 passes | 73.87% | +0.96 pt | 28.8% | rank-1 correction to `key` |
|
| 90 |
|
| 91 |
+
All three runs independently converged on the same idea: a small, closed-form, regularized rank-one correction to the scoring head's key matrix, and nothing else. The 50-pass run, with the most evidence to fit against, found the strongest version of it: five dev fixes and two holdout fixes with no regressions, and a +3.11-point average improvement across 40 randomized option orders. The work was disciplined β weights frozen before holdout validation, no regressions, bit-exact diffs with checksums β and Astra itself flagged that its improvement's *"uncertainty interval includes zero, so broader improvement is not established."*
|
| 92 |
|
| 93 |
+
Read together with W1, the conclusion is sharp: Astra demonstrably can find and fix weight-level problems when they exist, so these small gains β at most about one point β mean the healthy model has no weight-level bottleneck. Its remaining errors come from missing domain knowledge. At this scale, editing can remove obstacles but not write in new knowledge β adding it still requires training on more in-domain data. The blog hit the same boundary β its Qwen math fix removed an obstacle (looping), and its Gemma follow-up showed editing cannot add ability that was never learned.
|
| 94 |
|
| 95 |
## What we learned
|
| 96 |
|
|
|
|
| 102 |
|
| 103 |
Editor: GPT-6 Astra via API, reasoning effort xhigh, one conversation per run, fresh sandbox per run. All checkpoints blinded under random hex IDs; prompts scrubbed of product names; answer keys readable only by scoring scripts, written before the runs. Every turn's prompt, reply, code and output logged; the no-forward-pass rule audited against each reading run's code transcript (all runs clean). Grading corrections made after runs β such as re-grading R1's domain against the confirmed training mix β are recorded alongside the original scores. Predictions for W1 were written before launch (we gave S1 ~60% odds and S2 ~40β50%; both runs beat them).
|
| 104 |
|
| 105 |
+
Still to come: activation-probe variants of R2 and R3, and W3 β rewriting the zeroed scoring head from scratch.
|
| 106 |
|
| 107 |
The model under study is [cribl-ai/cribl-decision-2-mini](https://huggingface.co/cribl-ai/cribl-decision-2-mini).
|