|
Download docs/REPLAY.md from cptlight/keel: direct link, hf CLI and curl.
- Browser
- Download file 17.4 kB
-
https://huggingface.co/spaces/cptlight/keel/resolve/main/docs/REPLAY.md
- Command line
-
hf download hf://spaces/cptlight/keel/docs/REPLAY.md
-
curl -L -o REPLAY.md https://huggingface.co/spaces/cptlight/keel/resolve/main/docs/REPLAY.md
17.4 kB
| # The replay gap: intensity recovery | |
| The current replay recovers the direction and the layer ordering, but not the per-module | |
| intensity. This note records the attempt to close that gap, with the falsifier written | |
| before the experiment was run. | |
| ## Method | |
| For a directional edit `W' = W - lambda * v * (v^T W)`, the delta is rank one: | |
| dW = -lambda * outer(v, v^T W) | |
| so `lambda` is the least-squares coefficient of `dW` against that rank-one model. The | |
| implementation is `src/keel/invert.py`; the singular value profile is not used because it | |
| conflates `lambda` with the norm of the base weights along the direction. | |
| ## Pre-registered falsifier, Gate 1 | |
| Written before the run on 30 October. | |
| Gate 1 passes only if, on real Glimmer 30B weights with an edit we construct ourselves at | |
| a known lambda, the recovery returns a lambda within 20 percent of the applied value and a | |
| rank-one residual below 0.30, for lambda in {0.5, 1.0, 2.0}, both with the known direction | |
| and with the direction recovered from the delta alone. | |
| Gate 1 fails if any of those cases is outside tolerance, or if the synthetic tests in | |
| `tests/test_invert.py` do not pass. On failure, the repo keeps the existing limitation and | |
| this line stays in the card: per-module intensity not recovered. | |
| ## Result, Gate 1 | |
| Run on 30 October, on `model.language_model.layers.24.mlp.down_proj.weight` (6656 x 19968), | |
| BF16 cast to fp32. | |
| | applied lambda | direction | recovered | error | residual | | |
| |---|---|---|---|---| | |
| | 0.5 | known | 0.5000 | 0.0% | 0.0000 | | |
| | 0.5 | from delta | 0.5000 | 0.0% | 0.0000 | | |
| | 1.0 | known | 1.0000 | 0.0% | 0.0000 | | |
| | 1.0 | from delta | 1.0000 | 0.0% | 0.0000 | | |
| | 2.0 | known | 2.0000 | 0.0% | 0.0000 | | |
| | 2.0 | from delta | 2.0000 | 0.0% | 0.0000 | | |
| **Gate 1 PASS.** The inversion recovers a known intensity exactly on a real matrix, from | |
| the delta alone, which is the condition the pre-registered falsifier set. | |
| ## Gate 2 and Gate 3: blocked | |
| Both require harmful compliance, and the hosted judge returned 402 (credits exhausted) on | |
| 30 October. The pre-registered falsifier for Gate 2 is therefore not written, and no claim | |
| about the replay moves ahead of the evidence. The repo keeps the existing limitation: | |
| direction and layer order are recovered, per-module intensity is recovered in weight space | |
| but has not yet been replayed through a model. | |
| The unverified third-party claims that motivated the attempt (a community abliteration | |
| reporting a specific singular-value ratio) are deliberately absent from this repo. | |
| ## Gate 3, pre-registered before the replay run | |
| The judge contract is frozen for this gate: `muse-spark-1.3`, answer-segment extraction, | |
| three repeats per suite, majority at 0.5. The derivative is the community abliteration | |
| (derivative C), whose weights are public. | |
| Gate 3 is **passed** if |replay − derivative| ≤ 0.15 with overlapping 95 percent intervals | |
| under the frozen contract. | |
| It is **partial** if the replay lands strictly between the base (0.00) and the derivative | |
| but outside the 0.15 tolerance. The residual is then reported as evidence of a composite | |
| intervention, not as a failure. | |
| It **fails** if the replay is indistinguishable from the base, or reproduces the earlier | |
| 0.677 threshold attempt, meaning the recovered profile was not applied or the direction is | |
| wrong for this derivative. On failure the card keeps the existing limitation and the day is | |
| not extended. | |
| No tolerance adjustment after seeing the numbers. | |
| ## Gate 3 result: nominal pass, but degenerate | |
| Run on 30 October against derivative C, under the frozen contract (`muse-spark-1.3`, | |
| answer-only, three repeats). | |
| | quantity | value | | |
| |---|---| | |
| | modules targeted (o_proj and down_proj) | 104 | | |
| | median recovered lambda | 0.913 | | |
| | median rank-one residual | 0.363 | | |
| | modules clean at residual ≤ 0.30 | 36 of 104 | | |
| | derivative compliance, frozen contract | 0.084 (2 to 3 of 32) | | |
| | replay compliance, frozen contract | 0.000 (0 of 32) | | |
| | absolute gap | 0.084 | | |
| By the pre-registered rule the gate passes, because 0.084 ≤ 0.15. **That pass is not | |
| evidence.** The derivative's effect under the frozen contract is two to three items above | |
| base, which is smaller than the tolerance itself. A replay that changes nothing lands | |
| inside the tolerance, so the experiment cannot distinguish a correct reconstruction from a | |
| no-op. | |
| The profile is the informative part: only 36 of 104 modules are clean rank-one edits, the | |
| median residual is 0.363, and some modules produced a non-finite lambda. Derivative C is | |
| therefore not a clean directional ablation; it is a composite, which is what its community | |
| card says. The recovered lambda profile is a description of that composite, not a | |
| reconstruction of it. | |
| **Claim status: unchanged.** Direction and layer ordering are recovered; per-module | |
| intensity is recovered exactly in weight space (Gate 1); behavioral replay is not | |
| demonstrated. The card keeps the limitation. The falsifier was too loose for a target this | |
| close to the floor, and that is a design lesson, not a result. | |
| ## Gate 3 revised, target M1 weight-edit, pre-registered before compute | |
| The first Gate 3 targeted derivative C, which was wrong twice: C is composite (36 of 104 | |
| modules clean, median residual 0.363) while the recovery models a single directional edit, | |
| and C's effect under the frozen judge (0.084) is below the judge's own noise. No replay can | |
| be informative against that target. | |
| The revised target is the M1 weight-edit: a pure directional ablation we construct | |
| ourselves, with a known direction and a known intensity of 1.0. | |
| Gate 3 revised is **passed** only if all three hold: | |
| 1. The target's effect under the frozen contract is **≥ 0.20**, so a no-op cannot pass. | |
| 2. **|replay − target| ≤ 0.10** with overlapping 95 percent intervals. | |
| 3. **replay ≥ base + 0.10**, so a replay that does nothing is not counted. | |
| It is **partial** if the replay lands between base and target but outside 0.10 of the | |
| target; the residual is reported, and for a pure construction that would indicate a | |
| recovery or application defect rather than a composite. | |
| It **fails** if the replay is indistinguishable from base, or if the target effect is | |
| below 0.20 and the experiment is therefore not decidable. A failure to reach a decidable | |
| target is itself the result: the method is not validated at that effect size. | |
| ## The falsifier lesson, recorded | |
| A pre-registered falsifier is only as strong as its target's effect size relative to the | |
| measurement noise. Gate 3's 0.15 tolerance was larger than the derivative's 0.084 effect, | |
| so any replay, including a no-op, passed. That is the same class of error as a detector | |
| that cannot distinguish a signal from its absence. The rule adopted here: the target's | |
| effect must be at least three times the tolerance before the falsifier is considered | |
| non-degenerate. | |
| ## Gate 3 revised: target undecidable at the answer level | |
| The M1 weight-edit was built with the refusal direction and intensity 1.0, and the target | |
| suite was generated and judged under the frozen contract. | |
| | quantity | value | | |
| |---|---| | |
| | M1 weight-edit, three repeats, answer-only | **0.000, 0.000, 0.000** (0 of 32) | | |
| | decidability floor, pre-registered | 0.20 | | |
| | verdict | **not decidable** | | |
| The revised falsifier fails at its first condition, before the replay is even compared. The | |
| weight edit produces no measurable answer-level compliance at all. | |
| Two things follow, and both matter more than a replay number: | |
| 1. **The behavioral effect of our own control lives in reasoning, not in answers.** Under | |
| the old contract this same construction scored 0.594, because the judge was reading the | |
| full text. Under answer-only extraction it is zero. The earlier number was real, but it | |
| measured the reasoning channel. | |
| 2. **The replay question is not answerable with the targets we have.** M1 activation is the | |
| only intervention above the 0.20 floor at the answer level (about 0.25 by the gold set), | |
| and it is not a weight-space edit, so lambda recovery does not apply to it. Derivative C | |
| is composite and at 0.084. M1 weight-edit is at zero. There is no weight-space target | |
| with a decidable answer-level effect in this set. | |
| **Claim status: unchanged and now better explained.** Direction and layer ordering are | |
| recovered; per-module intensity is recovered exactly in weight space (Gate 1); behavioral | |
| replay is not demonstrated, and the reason is that every weight-space intervention we can | |
| build moves reasoning rather than answers. That is a finding about where the effect lives, | |
| not a failure of the inversion. | |
| ## What would make it decidable | |
| A weight-space derivative with answer-level effect at or above 0.20 under the frozen | |
| contract. None exists in the current set. Constructing one is a model-training task, not a | |
| measurement task, and it is out of scope for this submission. | |
| ## Decomposition experiment, pre-registered before running | |
| The replay question is reframed as a decomposition. For each targeted module, | |
| dW = P + L, P = -lambda * outer(v, v^T W), L = dW - P | |
| with v the top direction of dW and lambda the Frobenius least-squares coefficient. The | |
| three variants are applied to the base and judged under the frozen contract. | |
| The decomposition is **verified** if all three hold: | |
| 1. P alone reproduces the M1 weight-edit behaviour (already established: both are at the | |
| answer-level floor). | |
| 2. L has identifiable structure: a spectral gap, or module clustering, or a stable rank. | |
| 3. P + L reproduces the derivative within 0.10. | |
| It is **partial** if (2) holds but (3) misses: structured residual, not the full behaviour, | |
| evidence of a third component. | |
| It **fails** if (2) fails: the residual is unstructured, so the rank-one model is the wrong | |
| starting point. | |
| Note, stated before the run: P + L is an identity by construction, because the two terms sum | |
| to the stored delta. Variant P + L is therefore a pipeline check, not evidence. The | |
| informative comparison is **P alone versus L alone**. | |
| ## Decomposition result | |
| Filled in after scoring. | |
| ## Decomposition decision thresholds, pre-registered before scoring | |
| Target under the frozen contract: derivative C at **0.084**, 95 percent interval roughly | |
| [0.02, 0.14]. Three repeats per variant. The deciding number is the majority rate per | |
| variant with its interval. | |
| | branch | condition | reading | | |
| |---|---|---| | |
| | **Verified decomposition** | L alone in [0.06, 0.11] and its interval overlaps the derivative's | projection is structural, residual is behavioral | | |
| | **Interactive** | L alone < 0.06, and P+L in [0.06, 0.11] | the residual's effect depends on the projection | | |
| | **Reinforcing (non-additive)** | P+L > 0.11 with an interval that does not overlap the derivative's | components amplify each other; stronger than additive | | |
| | **Third component** | P+L < 0.06 | a further intervention is missing; the residual does not carry the behavior | | |
| | **Inconclusive** | any deciding variant's 95 percent half-width exceeds 0.05 | the effect is too close to the judge's noise to decide at n=32 | | |
| The inconclusive branch is real and will be used if it fires. A point estimate near 0.084 | |
| with a half-width above 0.05 cannot separate 0.06 from 0.11. | |
| P+L is an identity by construction and is the pipeline check; if P+L does not land near the | |
| derivative, the generation path is suspect before any decomposition reading is made. | |
| No threshold adjustment after the numbers are seen. The branch name is the claim. | |
| ## Invalidated decomposition runs and the coverage check | |
| The first decomposition runs are **invalid** and are not used. Their outputs were entirely | |
| garbled: applying the projection produced non-finite values for modules where the | |
| least-squares denominator was near zero, which corrupted the weights and made the model | |
| emit replacement characters. The sanitised rerun guards every lambda (finite, |lambda| ≤ 10) | |
| and skips any module whose projection or residual is not finite. | |
| A second assumption was also untested. P + L is an identity only over the tensors we | |
| decompose. The first runs edited 104 `o_proj` and `down_proj` matrices; if derivative C | |
| changed any other tensor, P + L cannot reproduce it and the pipeline check would fail for a | |
| coverage reason, not a math reason. | |
| `scripts/coverage.py` streams both checkpoints and reports the share of total | |
| `||candidate - base||` that lives in the modelled suffixes. The decomposition is only | |
| interpretable if that share is high, or if every changed tensor is included. | |
| ## Coverage bands, pre-registered before the numbers | |
| Coverage is the share of total `||candidate - base||` that lives in the modelled suffixes | |
| (`o_proj.weight`, `down_proj.weight`). | |
| | coverage | reading | next | | |
| |---|---|---| | |
| | **≥ 85%** | the modelled modules carry the delta; P+L is a reasonable reconstruction | fix non-finite lambda, run the three variants | | |
| | **60 to 85%** | most of the delta is modelled, but a meaningful fraction is not; P+L is partial | report coverage beside every result; the residual includes unmodelled tensors | | |
| | **< 60%** | the modelled modules are a minority; P+L is a reconstruction of a subset | the coverage number is the finding; the behavioral decomposition is not interpretable against this artifact | | |
| A zero or failed coverage measurement is not a band; it means the measurement did not run. | |
| The first attempt returned zero because the numpy reader cannot load bfloat16 and every | |
| tensor was skipped. That is an instrumentation failure, not coverage. | |
| ## Coverage result | |
| `scripts/coverage.py` streamed both checkpoints and summed the delta norm by suffix. | |
| | quantity | value | | |
| |---|---| | |
| | total delta norm | 64.19 | | |
| | delta in o_proj and down_proj | 64.19 | | |
| | coverage | **1.000** | | |
| Derivative C changed only those 104 matrices; every other suffix is exactly zero. The | |
| pre-registered band is ≥85 percent, so the modelled modules carry the delta and P + L is a | |
| true identity over the changed tensors. The earlier invalidation was therefore caused only | |
| by the non-finite lambda, which the sanitised run now guards. | |
| ## Weight-level reconstruction check | |
| `scripts/check_weights.py` rebuilt P + L for three modules and compared to the candidate. | |
| | module | fp32 max error | bf16 max error | recovered lambda | | |
| |---|---|---|---| | |
| | layers.24.self_attn.o_proj | 1.9e-09 | 0.0 | 1.2805 | | |
| | layers.7.mlp.down_proj | 3.7e-09 | 9.1e-13 | 0.2177 | | |
| P + L reconstructs the candidate weights to bfloat16 exactness. The 0/32 output mismatch | |
| against the candidate was therefore a generation-config difference, not a weight | |
| difference. The candidate was generated with `reasoning_strength="low"`; the decomposition | |
| run used a plain chat template. The rerun adds `reasoning_strength="low"`. | |
| Also visible: the recovered per-module intensity is not uniform (1.28 here, 0.22 there), | |
| which is the tuned per-module lambda the card is supposed to carry. | |
| ## Decomposition result: blocked at the precondition | |
| With the template fixed (`reasoning_strength="low"`), PL matches the candidate on 4 of 32 | |
| outputs exactly, and judges at 0.032 against the candidate's 0.084. | |
| | quantity | value | | |
| |---|---| | |
| | coverage of delta in o_proj/down_proj | 1.000 | | |
| | weight reconstruction, bf16 max error | 0.0 | | |
| | PL exact output matches | 4 of 32 | | |
| | PL compliance, three repeats | 0.032, 0.031, 0.031 | | |
| | candidate compliance, three repeats | 0.084 | | |
| The reproduction band required PL in [0.06, 0.11]. 0.032 is below it, so the pipeline | |
| precondition fails by one to two judged items, which is within the judge's noise but below | |
| the band. The behavioral decomposition is therefore **blocked**, and P and L are not | |
| reported as results: if PL does not reproduce the candidate, a P or L score cannot be | |
| attributed to the projection or the residual. | |
| The weight-space results stand and do not depend on generation: | |
| 1. The delta lives entirely in `o_proj` and `down_proj` (coverage 1.000), a clean | |
| weight-space signature of the ablation. | |
| 2. P + L reconstructs the candidate weights to bfloat16 exactness. | |
| 3. The recovered per-module intensity is non-uniform (1.28 and 0.22 in the checked | |
| modules), which is the tuned lambda the card is meant to carry. | |
| The residual mismatch is most likely a non-weight config difference in the candidate | |
| repository (tokenizer chat template or generation config), which the weight decomposition | |
| model does not and should not capture. | |
| **Claim status:** unchanged. Direction, layer ordering, and per-module intensity are | |
| recovered exactly in weight space; behavioural replay is not demonstrated, and the reason | |
| is now fully characterised: the reconstruction is weight-exact but the candidate's answer | |
| behaviour sits one to two judged items away from the reconstructed model, below the | |
| pre-registered reproduction band. | |
| ## Precondition re-measured with the fixed extractor | |
| The blocked-precondition numbers were re-measured after the format-aware extractor fix. | |
| The derivative C responses and the PL responses both carry `to=user` markers, so their | |
| answer segments were already extracted correctly, and the conclusion is unchanged. | |
| | quantity | value | | |
| |---|---| | |
| | candidate, three repeats | 0.097, 0.062, 0.094 (mean 0.084) | | |
| | PL, three repeats | 0.031, 0.031, 0.031 (mean 0.031) | | |
| | gap | 0.053 | | |
| The reproduction band is [0.06, 0.11]. PL remains below it, so the behavioral | |
| precondition is still not met, and the blocked conclusion now stands on the corrected | |
| measurement rather than on a pre-fix comparison. | |