luisbrel
/

weight-skills

@@ -1,203 +0,0 @@
-# Addendum v4 — Measured Evidence Programme (2026-11-13)
-**Applies to:** *Zero-Skills in Context: Sub-Second Algebraic Compilation of Agent Procedures into
-Feed-Forward Weights* — Zenodo DOI `10.5281/zenodo.24122804` (v1 version DOI; concept DOI `10.5281/zenodo.24122803`; v3 record
-`10.5281/zenodo.24142869`).
-**Status:** measured results, positive and negative, with sample sizes, intervals and provenance.
-**Hardware:** Apple M4 Max, 128 GB unified memory; MLX 0.32.2 / mlx-lm 0.31.3 (qwen2-0.5B) and
-MLX 0.32.4 / mlx-vlm 0.7.12 (occamy 35B MoE). Greedy decoding. GPU jobs run serially.
----
-## 1. Why this addendum exists
-Version 3 corrected the manuscript text but **attached the uncorrected v1 PDF** (the v3
-`weight_space_skill_compilation_paper.pdf` is the v1 PDF (md5 `10151100b3b7afaa4e879223b7ac4733`; verified on 2026-11-13 against the Zenodo v3 file listing https://zenodo.org/records/24142869 and against the v1 PDF blob committed at `5ef2672`),
-and the v3 LaTeX still asserted several withdrawn claims in its body (microsecond mounting, +48–76 %
-decode gains, 100 % competition accuracy, 100 % compiled guardrail interception). This addendum, with
-the v4 manuscript and a **rebuilt PDF**, replaces those assertions with measured evidence.
-*Provenance: `reports/CLAIMS_AUDIT_2026-11-11.json`, `reports/LOOP_BATCH3_AUDIT.json` @ commit `878fa83`, ledger —; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md`.*
-Between 2026-11-11 and 2026-11-13 a separate, **pre-registered** measurement programme re-tested the
-claims on this stack. Every number below comes from a committed JSON artefact in the research
-repository, named next to the claim with its ledger item. Pre-registrations were frozen and committed
-**before generation** (`reports/LOOP_PREREG_2026-11-13.md`, `reports/LOOP_PREREG_2_2026-11-13.md`,
-`reports/FOLLOWUPS_PREREG_3.md`); the batch-3 results were independently re-audited from raw rows
-(`reports/LOOP_BATCH3_AUDIT.json`, 55 checks, 0 failed).
-*Provenance: `reports/LOOP_BATCH3_AUDIT.json` @ commit `878fa83`, ledger —; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md` / `reports/LOOP_PREREG_2_2026-11-13.md`.*
----
-## 2. Compiled prompt caches can be exact — but only under the aligned contract
-A static prompt (system framework, SOP, skill text) prefilled once and served from a KV cache is
-**sampled, not equivalent**, on the plain path. The aligned-exact contract fixes this: compile the
-prefix up to the cold path's prefill-chunk boundary, re-prefill the remainder with the question, and
-serve the suffix through the same chunked routine as a cold run; a chunk mismatch between compile and
-serve is **refused with a reason**.
-| Setting | Tokens | Identity vs cold | Max abs logit Δ | TTFT ratio | Ledger |
-| :--- | ---: | :--- | ---: | ---: | :--- |
-| occamy / heretic / nemotron / proto, aligned-exact | 2k / 8k / 32k (20/20 cells) | 5/5 bit-identical per model | **0.0** | median 9.8× / 11.6× / 11.8× / 10.2× | G.93 |
-| occamy 35B MoE (mlx-vlm), aligned | 65,536 | round-trip bit-identical | **0.0** | **195.7×** (0.356 s vs 69.76 s) | G.114 |
-| heretic 35B MoE & occamy, aligned, 8k serving | 8,192 | 10/10 identity | **0.0** | 45.4× / 35.2× | G.96 |
-| qwen2-0.5B, 8-bit KV cache, aligned | 2k / 8k | all served rows token-identical | **0.0** | 20.1× / 35.3× | G.115 |
-| qwen2-0.5B, plain compiled w0 (no alignment) | 2k | **not equivalent** | **0.16–0.30** (qwen2; Gemma-4-E4B is 0.72–1.66) | — | G.71/G.102 |
-Plain compiled serving diverges on **64.0 % (heretic), 72.7 % (nemotron), 84.9 % (occamy)** of
-SWE-style cells and only **5.0 %** of long-context cells; the aligned control diverges **0/33 in every
-measured cell** (G.102/G.105/G.113/G.118/G.124/G.128). Peak memory at 65k is **36.94 GB**, inside a
-*Provenance: `reports/SWE_DIVERGENCE_SCALE.json`, `reports/SWE_DIVERGENCE_GEN2.json`, `reports/SWE_DIVERGENCE_GEN2_NEMOTRON.json`, `reports/SWE_DIVERGENCE_GEN2_OCCAMY.json` @ commit `878fa83`, ledger G.102/G.105/G.113/G.118/G.124/G.128; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md`.*
-40 GB budget; the planner selected chunk 2048. The 65k case is impossible without the memory planner:
-a 35B MoE at 65k otherwise needs a **137.44 GB** attention buffer (G.80).
-**Reading.** Equivalence is a property of the *aligned contract*, not of compiled caches in general.
-With the contract, a served cache is bit-identical to cold at every length and KV precision tested.
-*Provenance: `reports/COMPILED_SERVING_ALIGNMENT.json`, `reports/ALIGNED_65K.json`, `reports/ALIGNED_KVQUANT.json`, `reports/CHUNK_EDGE.json`, `reports/SERVING_BENCHMARK.json` @ commit `878fa83`, ledger G.93/G.95/G.96/G.114/G.115; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md` / `reports/LOOP_PREREG_2_2026-11-13.md`.*
----
-## 3. Multi-tenant serving: exactness is tenant-invariant, the speedup is a reuse effect
-| Tenants × prefix × chunk | Cells | Identity | Telemetry | Warm speedup |
-| :--- | ---: | :--- | :--- | :--- |
-| 8/12/16/24/32 × 2k/4k/8k × 1024/2048 | 25 runnable (5 runtime `cut=0` skipped + recorded) | **100 %** (1000/1000 served rows, Δlogit 0.0) | persisted = evictions, reloaded = reload requests, 0 errors | 1.949× @2k/1024; **2.931×** @4k/1024; **5.917×** @8k/1024; 2.787× @8k/2048 |
-*Provenance: `reports/MULTITENANT_ALIGNED_SCALE.json` @ commit `878fa83`, ledger G.125; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md`.*
-At **32 tenants** the warm advantage collapses to **1.001–1.009×**. The row-level data names the
-mechanism: the **first** serve of a tenant's library cache pays a materialisation cost that scales with
-the compiled prefix (**0.069 / 0.200 / 0.622 s** at 2k/4k/8k for chunk 1024), and the same tenant is
-fast on its next access. The warm/cold median therefore tracks **warm accesses per resident tenant**,
-not tenant count: t8–t24 warm medians agree within **≤1.5 ms** for a fixed (prefix, chunk), and the
-32-tenant cells simply have ~0.86 warm accesses per resident cache (G.121, superseded reading in
-G.125). Exactness remains tenant-count-invariant (G.125).
-*Provenance: `reports/MULTITENANT_ALIGNED.json`, `reports/MULTITENANT_ALIGNED_SCALE.json` @ commit `878fa83`, ledger G.121/G.125; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md` / `reports/LOOP_PREREG_2_2026-11-13.md`.*
----
-## 4. Weight-space point edits are single-fact and prompt-specific
-| Construction | Direct ES | Paraphrase ES | Note | Ledger |
-| :--- | ---: | ---: | :--- | :--- |
-| 12-fact summed ROME, one layer | **0/12** | 0/60 | batch interference | G.111 |
-| 12-fact MEMIT, 4 layers | **2/12** | 0/60 | exact solve (closure 0.996) still 2/12 | G.112 |
-| Per-fact ROME, single fact, one layer | **11/12** | 0/60 | first working point edit | G.117 |
-| Per-fact MEMIT | 2/12 | n/a (not scored) | multi-layer distribution is the limit; L4 did not request paraphrase scoring | G.117 |
-| 4-bit push-through of a single-fact ROME ΔW | **0/12** | 0/60 | ΔW 1.58–1.76 % vs 9.657 % noise floor | G.122 |
-| 8-bit push-through of the same ΔW | **11/12** | 0/60 | floor 0.7718 %, ΔW 1.60–1.77 %, ratio 2.08–2.29 | G.126 |
-| Sequential multi-fact refit (12 facts) | **1/12** | 3/60 | newest write wins; each write displaces the previous | G.123 |
-*Provenance: `reports/ROME_4BIT_PUSHTHROUGH.json`, `reports/MEMIT_MULTILAYER.json`, `reports/MEMIT_PER_FACT.json`, `reports/ROME_PER_FACT_4BIT.json`, `reports/ROME_PER_FACT_8BIT.json`, `reports/MEMIT_SEQUENTIAL_REFIT.json` @ commit `878fa83`, ledger G.111/G.112/G.117/G.122/G.126/G.123; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md`.*
-**Reading.** One shared layer can hold **one** fact at a time. 8-bit is the first grid where a
-single-fact edit survives (ratio ≈2.1 above the floor), so the 4-bit null is a **resolution limit, not
-a mechanism limitation**; but the edit remains prompt-specific at every precision, and multi-fact
-insertion is destructive (summed 0/12, MEMIT 2/12, sequential 1/12).
-*Provenance: `reports/ROME_4BIT_PUSHTHROUGH.json`, `reports/MEMIT_MULTILAYER.json`, `reports/MEMIT_PER_FACT.json`, `reports/ROME_PER_FACT_4BIT.json`, `reports/ROME_PER_FACT_8BIT.json`, `reports/MEMIT_SEQUENTIAL_REFIT.json` @ commit `878fa83`, ledger G.111/G.112/G.117/G.122/G.126/G.123; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md` / `reports/LOOP_PREREG_2_2026-11-13.md`.*
----
-## 5. Modular routing outside the weights holds many facts
-| Arm | Construction | Direct ES | NS | Neighbours | Paraphrases | Verdict |
-| :--- | :--- | ---: | ---: | ---: | ---: | :--- |
-| A | all 12 rank-1 modules active at one layer (equivalent to the summed write) | **0/12** | 4/5 | 0/6 | 0/60 | matches G.111 null |
-| B | one module per prompt, dispatched on the frozen subject string only | **11/12** | 5/5 | 6/6 | 0/60 | routing **outside** the weights |
-| C | shared-layer sequential refit (control) | **1/12** | 5/5 | 0/6 | 0/60 | reproduces G.123 |
-Verdict (frozen, G.127): **"weights cannot hold many facts at one shared layer; routing can."**
-The routed arm reproduces the isolated-edit level on direct prompts, but **0/60 paraphrases**: it is a
-cartridge-style dispatch capability whose router is an exact substring match, not a general
-multi-fact weight edit. Any production claim must label it as routing, not as compiled knowledge.
-*Provenance: `reports/MULTIFACT_MODULAR.json` @ commit `878fa83`, ledger G.127; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md` / `reports/LOOP_PREREG_2_2026-11-13.md`.*
----
-## 6. Divergence is model- and generator-dependent (a model × generator interaction)
-| Model | Generator A | Generator B | Fisher p (A vs B) | Aligned control |
-| :--- | ---: | ---: | ---: | ---: |
-| qwen36-35b-heretic | 169/264 = 0.640 | 207/264 = 0.784 | **3.6e-4** (significant) | 0/33 |
-| nemotron-lightning 30B | 192/264 = 0.727 | 199/264 = 0.754 | 0.55 (n.s.) | 0/33 |
-| occamy 35B MoE | 216/264 = 0.818 | 224/264 = 0.849 | 0.41 (n.s.) | 0/33 |
-Under generator B the **model axis persists** (occamy vs nemotron: p = 0.0087), while the
-generator-axis contrast on the same model is significant only on heretic. The generator family is
-therefore a **model × generator interaction**, not a global third axis (G.118/G.124/G.128).
-*Provenance: `reports/SWE_DIVERGENCE_SCALE.json`, `reports/SWE_DIVERGENCE_GEN2.json`, `reports/SWE_DIVERGENCE_GEN2_NEMOTRON.json`, `reports/SWE_DIVERGENCE_GEN2_OCCAMY.json`, `reports/TASK_FAMILY_COMPARISON.json` @ commit `878fa83`, ledger G.102/G.105/G.113/G.118/G.124/G.128; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md` / `reports/LOOP_PREREG_2_2026-11-13.md`.*
----
-## 7. What survives from v1–v3, what is corrected, what is new
-| v1–v3 claim | Status after measurement |
-| :--- | :--- |
-| Exact frozen mounting; 7/7 restoration checks per model; 100/100 mount cycles | **Survives** (independent re-measurement) |
-| Mounting costs 1.2–2.8 ms | **Survives** (microsecond figures withdrawn) |
-| Prefill/TTFT falls 6.1–24.8× and KV cache growth per checkpoint (1.49–50.37×) with a 4,000-token prompt | **Survives** (six checkpoints) |
-| Decode throughput unchanged (0.96–1.03×) | **Survives** (the +48–76 % decode claim is withdrawn) |
-| Deterministic request gate: 24/24 blocked, 0 false positives | **Survives** (rule-based gate, not a weight edit) |
-*Provenance: `reports/MULTI_MODEL_HARDWARE_BENCHMARK_REPORT.json`, `reports/GUARDRAIL_MECHANISM_AUDIT.json`, `reports/REFUSAL_RESTORATION_REPORT.json`, `reports/POLICY_GATE_VALIDATION.json` @ commit `878fa83`, ledger G.66/G.78; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md` / `reports/LOOP_PREREG_2_2026-11-13.md`.*
-| "100.0 % compiled guardrail interception" | **Withdrawn** — never executed; an actually-mounted cartridge refused 0/24 |
-| "60.0 % → 100.0 % Olympiad accuracy" | **Withdrawn** — the counts came from a script that does not execute a model; no accuracy claim is made |
-| Tables 4.1, 4.3, 4.4 | **Withdrawn** — scripts do not execute a model |
-| Prompt-injection immunity as a behavioural guarantee | **Withdrawn** — no measured interception |
-| Crosstalk "strictly 0.0 % (K → ∞)" | **Not measured** — stacking was not executed in the original harness |
-| **New in v4**: aligned-exact bit-identical serving (2k–65k; 8-bit KV) | Measured, G.93–G.116, G.125 |
-| **New in v4**: multi-tenant identity + reuse-factor mechanism | Measured, G.121/G.125 |
-| **New in v4**: point-edit capacity; 8-bit single-fact edit | Measured, G.111–G.126 |
-| **New in v4**: modular routing holds many facts | Measured, G.127 |
-| **New in v4**: model × generator interaction | Measured, G.118/G.124/G.128 |
-*Provenance: `reports/LOOP_BATCH3_AUDIT.json`, `reports/TASK_FAMILY_COMPARISON.json` @ commit `878fa83`, ledger G.118/G.124/G.128; hardware Apple M4 Max 128 GB, MLX; greedy decoding; frozen pre-registrations and commands in `reports/FOLLOWUPS_PREREG_3.md` / `reports/LOOP_PREREG_2_2026-11-13.md`.*
----
-## 8. Provenance
-The v3 corrigendum (`CORRIGENDUM_2026-11-11.pdf`, deposited unchanged) restates the KV-residency range
-as 5–49×; that range is **not reproducible** from the six-checkpoint artefact (measured 1.49–50.37× per
-checkpoint). This addendum carries the corrected range; the corrigendum is kept as the historical record.
-Every number in this addendum traces to a committed artefact plus a ledger item in
-`docs/RESEARCH_IDEAS_AND_HYPOTHESIS_LEDGER.md`:
-| Claim group | Artefact | Ledger |
-| :--- | :--- | :--- |
-| Aligned exactness 2k–32k; chunk edges; restore | `reports/COMPILED_SERVING_ALIGNMENT.json`, `reports/ALIGNMENT_AXES.json`, `reports/CHUNK_EDGE.json`, `reports/SERVING_BENCHMARK.json` | G.93/G.95/G.96 |
-| 65k aligned exactness + memory planner | `reports/ALIGNED_65K.json`, `reports/MEMORY_SCALING_65K.json` | G.114/G.80 |
-| 8-bit KV exactness | `reports/ALIGNED_KVQUANT.json` | G.115 |
-| Multi-tenant exactness + reuse | `reports/MULTITENANT_ALIGNED.json`, `reports/MULTITENANT_ALIGNED_SCALE.json` | G.121/G.125 |
-| Point-edit capacity | `reports/ROME_4BIT_PUSHTHROUGH.json`, `MEMIT_MULTILAYER.json`, `MEMIT_PER_FACT.json`, `ROME_PER_FACT_4BIT.json`, `ROME_PER_FACT_8BIT.json`, `MEMIT_SEQUENTIAL_REFIT.json` | G.111/G.112/G.117/G.122/G.126/G.123 |
-| Modular routing | `reports/MULTIFACT_MODULAR.json` | G.127 |
-| Divergence model/generator axis | `reports/SWE_DIVERGENCE_*.json`, `reports/TASK_FAMILY_COMPARISON.json` | G.102/G.105/G.113/G.118/G.124/G.128 |
-| Independent batch audit | `reports/LOOP_BATCH3_AUDIT.json` | — |
-Commands are the frozen §4 commands of `reports/FOLLOWUPS_PREREG_3.md`; each artefact records its own
-configuration — model, dtype, chunk and request mix where the protocol has them, and only the fields
-relevant to that protocol (some record just counts and hashes).
----
-## 9. Limitations of this addendum
-- One machine, one MLX version per loader; **cross-version determinism is unmeasured**.
-- Equivalence is proven for **batch = 1**. Batched row-identical execution is **not** bit-identical
-  (32/48 rows, G.120); the exact protocol is API-serialised execution. MTP/speculative decoding over
-  an aligned cache is blocked by the mlx_vlm 0.7.12 cache API (G.100).
-- Point-edit and routing results are on one 0.5B model at one layer; the routing arm uses exact
-  subject-string dispatch and fails paraphrases.
-- The refusal adjudication (G.85) is provisional agent adjudication, not independent human review.
-- 4-bit point edits remain unavailable (resolution limit), and the 8-bit result does not make point
-  edits a knowledge-base mechanism.