references+attribution section, inline citations

TECHNICAL.md

@@ -1,6 +1,6 @@
 # AlephLM-0 — Technical Companion
 
-**Repo:** `AbstractPhil/alephlm-0` · **Author:** AbstractPhil · **Status:** complete — seed-0 and seed-1 verdicts in, two-seed tie (Section 5)
+**Repo:** [`AbstractPhil/alephlm-0`](https://huggingface.co/AbstractPhil/alephlm-0) · **Author:** AbstractPhil · **Status:** complete — seed-0 and seed-1 verdicts in, two-seed tie (Section 5)
 
 This document accompanies the week-in-review article and is written to be self-contained: every claim below carries its number, and every number is inherited from the campaign's verified ledger. All weights referenced here — every arm, every seed, including the control and frozen arms — ship to this repository with per-run ledgers.
 
@@ -8,11 +8,11 @@ This document accompanies the week-in-review article and is written to be self-c
 
 ## 1. Overview
 
-AlephLM-0 is the first experiment in which the anchored differentiation mechanism — the "aleph": dense signed dispatch through sinh/cosh reads over oriented half-axes, with no argmax, no softmax routing, and no selection event anywhere — serves as the trunk of an encoder rather than as an adapter attached to one. The question it asks: when the routing mechanism *is* the trunk, does an anchored encoder match a parameter-matched dense encoder under an identical objective?
+AlephLM-0 is the first experiment in which the anchored differentiation mechanism — the "aleph": dense signed dispatch through sinh/cosh reads over oriented half-axes, with no argmax, no [softmax routing](https://arxiv.org/abs/1701.06538), and no selection event anywhere — serves as the trunk of an encoder rather than as an adapter attached to one. The question it asks: when the routing mechanism *is* the trunk, does an anchored encoder match a parameter-matched dense encoder under an identical objective?
 
 Two claims were separated at design time and preregistered:
 
-- **E1** (this experiment): an anchored encoder trunk, trained as a trunk, matches or beats a parameter-matched dense trunk under the identical objective. E1 was explicitly labeled **pre-placement** — the routing sits inside the FFN blocks, upstream of any output distribution.
+- **E1** (this experiment): an anchored encoder trunk, trained as a trunk, matches or beats a parameter-matched dense trunk under the identical objective. E1 was explicitly labeled **pre-placement** — the routing sits inside the [FFN blocks](https://arxiv.org/abs/1706.03762), upstream of any output distribution.
 - **E2** (the program's bet, next stage): the aleph read pays in a *generative* output path, where the address parameterizes the token distribution directly. The plan preregistered that an E1 refutation would itself be the placement law's predicted outcome and would leave the program's direction unchanged.
 
 The headline result — a three-way capability tie at .6031 / .6026 / .6033 (seed 0) — is one of the two outcomes the design named in advance, and the design says exactly what it means. Sections 6–8 show what sits underneath the tie: a 3× sample-efficiency gap at small budget, a shared training-gauge attractor across eight separately-trained trunks, and an inhibition-majority signed structure in the dispatch.
@@ -29,18 +29,18 @@ Three arms, two seeds each, on the identical objective, data, and schedule:
 
 **Exact capacity parity.** Per block, the dense trunk's 2048-wide FFN is replaced by a bank of four experts: one always-on trunk expert (hidden 512) plus three dispatched experts (hidden 512 each) — total hidden width 2048, exactly the dense baseline. Parameter counts: 58,345,764 anchored vs 58,308,864 dense — **+0.063%**, with the delta accounted exactly (codebooks 18,468 + extra biases 18,432). Any behavioral difference is attributable to routing, since capacity is held fixed.
 
-**Zero-init null-path construction.** Dispatched experts enter through gates initialized at sigmoid(−3) with zero-initialized output layers, so at initialization the dispatch contributes *nothing*: gates-off equals the trunk-expert-only path **bit-exactly**, asserted in fp32 at init and at every checkpoint. The trunk starts as a dense trunk and grows its routed function from silence. One measured consequence was preregistered before training: the routing gradient wakes at step 1 rather than step 0 (dL/dw = sigmoid(g)·E(x), and E ≡ 0 at init) — one dead routing step is the price of the bit-exactness guarantee.
+**Zero-init null-path construction.** Dispatched experts enter through gates initialized at sigmoid(−3) with [zero-initialized output layers](https://arxiv.org/abs/2106.09685), so at initialization the dispatch contributes *nothing*: gates-off equals the trunk-expert-only path **bit-exactly**, asserted in fp32 at init and at every checkpoint. The trunk starts as a dense trunk and grows its routed function from silence. One measured consequence was preregistered before training: the routing gradient wakes at step 1 rather than step 0 (dL/dw = sigmoid(g)·E(x), and E ≡ 0 at init) — one dead routing step is the price of the bit-exactness guarantee.
 
 **Why this construction matters.** The toggle law travels with the trunk: because gates-off is exactly the dense path, switching the dispatch off at evaluation time reads the routed experts' contribution *causally*, on any downstream task, with no retraining and no confound. Every capability number in this document has a toggle row beside it for exactly this reason.
 
 ## 3. Training configuration
 
-The recipe is inherited verbatim from the certified dense line:
+The recipe is inherited verbatim from the [certified dense line](https://huggingface.co/AbstractPhil/captionbert-8192-v2):
 
 | item | value |
 |---|---|
-| objective | symmetric InfoNCE + MSE against the 5-BERT reference-member consensus target |
-| corpus | 31,905,616 rows × 4 epochs = 62,312 steps |
+| objective | symmetric [InfoNCE](https://arxiv.org/abs/1807.03748) + MSE against the 5-[BERT](https://arxiv.org/abs/1810.04805) reference-member [consensus target](https://huggingface.co/datasets/AbstractPhil/captionbert-8192-v2-consensus) |
+| [corpus](https://huggingface.co/datasets/AbstractPhil/conceptual-captions-12m-webdataset-berts) | 31,905,616 rows × 4 epochs = 62,312 steps |
 | batch | 2048 |
 | lr | 6e-4, cosine schedule |
 | optimizer | pure Adam, weight decay 0 |
@@ -49,7 +49,7 @@ The recipe is inherited verbatim from the certified dense line:
 | seeds | 0 and 1 per arm |
 | hardware | rented consumer/datacenter GPUs (RTX 5090-class, then A40-class) |
 
-**Wall-clock.** Aleph arms take roughly 2× the dense step time under inner activation checkpointing (53 vs 115 steps/min). On the seed-0 campaign this put the dense arm at ~10.1 h and the frozen-anchor arm (a3-s0) at ~20.5 h.
+**Wall-clock.** Aleph arms take roughly 2× the dense step time under inner [activation checkpointing](https://arxiv.org/abs/1604.06174) (53 vs 115 steps/min). On the seed-0 campaign this put the dense arm at ~10.1 h and the frozen-anchor arm (a3-s0) at ~20.5 h.
 
 **Memory.** After an activation-checkpointing fix, the anchored arm's worst-case training memory came in *leaner* than dense (14.77 GB vs 15.47 GB at the probe scale) — the routed trunk carries no memory tax.
 
@@ -57,13 +57,13 @@ The recipe is inherited verbatim from the certified dense line:
 
 ## 4. Endpoint results
 
-**Harness certification first.** Before any new row was read, the evaluation harness replicated the historical benchmark record to four decimals (distilbert .5883, v2 .6077, v2-B .6031, v2+arms .7287, B+arms .7294, MiniLM .7925), certifying the eval box for the campaign.
+**Harness certification first.** Before any new row was read, the evaluation harness replicated the historical benchmark record to four decimals ([distilbert](https://arxiv.org/abs/1910.01108) .5883, [v2](https://huggingface.co/AbstractPhil/captionbert-8192-v2) .6077, [v2-B](https://huggingface.co/AbstractPhil/captionbert-8192-v2-B) .6031, v2+arms .7287, B+arms .7294, [MiniLM](https://arxiv.org/abs/2002.10957) .7925), certifying the eval box for the campaign.
 
 **Readout note.** AlephLM rows are read through their own trained CLS pooling, while all other models in the harness are mean-pooled. The dense control established that the CLS in-domain advantage is capability-neutral out of domain (Section 8, S0e), so this asymmetry is measured rather than assumed away.
 
 The seed-0 verdict:
 
-| arm | train endpoint (cos / erank) | capability (8-task mean) | dispatch OFF | toggle delta |
+| arm | train endpoint (cos / erank) | capability ([8-task](https://arxiv.org/abs/2210.07316) mean) | dispatch OFF | toggle delta |
 |---|---|---|---|---|
 | a1 anchored (learned) | .8394 / 98.6 | **.6031** | .5743 | −.0288 |
 | a2 dense (control) | .8418 / 99.2 | .6026 | — | — |
@@ -141,7 +141,7 @@ That is target cos ~.84 and erank ~99–103 across two poolings, two corpus trea
 
 ## 7. Sign census — inhibition is the majority mode
 
-Every prior campaign vital was a density gauge (absolute values, norm ratios, squared spectra) — the *signed* structure of dispatch was invisible to the instruments themselves. The first signed census, run on the finished trunks, examined the **frozen-anchor trunk's (a3) K=3 dispatch** and found that **negative routing weights are the majority mode**: fraction-negative **.42–.74 per expert**, mean weight negative in most cells (one block reads −.07 / −.14 / −.03), |w| ~.12–.19, block-structured. Subtraction is how this trunk works. A companion system's K=64 relays showed a second regime — balanced bipolar, fraction-negative ~.50 — giving two distinct signed regimes within one mechanism family.
+Every prior campaign vital was a density gauge (absolute values, norm ratios, squared spectra) — the *signed* structure of dispatch was invisible to the instruments themselves. The first signed census, run on the finished trunks, examined the **frozen-anchor trunk's (a3) K=3 dispatch** and found that **negative routing weights are the majority mode**: fraction-negative **.42–.74 per expert**, mean weight negative in most cells (one block reads −.07 / −.14 / −.03), |w| ~.12–.19, block-structured. Subtraction is how this trunk works. A [companion system's](https://huggingface.co/AbstractPhil/aleph-splat-0) K=64 relays showed a second regime — balanced bipolar, fraction-negative ~.50 — giving two distinct signed regimes within one mechanism family.
 
 The standing assumption that the mechanism's negative-weight capability was still unused was refuted by measurement: it was in use the whole time, invisible to every abs()-based vital. This changes what the tie *is* — a substantial fraction of the trunk's compute is organized as structured inhibition, detachable by a switch: a qualitative mechanism finding sitting underneath a quantitative tie.
 
@@ -177,7 +177,40 @@ Across the campaign, every decision rule — tie bands, match/beat bars, abort c
 
 - **Amplitude, not choice (at trunk scale).** Routing under conserved signed amplitude apportions rather than selects. E1 is the strongest datapoint yet: three partitions of identical capacity (learned, frozen, none) reach the same capability in a pooled-objective encoder, while the amplitude itself is real, causal, and detachable (toggle −.0288 / −.0261). Design consequence: expert count and addressing sophistication are empirical questions, never free wins, in this placement.
 - **Endpoint convergence is a weaker statement than equivalence.** Two systems tied at the endpoint differed threefold in convergence speed (mimicry .9055 vs .7790 at small budget). Verdicts read only at convergence miss dynamics; sample efficiency is a separately valuable axis, and small-rig screens are the cheap instrument that reads it.
-- **The placement law (Law 2), pre-placement reading.** The address's chain-rule advantage pays only where the composed address directly parameterizes the output distribution. E1 was pre-placement by design and behaved exactly as the law predicts. Independently, an adoption experiment on a frozen 0.5B LLM watched the aleph itself elect the output-distribution placement — opening its pre-head gate ~8× while a comparison mechanism closed the same position ~10×. The mechanism votes for where E2 puts it: the generative bed, where dispatch feeds the token distribution.
+- **The placement law (Law 2), pre-placement reading.** The address's chain-rule advantage pays only where the composed address directly parameterizes the output distribution. E1 was pre-placement by design and behaved exactly as the law predicts. Independently, an [adoption experiment](https://huggingface.co/AbstractPhil/alephlm-adopt-0) on a frozen 0.5B LLM watched the aleph itself elect the output-distribution placement — opening its pre-head gate ~8× while a comparison mechanism closed the same position ~10×. The mechanism votes for where E2 puts it: the generative bed, where dispatch feeds the token distribution.
+
+---
+
+## References and attribution
+
+Only sources this document trains on, evaluates against, or inherits methods from are listed.
+
+**Models**
+
+- [BERT](https://arxiv.org/abs/1810.04805) (Devlin et al.) — reference frame and tokenizer for the 5-BERT consensus target; bert-base is the raw baseline in Section 4 and the S0a reference member in Section 8.
+- [DistilBERT](https://arxiv.org/abs/1910.01108) (Sanh et al.) — harness-certification row (.5883).
+- [MiniLM](https://arxiv.org/abs/2002.10957) (Wang et al.), via [Sentence-BERT](https://arxiv.org/abs/1908.10084) (Reimers & Gurevych) — harness-certification row (.7925).
+
+**Data**
+
+- [captionbert-8192-v2-consensus](https://huggingface.co/datasets/AbstractPhil/captionbert-8192-v2-consensus) (AbstractPhil) — the 5-BERT reference-member consensus targets all arms train against.
+- [conceptual-captions-12m-webdataset-berts](https://huggingface.co/datasets/AbstractPhil/conceptual-captions-12m-webdataset-berts) (AbstractPhil) — the training corpus, built from [CC12M](https://arxiv.org/abs/2102.08981) (Changpinyo et al.) as distributed in [pixparse/cc12m-wds](https://huggingface.co/datasets/pixparse/cc12m-wds).
+- [MTEB](https://arxiv.org/abs/2210.07316) — source of the eight capability suites (STS12–16, STS-B, SICK-R, BIOSSES) read through the campaign's certified harness.
+
+**Methods**
+
+- [Transformer](https://arxiv.org/abs/1706.03762) (Vaswani et al.) — the encoder blocks whose FFNs host the expert banks.
+- [InfoNCE](https://arxiv.org/abs/1807.03748) (van den Oord et al.) — the contrastive half of the training objective.
+- [Mixture-of-experts](https://arxiv.org/abs/1701.06538) (Shazeer et al.) — the routing lineage the aleph departs from: dense signed dispatch with no selection event, in contrast to softmax-gated expert selection.
+- [LoRA](https://arxiv.org/abs/2106.09685) (Hu et al.) — lineage of the zero-init null-path construction: zero-initialized output layers so the dispatch starts as an exact no-op.
+- [Gradient checkpointing](https://arxiv.org/abs/1604.06174) (Chen et al.) — the inner activation checkpointing under which the aleph arms train (Section 3).
+
+**Infrastructure**
+
+- [PyTorch](https://pytorch.org) — training and evaluation framework.
+- [RunPod](https://runpod.io) — the A40-class GPU rental on which the interrupted campaign completed (Section 3).
+
+**Related artifacts:** [alephlm-0](https://huggingface.co/AbstractPhil/alephlm-0) (this experiment: all arm/seed weights and per-run ledgers) · [captionbert-8192-v2](https://huggingface.co/AbstractPhil/captionbert-8192-v2) and [captionbert-8192-v2-B](https://huggingface.co/AbstractPhil/captionbert-8192-v2-B) (the certified dense line and its replication) · [alephlm-adopt-0](https://huggingface.co/AbstractPhil/alephlm-adopt-0) (the adoption experiment, Section 10) · [aleph-splat-0](https://huggingface.co/AbstractPhil/aleph-splat-0) (the K=64 companion system, Section 7) · prior installment: [geometric-memory-ft4](https://huggingface.co/blog/AbstractPhil/geometric-memory-ft4)
 
 ---