docs/technical/04_the_head.md

4. The head

(2026-08-27 → 09-01.) The aleph head is a second output read beside the craft's linear base head w_h (a 1024→256 linear): a projection, a 256-anchor book at address dimension 256, and an output W_s born weight-zero under the born-null law. Through 24,000 steps (6.3 billion tokens; 6.5 by the halt at 24,860) its head-off toggle sat between −2.3e-6 and −3.4e-7 at every boundary (warmup 1,145 at 0.30B, validation 2.3626 bpb; cap 8,320; fineweb 20,219 at 1.1172; s0 22,890 at 1.2522 — one craft, fineweb-edu holdout) while the hubs went +1.50 → +2.54 and the banks +0.04 → +2.70 (warmup to s0, hub JSON). TensorBoard shows what the toggle could not: W_s norm growing 19.1 (step 2,000) → 40.6 (24,000) on a read carrying nothing, consumed rank falling 53.9 → 9.2 (16,000) before climbing back to 40.2. Magnitude without function, again — the mother's arc (0.027 → 24.845 with head-off 0.0000 throughout) at a new scale. One rider fell at once: reading the warmup null as "v1 head-election lore" was retracted the same day — the lore itself was retracted (below).

(2026-08-27, 12:11: the dead head named.) An offline diagnostic on the step-12,000 checkpoint (3.1B tokens; local 4090; fp32 head math; one checkpoint) found the read dead by learned avoidance: signed-read norm ~6e-4 per position; proj(h)'s cosines against all 256 anchors ~1e-3 against chance 1/√256 = 0.0625 — 60× below chance; W_s at norm 31 multiplying nothing; head gradients five orders below w_h's; input rank intact (334 of 1,024 post-norm). A ridge solve on the dead reads recovered 0.0006 bpb: least-squares ignition alone was refuted. The mechanism: early aleph noise is cheapest suppressed by rotating the projection into codebook-orthogonality, silencing the address before W_s can earn. An eight-arm trial on the same checkpoint (committed 12:48; ~65k held-out positions, paired per-block bars) priced an offline write: as-trained +0.000001 at 0.014× chance; ridge on the buried reads −0.0014 ±0.0005; the full revival (whitening projection, data-seeded governed book, fp64 ridge) −0.0007 ±0.0003 at 3.3× chance; a fresh random book instead −0.0023 ±0.0008; the op without the 45° projection −0.0008 ±0.0003, a tie with the governed arm; frozen random −0.0023 ±0.0007, a tie with the fresh book; gradient ratio ||g_Ws||/||g_wh|| 1.4e-5 → 6.9e-3 (500×). Offline extraction is worth one to two millibits in every live arm — the linear base already takes everything linearly available — so function can only be earned nonlinearly in training; k-means seeding bought nothing and the governor was exonerated. A retro sweep found burial complete in the first 2,000 steps — 0.032× of chance at the first stored checkpoint (its step not stated), deepening to 0.014× at 12,000 — so prevention must be at birth; the pre-revival read at 24,860 later gave 0.0103×. No JSON of the trial shipped.

(2026-08-27, 12:48: the lineage correction.) The same hour re-read the mother. "Never elected" holds on every record — head-off exactly 0.0 at 51,882, 58,664 and 88,508 (one craft; fineweb gauge). The autopsy's weight claim does not: it recorded "||W_s|| = 0.0 exactly, untouched zero-init after 8.4B tokens," but the shipped mini-beatrix-1 AutoModel is the step-58,664, 17.301B-byte post-fold core, and a direct safetensors read of its head on 09-04 gives W_s norm 0.0270 (the 0.0015 gate folded into a 17.9-norm weight on 08-15), gamma 1.0, K=512 at D=32, all 131,072 entries nonzero. The function claim survives; the zero-weight line is retracted here as checkpoint-specific and numerically off; the graduate at 88,508 carries 24.845 with head-off −0.0000. The "patience precedent" (v1 elected late, so wait) fell with the lore; born-null burial stood at 2/2 crafts; the mother's model card still ships the head-election asset.

(2026-08-29, 07:55: a third category of intervention.) A closed-form write carrying task signal fit neither the governor statute nor the no-balance-machinery law, so boundary-write was minted (S-1003) as a third category — data-dependent, so outside the governor; never in the task gradient, so outside balance machinery — permitted only at run boundaries (phase edges, halted-session resumes), every application logging provenance. alephllm 0.8.2 shipped the op — projection ← whitening-PCA of the post-norm states; book ← fresh Gaussian seed plus 45° minimum-separation; W_s ← fp64 ridge on the base head's cross-entropy residual, alpha swept 0.25–32 held-out — with a frozen-address flag, a liveness gauge in every census (chance from 512 random unit inputs through the same book; buried below 0.1×). Applied to the halted craft at step 24,860 (in s1, ~6.5B tokens; one craft, book seed 13), promoted at 08:08 on Phil's go: zero separation pushes, alpha 8.0 interior, W_s installed at norm 481.5, liveness 0.0103× → 0.9964× of chance, held-out loss 1.2768 → 1.2339 bpb (−0.0429) on 65,536 positions. That delta, twenty times the trial's millibits, is flagged as partly stage-drift recovery — the base head had drifted under the s0 diet, and a ridge fit absorbs what is linearly recoverable. The freeze is verified in the log: codebook rank 197.2 at 24,000 → 155.1 at 25,000, constant through 61,000; consumed rank 40.2 → 144.9.

(2026-08-29 → 08-31: the head lives, then wobbles.) Seven hundred steps later the s1 boundary (25,561; 6.70B) read +0.025805 bpb (head-off 1.280245 vs 1.254440) — the first nonzero aleph-head toggle in the lineage — with W_s trimmed 481.5 → 478.1 by the gradient, not dismantled. The install's −0.043 became +0.026 on the trainer's gauge as the base head reabsorbed the drift-recovery share (two gauges). The trajectory by boundary, one craft on the fineweb-edu holdout, paired per-block bars ~±0.0008 recorded in prose only:

boundary	step	tokens	toggle	val bpb
s1	25,561	6.70B	+0.0258	1.2544
s2	29,376	7.70B	+0.0140	1.2177
s3	33,954	8.90B	+0.0072	1.2318
s4	38,532	10.10B	+0.0130	1.2356
s5	42,347	11.10B	+0.0278	1.2504
cap (session)	43,130	11.31B	+0.0140	1.2614
s6	45,399	11.90B	+0.0126	1.2683
s7	50,740	13.30B	−0.0010	1.1883
s8 (fixed gauge)	53,792	14.10B	+0.0149	1.1911
prose anneal	57,607	15.10B	+0.0101	1.1153
chat anneal, final	61,422	16.10B	+0.0123	1.1097

The s8 row is a retraction stated where it fell: the curriculum_s8 report ran on the trainer's synthetic fallback (1.0737 bpb, head +0.0090, "do not cite"); the fixed-gauge re-issue is the +0.0149 above, yet +0.0090 still travels in the canon toggle chains, the 09-04 audit's head row and this section's previous draft. The narrative between boundaries fell three times — "the decay reversed" at s2 (oscillation, by 30,000), a "durable floor near +0.012" (broken at 32,000: +0.0024 on the s3 rule-chain diet), then "slow reabsorption" — before the settled read at s3, whose own "+0.002 to +0.015, never zero" widened when s7 touched zero exactly as the base head dropped 1.268 → 1.188. The settled read: a small, diet-sensitive, volatile niche, band −0.001 to +0.028, mean about +0.010, never a floor — one craft, a candidate, and the source of a reporting rule: head verdicts at phase boundaries or on band-exit only. Its two exits above the band arrived on the richest diets (s5 causal +0.0278; eval 48,000 on s7's mix +0.0253) — a demand-diversity growth hypothesis the anneal did not honor (+0.0101, +0.0123). Beneath the toggle, W_s grew monotonically 475.1 (28,000) → 515.9 shipped while function did not; consumed rank drifted 147 → 136; the frozen address, logged only from 54,000, sits at 0.81–0.84× of chance against the 0.9964× installed — a drift the record never remarks.

(2026-08-31 → 09-04: the third craft.) The softmax twin of section 5 trained its born-null head with a free address for its whole life — a near-miss fixed in 0.8.7 before launch, since it would otherwise have inherited the treatment's frozen address and been buried by construction. Its head-off toggle never left ±1.1e-5 at any boundary report through 16.1B tokens (+5.9e-6 at 1,145 to −1.06e-5 at 61,422; fourteen rows, one craft). Liveness read 0.23× at step 500, 0.0295× by 3,000 (buried flag set from 3,000 through 61,000), 0.0098× at 38,000, 0.028× at 60,500; W_s climbed 13 → 127 on nothing; consumed rank reached 66–103 mid-run — structure without function. Born-null self-burial is a statute at 3/3 crafts (S-1002): regardless of governor, address freezing, attention family or scale, a weight-zero aleph head racing a same-target linear base buries within ~2,000 steps. The 08-30 selection-mechanism review named the precondition structural: abstention and burial are the same geometry in the signed read; hubs never buried because attention is their read — no bypass — while the head races a parallel path.

(2026-08-31: the cultivation screen.) Phil's question: can the head be cultivated from baseline within a thousand steps or so with the new head state, or does it need an architectural change. Four fresh 237M crafts on the mission card (wikitext-103, ctx 4,096, micro 8, seed 1337, 1,200 steps, gauges every 300 on a light 32,768-position val slice; one seed): A, born null — buried at the first gauge, liveness 0.089× → 0.045×, toggle within 1.1e-5 of zero, W_s 9 → 18; B, the revival op at step 0 (birth ridge −2.95 against an untrained base, alpha 16, W_s 6,345) — toggle +3.27 → +3.50, liveness 0.975× → 1.125×; C, the finished craft's 61,422 head transplanted (projection, book, W_s; not w_h) onto a newborn — +0.0375 → +0.0498 → +0.0723 → +0.0922 (monotone, accelerating), liveness 1.56× → 2.16×, W_s 516.5 → 520.5; D, B's address with W_s zeroed — toggle between −1.2e-4 and −4.6e-5, the address alive at 1.03–1.05×. D is decisive: weight-zero cannot bootstrap against the linear base even without burial; the solve is mandatory. B and C carried and grew: architecture acquitted, bottleneck rung deprioritized. Two caveats: the op fits on the validation batches the gauges later use (surviving inflation unmeasured), and B's toggle measures relocation — at birth the solved head is the primary readout.

(2026-08-31, 13:49 → 23:42: born with function, and its price.) Phil designated the v3 doctrine mid-screen — heads born live, address frozen so burial is structurally impossible — and D tightened it to born with function: address construction plus the birth ridge, or a matured warm-start. The owed supplement ran that night on the local 4090 (B2; fineweb-edu validation over 32,768 positions, micro 2, seed 1337, 1,200 steps): born-null A 3.1576 → 3.0750 bpb with head toggle within 3e-5 of zero; born-live B 3.1740 → 3.0968 with the head carrying +2.2343 bpb of the model's function. B led at 900 steps (3.1340 vs 3.1369) and finished +0.0218 worse — one seed, a candidate: born-live relocates labor at a small early tax; the doctrine keeps its structural justification (guaranteed participation, transplantability, conditioning-facing roles) and loses the free lunch. Whether a trunk co-organized around a live head pays later is a full-run question with no data; arm C has no full-gauge number. Two grading inconsistencies stand: the law-family file carries S-1002 as candidate rows beside the statute, and grades B2 a law where both dockets say candidate.

(2026-09-01.) What the head is still for: B3 (01:48, local 4090) trained only W_s of the finished craft on ~24k plain-English captions (ctx 1,024, pure Adam 3e-4, 1,500 steps; one seed, one domain): caption validation 1.6914 → 1.6728 (−0.0186, not plateaued), arm-off restoring the original W_s bit-exactly, fineweb interference +0.0003 — the whole specialization lives in a swappable W_s. Then the Fable 5.1 review (13:02) called the lane a sunk cost — burial 3/3, a revived head worth ~12 millibits, born-live paying early — and deprioritized head cultivation below the layer-14 attach of section 7; Phil amended rather than agreed: v3 must carry a guaranteed global-frame organ if the head cannot encompass the behavior, its mechanism decided before the v3 spec (S-1004). The born-state statutes now chain: gate-zero deadlocks (S-1001), weight-zero self-buries (S-1002), function is written at a boundary or born with (S-1003, S-1004), the read modulates and never sole-carries (S-1006). The refinement rungs — abstain channel, τ·√D ladder, w_h bottleneck, θ ladder — remain unarmed; every number in this section is one seed.