TECHNICAL companion for the 2s era (verified code-side + hub-side, 2026-09-04)
907da49 verified about 2 months ago
65.9 kB
<!-- VERIFIED 2026-09-04: code-side (wf_15fc8b16) and hub-side (wf ft2-finish) passes applied. -->
# alephllm-mini-beatrix-training — Technical Companion (the 2s era: the full-splat craft and its softmax twin)
Companion to *Twinning Beatrix: A Full-Splat Byte Model, Its Softmax
Control, and What Reaches an Image Generator* (AbstractPhil with Claude
Fable 5 and Claude Fable 5.1, September 2026). It covers two crafts and
one packaged model: `mini-beatrix-2s/` and `mini-beatrix-2s-control/`
under AbstractPhil/alephllm-mini-beatrix-training, and the AutoModel at
AbstractPhil/mini-beatrix-2s; the dates run from the first launch
(run0, 2026-08-26) through the control's endpoint (2026-09-03) and the
claim audit that graded it (2026-09-04). It is written for a reader who
will load the checkpoints and read the reports. It is not the
mini-beatrix-1 life record (`TECHNICAL.md`, which stays as it is), not
the article's argument, and not a manual for the side campaigns that
share the repo root (`reports/v2/{battery,wide_moe,aflow,bcd}` appear
only as a catalogue line). Every number carries its pointer: `file:line`
means E:/mirel/AlephLLM/geolip/alephllm/ at HEAD 4f4b7d6 (package
0.8.7, the last commit of the 2s era); hub paths are relative to the
craft prefix; journal paths are relative to the claude-mind repo. Where
the journal and the code disagree, the code wins and the disagreement
is reported in place. Stack-side detail lives in the
[AlephLLM companion](https://github.com/AbstractEyes/alephllm), which
covers the 0.7.0 mechanisms only; the 0.8.x organs (specials, driver
contract, revival, gauge fallback, twin isolation) are carried here.

## The two crafts at a glance

| | mini-beatrix-2s | mini-beatrix-2s-control | mini-beatrix-1 |
|---|---|---|---|
| d_model / layers / heads | 1024 / 20 / 16 | 1024 / 20 / 16 | 768 / 16 / 12 |
| attention | CausalSplatHUB in all 20 blocks | CausalSDPA (16 × 64) in all 20 | 13 sdpa + 3 hubs (layers 4, 9, 14) |
| hub books × anchors @ D | 4 × 64 @ 128 (supply 0.5×) | — | 1 × 512 @ 32 (supply 16×) |
| banks | 3 experts @ ff 1024 | 3 @ 1024 | 3 @ 768 (full width) |
| head (K @ D) | 256 @ 256, revived at 24,860, address frozen after | 256 @ 256, born null, never elected | 512 @ 32, never elected (W_s = 0.0 at ship) |
| context / vocab | 4096 / 256 byte-trigram | 4096 / 256 | 2048 / 256 |
| params | 237,125,949 | 257,442,109 (+20,316,160, all attention) | 112,524,593 |
| tokens per step | 16 × 4 × 4096 = 262,144 | 262,144 | 48 × 3 × 2048 = 294,912 |
| steps / tokens | 61,422 / 16,101,408,768 | 61,422 / 16,101,408,768 | 88,508 / 26,102,071,296 |
| wall-hours / hosts | 78.33 on one RTX PRO 6000 | 49.98 on three hosts | 70.45 |
| final val bpb (report ledger) | **1.1097** | **2.8846** | 1.121 |
| optimizer | Muon 2e-2 + pure Adam 3e-4, flat after 200 | same | same |
| governor | minsep 45°, every 8 steps, from birth | inherited (head book only) | off |
| package at birth → end | 0.8.1 → 0.8.6 (revival under 0.8.2) | 0.8.7 throughout | cc89a87 → 0.6.0 (pre-specials) |

Shape and knobs: `presets.py:191-203` (2s), `:214-235` (the `-control`
derivation and the head_addr_frozen override), `:155-160` (mb1).
Parameter counts were computed by constructing each preset on the meta
device (`alephlm.py:218-224` `param_count`): 2s = embed 4,982,784 +
20 blocks × 11,574,275 (hub attention 3,178,496 + bank 8,391,683 +
norms 4,096) + final norm 2,048 + head 655,617; the control's block
swaps the hub's 3,178,496 for sdpa's 4,194,304 (qkv 3072×1024 + o
1024×1024, no biases; `attention.py:24-32`). Tokens per step is
`micro_batch × grad_accum × context` (`trainer.py:186`); 61,422 ×
262,144 lands on both manifests' `tokens_seen` exactly. The mother's
numbers are the exemplar's (`TECHNICAL.md`, `mini-beatrix-1/manifest.json`).

## Layout

```
mini-beatrix-2s/
  manifest.json                  # 14 phases, 142 checkpoint entries, notes
  checkpoints/step_XXXXXXXX.safetensors   # 46 bf16 (2000-cadence + every boundary and cap)
  checkpoints/step_00024860_revived.safetensors   # fp32, the post-BOUNDARY-WRITE head
  checkpoints/fp8/               # 20 e4m3 ship-only copies
  resume/latest.pt               # fp32 model + Muon/Adam + stream + RNG + manifest
  resume/boundary_<phase>_step<N>.pt      # 13 immutable boundary archives
  resume/pre_revival_step24860.pt         # the state the revival overwrote
  reports/v2/                    # 13 boundary + 3 session_cap + revival + 4 cultivation
  runs/                          # 5 TensorBoard event files (one per session)
  archive/run0-nospecials/       # the pre-specials first attempt, whole
  beatrix_v2_colab.ipynb         # the driver
mini-beatrix-2s-control/
  manifest.json                  # identical phases and budgets; 136 entries
  checkpoints/step_XXXXXXXX.safetensors   # 44 bf16 (incl. the 22,854 cap)
  checkpoints/fp8/               # 20 e4m3 ship-only copies
  resume/latest.pt  resume/boundary_*.pt  # latest + 13 boundary archives
  reports/v2/                    # 13 boundary + 2 session_cap (22,854; 53,792)
  runs/                          # 6 event files (two are 88-byte aborted starts)
```

The control has no notebook file and no AutoModel packaging; these
checkpoints are its weights. Files: bf16 475,876,922 B (2s) and
515,164,234 B (control); fp8 238,219,518 / 257,813,142 B; resume
archives ~1.904 / ~2.064 GB each; the revived fp32 file 951,692,996 B
(hub headers read by ranged GET; `checkpoint.py:102-128` for the two
export rules).

## Architecture in detail

**The block.** Pre-norm, two residual branches: `x = x + attn(n1(x))`,
`x = x + bank(n2(x))` (`alephlm.py:37-70`). `attn` is CausalSplatHUB
when the layer index is in `hub_layers`, else CausalSDPA
(`alephlm.py:41-50`); every block carries an AnchoredBank
(`alephlm.py:51`). `hub_ckpt` (0 on 2s: the retained scan fits the
96 GB card at 237M, `presets.py:184-198`) would wrap the branches in
activation checkpointing (`alephlm.py:60-68`). The model is embedding →
20 blocks → final LayerNorm → DualHead (`alephlm.py:81-96`); `forward`
takes pre-shifted `targets` (ours) or HF-style `labels` (shifted
internally) and ignores `attention_mask` deliberately
(`alephlm.py:98-124`).

**Trigram embedding and positions.** `e_t = emb0[x_t] + emb1[x_{t-1}] +
emb2[x_{t-2}] + pos[t]`; the shift tables have a 257th PAD_ROW so the
two positions shifted in at the sequence start never read a legal byte
(`embedding.py:19-34`; the pad law — padding with a real byte starved
address consumption, measured +.05..+.11 on repair). `pos` is a learned
absolute table `(1, 4096, 1024)` (`embedding.py:28`); there is no RoPE.
**CausalSplatHUB — the address.** Each book is an `AlephAddress`: K unit
anchors in D dims (`address.py:24-36`, with a `home` buffer holding the
birth copy for drift). The read is a cosine at temperature,
`u_k = cos(x̂, â_k)/τ` with τ = 0.1 (`address.py:38-40`); the oriented
form is the 2K-softmax over the axes `(+a_k, −a_k)`, and the signed read
is its half-difference `w_k = sinh(u_k)/Σ_j cosh(u_j)` — inhibition
(negative w) is first-class; there is no softmax over choices, no
argmax, no top-k (`address.py:1-13,42-66`). The hub's q and k frames are
`Linear(d, D)` per book, orthogonal-init; v and o are shared `Linear(d,
d)` (`attention.py:64-79,106-109`). Construction warns when K > 2D — the
supply law from ROUND 5e and the mini-beatrix-1 hub census
(`attention.py:85-92`); 2s runs 64 @ 128, half of the lawful ceiling.
**CausalSplatHUB — books and budget.** With `n_const = 4` the four books
each own a codebook and q/k frames (`attention.py:64-79`); their
numerators and agreement masses are summed across books before the one
divide — composition by budget, never softmax over books
(`attention.py:245-254`; B4 measured comparative composition at −.10).
`n_const = 1` is the v1 layout with the same state-dict keys
(`attention.py:93-101`), which is how mini-beatrix-1 checkpoints still
load.

**CausalSplatHUB — the exact chunked scan (prefix memory).**
`_code_cat_qk` computes q and k for all books in one stacked GEMM,
folds the row-norm and τ into one post-GEMM scale, and takes ONE fused
softmax over `cat([u, −u])` — the module's docstring names this a
kernel substitution, not a mechanism change: the softmax that exists is
over the 2K oriented axes of a single query, never over positions
(`attention.py:154-212`). `_scan_cat` splits the sequence into chunks of
C = 256 (16 chunks at ctx 4096), forms per-chunk `(2K × d)` sums, takes
the exclusive prefix over chunks as a strictly-lower-triangular GEMM
with fp32 accumulation, adds the within-chunk causal affinity, and
divides the numerator by the scalar agreement mass `den`, floored at
`dtype_floor` (`attention.py:139-151,229-281`; `address.py:16-22`: fp16
flushes 1e-12 to zero, the measured landmine). Exactness is
chunk-independent (`forward_naive` is the oracle, `attention.py:320`;
fp reorder ~1.5e-06). Decode carries a constant-size state per book —
`{Σ k⁺⊗v, Σ k⁻⊗v, Σ k⁺, Σ k⁻}` — regardless of length
(`attention.py:284-318`); the generate path falls back to a sliding
window once the position table is exhausted because "the hub's
constant-size state cannot evict" (`alephlm.py:180-201`).

**The autocast trap.** `torch.einsum` sits in autocast's PROMOTE
category and `F.normalize`/`exp`/`softmax` on its fp32 list, so one fp32
operand dragged the whole scan to fp32 (`attention.py:167`); 0.7.2
(@2be28dd) cast the operands explicitly. Every earlier "bf16" splat run
had silently run fp32 scans; sdpa was unaffected — so every pre-0.7.2
splat-vs-sdpa wall-clock comparison favoured sdpa, and accuracy
verdicts stand (autocast einsum-promote law).

**The governor.** `minsep_project_` bounds `|cos(a_i, a_j)| ≤
cos(θ_min)` for every anchor pair by an exact symmetric geodesic
push-apart, largest violation first, projected to θ_min + 0.5°
(`governor.py:35-80`); `govern_model` collects hub and head codebooks,
runs one batched `bmm` slack check with one device sync, and projects
only the violating books (`governor.py:92-137`). It is a
post-optimizer-step projection, never an auxiliary loss
(`governor.py:11`); the trainer calls it every 8 steps after `opt.step`
and accumulates pair-pushes into `governor/hits_cum`
(`trainer.py:263-267,294-296`; `presets.py:99-105`: θ = 45°). Banks are
excluded by default (a 3-anchor book never crowds). The docstring
records the small-rig verdict it shipped on (gov25m: fires only when
needed, census tail cut 5.4×, fragile-seed +.007, stable-seed +.002;
`governor.py:19`). Code-wins note: the journal's covariance program
describes a *metric* governor through the frame W_a; alephllm's is the
plain projective form on the D-space books, justified in place —
"Hub and head codebooks live natively in their own D-dim address space"
(`governor.py:14`), since the q/k frames project d → D before the cosine
read (`address.py:38-40`).
**CausalSDPA in the twin.** `qkv = Linear(d, 3d)`, `o = Linear(d, d)`,
both orthogonal-init and bias-free; `F.scaled_dot_product_attention(q,
k, v, is_causal=True)` over 16 heads of 64 (`attention.py:24-42`);
the KV cache grows O(n) (`attention.py:43-61`). No QK-norm, no logit
softcap.
**AnchoredBank.** `Bank(x) = trunk(x) + Σ_k w_k(x)·σ(g_k)·expert_k(x)`
(`bank.py:3`): a dense trunk `t_out(GELU(t_in(x)))`, three experts
`(3, 1024, 1024)` with `w_out` zero-initialised (the null path), gates
at −3.0, and dispatch `w = addr.signed(x)` through a 3-anchor book in d
dims. `disable_dispatch` returns the trunk alone — bit-identical to the
dense control at birth (the C6 null); "No balance machinery of any
kind" (`bank.py:10,41-50`).

**DualHead.** `logits = W_h h + γ·W_s s(h)`, `s(h) = addr.signed(proj(h))`
with proj `Linear(1024, 256)`, a 256 @ 256 book, `W_s` zero-initialised
("weight-zero: self-starting") and γ = 1 frozen as an ablation knob
(`head.py:23-51`). The docstring carries the law that set it: on
mini-beatrix-1 over 8.4B tokens a gamma-gated branch DEADLOCKS while a
weight-zero bank elected into +2 bpb (`head.py:8-11`; gate-zero deadlock
law). `disable_aleph` returns `W_h h` alone — the head toggle.

**The special-token control plane.** Thirteen ids that no UTF-8 encoder
can emit — `{0xC0, 0xC1} ∪ [0xF5, 0xFF]`, the overlong leads and the
leads past U+10FFFF — carry structure inside vocab 256: DOC 0xFF, USER
0xFE, MODEL 0xFD, END 0xFC, SYS 0xFB, THINK 0xFA, DATA 0xF9, SEP 0xF8,
MODE 0xF7 (+1 ASCII payload), CUE 0xF6, ESC 0xF5 (+payload = 256
extended slots; 0x00 the null guard, 0x01 CANARY), RES0 0xC0, RES1
0xC1 (`special_tokens.py:20-34,50-71`; `assert len(SPECIALS) == 13`).
The proof is exhaustive — every codepoint 0..0x10FFFF encoded and
checked — and the Trainer runs it at every byte-craft launch
(`trainer.py:72-73`). Doctrine: DOC is appended by the packer after
every document in every phase from birth, identity-gated to the
byte-trigram tokenizer (`streams.py:59-62,209,355-356`); the chat frame `[SYS] text
[END] [USER] text [END] [MODEL] text [END]` (`special_tokens.py:112-136`)
is unforgeable (content is UTF-8) and appears only in the anneal
(`special_tokens.py:36-44`; chat-in-anneal law). The canary frame moved
to CUE/SEP because the old QRY=255 collided with DOC
(`canaries.py:20-29`); `recall_acc` re-baselines at 0.8.0. The trigram
embedding feeds each special into the next two positions' input
representations by construction.

## The recipe (pinned from source; unchanged cc89a87 → b472815)

- **Split**: every parameter with `ndim ∈ {2, 3}` that is not an
  `nn.Embedding` weight rides Muon; everything else — the three byte
  tables, LayerNorms, bank gates, γ, the head bias — rides
  `torch.optim.Adam(weight_decay=0.0)` (`optim.py:54-79`; "pure Adam,
  wd=0 — never AdamW", `presets.py:82`). Measured split for 2s: Muon 385
  tensors / 236,253,184 numel, Adam 107 / 872,765; the control's Muon
  set is 145 / 256,569,344 with the same Adam set. Code-wins note: the
  `optim.py:4-5,55-58` docstrings say positions ride Adam; the code
  sends `embed.pos` (shape `(1, 4096, 1024)`, ndim 3, not an
  `nn.Embedding`) to Muon (`optim.py:68`), verified on the meta device.
  The journal already records the code's behaviour.
- **Muon**: 5-step Newton–Schulz (a, b, c = 3.4445, −4.7750, 2.0315) on
  the momentum buffer after whole-matrix Frobenius normalisation
  `X/(‖X‖_F + 1e-7)`; plain momentum 0.95, no Nesterov, no shape
  scaling; 3-D stacks orthogonalised per slice (`optim.py:20-51`).
- **Learning rates**: Muon 2e-2, Adam 3e-4; `lr_scale = (step+1)/200`
  for 200 steps, then 1.0 forever (`optim.py:82-93`, `presets.py:80-83`).
  The tfevents read `train/lr_scale` 0.25/0.50/0.75 at steps 50/100/150
  and 1.000 from step 200 to the last log on both crafts, through both
  anneal phases: the anneal is a diet change, not a decay.
- **Clip**: `clip_grad_norm_(…, 1.0)` over all parameters; the logged
  `train/grad_norm` is the PRE-clip norm it returns (`trainer.py:244,
  287`). Because Muon normalises the momentum buffer before
  orthogonalising, its update magnitude is independent of gradient
  scale: clipping never changes the magnitude of a Muon step; it only reweights
  steps inside the momentum buffer. The Adam set (`torch.optim.Adam`,
  wd 0, default eps 1e-8; `optim.py:79`) is per-coordinate
  scale-normalised (`m/sqrt(v)`), so under a persistent clip its step
  magnitude is likewise unchanged and the clip only reweights the two
  moment averages (Muon is clip-blind; Adam is nearly so).
- **Precision and batch**: fp32 masters, bf16 autocast on CUDA, TF32 on
  (`trainer.py:14-15,65-66,239`); micro-batch 16 × accum 4 × ctx 4096;
  seed 1337 (`presets.py:97,199`); each micro-batch row is 4097 ids
  scored at 4096 positions.
- **Guards**: a non-finite loss or gradient raises before any optimizer
  step, weights untouched and the resume state not overwritten
  (`trainer.py:250`); the spike flag `loss > 2·EMA + 0.5` is logged
  only (`trainer.py:273-275,293`). Neither fired on either craft.
- **Cadences**: log 50, health census 500, eval and checkpoint 2000,
  fp8 every 5th checkpoint, TB upload 1000, val 262,144 tokens, 128
  canary episodes (`presets.py:88-96`).
- **Absent by design**: no QK-norm, no logit softcap, no z-loss, no
  weight decay, no dropout, no RoPE, no LR decay in any phase.

## The diet (all on one gauge: fineweb-edu holdout bpb)

| phase | dataset | budget | → step | 2s bpb | control bpb | note |
|---|---|---|---|---|---|---|
| warmup_wikitext | wikitext-103 | 0.30B | 1,145 | 2.3626 | 2.1056 | sdpa led by 0.257; protocol pre-registered here |
| fineweb_main | fineweb-edu | 5.0B | 20,219 | **1.1172** | 1.1763 | the crossover; hubs +2.22 / banks +3.55 |
| fineweb_extended | fineweb-edu | 10B | — | — | — | DEFERRED, unspent on both |
| curriculum_s0 | curriculum-s0 | 0.7B | 22,890 | 1.2522 | 1.5047 | |
| REVIVAL | — | — | 24,860 | 1.2768 → 1.2339 (op gauge) | — | BOUNDARY-WRITE; `pre_revival_step24860.pt` archived |
| curriculum_s1 | curriculum-s1 | 0.7B | 25,561 | 1.2544 | 1.3917 | first live head toggle in the lineage, +0.0258 |
| curriculum_s2 | curriculum-s2 | 1.0B | 29,376 | 1.2177 | 1.3938 | 2s grad-norm curriculum+anneal max 0.619 @26,750 |
| curriculum_s3 | curriculum-s3 | 1.2B | 33,954 | 1.2318 | 1.5317 | |
| curriculum_s4 | curriculum-s4 | 1.2B | 38,532 | 1.2356 | 1.5689 | both 2s mechanisms above +3.0 |
| curriculum_s5 | curriculum-s5 | 1.0B | 42,347 | 1.2504 | 2.1762 | control grad-norm median 126 |
| curriculum_s6 | curriculum-s6 | 0.8B | 45,399 | 1.2683 | 3.1940 | control max 41,070 @44,600 |
| curriculum_s7 | curriculum-s7 | 1.4B | 50,740 | 1.1883 | 3.1201 | |
| curriculum_s8 | curriculum-s8 | 0.8B | 53,792 | *1.0737* (synthetic gauge) / **1.1911** (cap) | 3.0810 | 2s row STRUCK; true ledger = `session_cap_step53792` |
| anneal_nochat | anneal-nochat | 1.0B | 57,607 | 1.1153 | 2.9270 | the pre-chat baseline; probe mean .389 |
| anneal_mix | anneal-mix | 1.0B | 61,422 | **1.1097** | **2.8846** | run best; probe mean .389 == pre-chat |

Phases and budgets: `presets.py:132-146` (the preset plan carries
anneal_mix at 2B deferred; the notebook's C6 cell rewrote it to 1B and
inserted anneal_nochat before it — "two-phase anneal activated: nochat
1B then chat 1B (plan of record 2026-08-31)" in both manifests' notes; the
hub copy of `beatrix_v2_colab.ipynb`, uploaded 2026-08-28T23:00Z with cells
C1–C5 and pinned to `geolip-alephllm @2a902b7` (0.8.1), predates the
amendment and carries no C6 cell),
`curriculum.py:657-663` (STAGE_TOKENS), `curriculum.py:666-689`
(append at runtime). Manifest order is not chronology: the S0–S8 phases
are listed after the anneals but ran before them, because
`current_phase()` activates the first `planned` phase and deferred
phases are invisible until flipped (`manifest.py:33-41`); count back
from the totals (16,101,408,768 − anneal_mix's 1,000,079,360 lands on
step 57,607) rather than reading the list.
**The mixes** (`curriculum.py:614-655`; weights sum to 1): s0
tinystories .74, simple-wiki .20, aochildes .01, recall-synth .03,
counting-synth .02 · s1 perspective-synth .30, tinystories .28,
simple-wiki .20, cosmo-young .12, concept-synth .04, recall-synth .03,
siqa-narrative .02, aochildes .01 · s2 concept-synth .30, cosmo-young
.30, simple-wiki .12, tinystories .10, definitions .08, wikipedia-en
.07, recall-synth .03 · s3 rulechain-synth .32, proofwriter-prose .22,
simple-wiki .12, tinystories .10, wikipedia-en .08, cosmo-young .08,
concept-synth .04, recall-synth .03, babi-prose .01 · s4 arith-synth
.35, cosmo-young .32, simple-wiki .12, tinystories .08, rulechain-synth
.06, counting-synth .04, recall-synth .03 · s5 causal-synth .32,
atomic-flicker .25, tinystories .12, simple-wiki .10, gutenberg .08,
cosmo-young .07, recall-synth .05, siqa-narrative .01 · s6
tryfail-synth .32, tinystories .22, arith-synth .13, cosmo-young .10,
simple-wiki .10, causal-synth .08, recall-synth .05 · s7 tinystories
.10, cosmo-young .10, rulechain-synth .10, arith-synth .10, fineweb-good
.10, simple-wiki .08, atomic-flicker .08, causal-synth .08,
tryfail-synth .08, proofwriter-prose .08, recall-synth .05,
perspective-synth .04, babi-prose .01 · s8 definitions .25,
register-synth .25, simple-wiki .15, cosmo-young .12, wikipedia-en .10,
tinystories .09, recall-synth .03, siqa-narrative .01. Rails audited at
append time: MAX_EPOCHS 4.0, MAX_GENERATOR_SHARE 0.35, MIN_BALLAST 0.30
natural text (`curriculum.py:548-551,565-596`); holdouts enforced at
the generator — bAbI tasks 16/19, ProofWriter depth 5, rule-chain
depth 5, three-digit subtraction (`curriculum.py:26-31`). No chat
template in s0–s8.
**The anneal recipes** (`streams.py:68-77`): ANNEAL_MIX = fineweb-edu
.45, cosmopedia .20, tinystories .15, soda-dialogue .12,
beatrix-texture-sp .05 (the specials-native SYS/USER/MODEL/END
identity texture), recall-synth .03; ANNEAL_NOCHAT is the same list
minus beatrix-texture-sp, renormalised by MixStream — so the nochat
boundary is the exact pre-chat baseline on the same distribution.
MixStream draws one component per batch from a counter-seeded RNG
(`streams.py:375-404`): reproducible and resumable, but its state
carries no recipe fingerprint (fresh starts only across recipe
changes).
**The DOC plane on the diet.** The packer appends DOC after every
document (`streams.py:192-372`); the specials gauge reads `doc_bpb`
(bits at positions whose target is DOC) and `reset_bpb` (mean bits over
the 16 targets after each DOC) on the fixed val batches
(`instruments.py:310-352`). doc_bpb fell 12.83 → 3.29 by 0.52B and
sits at 3.862 at the endpoint (control 5.422); reset_bpb 4.287 → 2.130
(control 3.825). doc_count is 70 in every 2s and control report except the struck 2s
`curriculum_s8_step53792.json`, which reads doc_count 1612 (the synthetic
fallback stream) — the same 70 documents every time on the fineweb-edu gauge.

**The val gauge.** `_val()` builds 4 batches of (16, 4097) from
`build_stream(dataset, …, seed = 1337 + 9999, role="val")`
(`trainer.py:384-411`); for fineweb-edu the val region is the first
2,048 unshuffled rows, which the training stream skips
(`streams.py:56,279-282`), and `build_stream` routes every
`curriculum-*` and `anneal-*` val request to that same fineweb-edu head
holdout — "gauge continuity" in the source (`streams.py:423-447`). So
from fineweb_main onward every row of both crafts is one gauge; the
canon's CB2/CB3 "own slice" wording is struck by erratum
(`canon/splat_aleph_battery.md:583`). **The s8 fallback caveat**: at the
terminal curriculum boundary `current_phase()` is None (everything done
or deferred) and pre-0.8.5 `_val` fell to the `synthetic` stream, so the
2s `curriculum_s8_step53792` ledger is an incomparable artifact; 0.8.5
(@7d48328) makes `_val` fall back to the last DONE phase's dataset
(`trainer.py:397-401`), and the true end-of-curriculum ledger is
`session_cap_step53792`, taken at the same step on the restored gauge
(1.1911; banks +3.214, hubs +3.341, head +0.0149). The control, born
under 0.8.7, measured its s8 row on the repaired gauge; the s8 pair is
therefore not same-gauge and is struck from the paired count.

## The life (`reports/v2/`, `resume/`, `runs/`)

**Sessions.** The 2s craft ran five sessions on the one RTX PRO 6000
(Blackwell, 96 GB; 4.54 s/step, ~58k tok/s), each its own event file:
steps 50–8,300 (host 4d130c0e7351), 8,350–24,850 (91b9f3518593),
24,900–43,100 (e935248f0778), 43,150–53,750 (68304a2b8506),
53,800–61,400 (c2dc3d694638). The breaks are the session caps at 8,320
(fineweb, 2.181B), 43,130 (inside s6, 11.306B) and 53,792 (the s8 boundary
itself, the resumed session's first action) — `session_cap_step*.json`
is written when a session hits MAX_HOURS mid-phase — and the revival
halt at 24,860. Sessions are boundary-exact: the driver calls
`train(max_hours, stop_at_boundary=True)`, which returns after each
boundary checkpoint with `_last_boundary` set, and never auto-resumes a
manual stop (`_interrupted`; `trainer.py:170-184,328-335`; the 0.8.1
driver contract, minted after run0 rolled through its boundaries and
swallowed a KeyboardInterrupt).

**Resume.** `_restore` loads `resume/latest.pt` — fp32 model, both
optimizer states, the stream position, torch and CUDA RNG, and the
manifest snapshot embedded in the payload, which wins over the hub's
`manifest.json` (`trainer.py:110-153`; `checkpoint.py:131-175`). A
missing payload over a manifest with steps > 0 refuses to start fresh
(`trainer.py:110-120`). Every phase boundary archives an immutable copy
`resume/boundary_<phase>_step<N>.pt` (`trainer.py:469`;
`checkpoint.py:134-154`). Resume is not row-exact: the datasets shuffle
buffer (10,000 rows, `streams.py:292-293`) is not part of the saved
state, so each resume refills it (`streams.py:12`, the honesty note).

**The revival at 24,860.** The born-null head buried itself inside the
first 2,000 steps (0.032× chance liveness at the first stored
checkpoint; ‖s‖ ~6e-4; proj cosines ~60× below chance — learned
avoidance; `repos/alephllm.md:1564-1643` — journal readings from
checkpoint analysis: the craft's TB carries no `head/liveness_ratio`
scalar before 0.8.4; the control's TB reads 0.052× @2,000, 0.014×
@12,000, 0.010× @24,000). The BOUNDARY-WRITE op
(`revival.py:44-104`) ran on the halted run: proj ← uncentered
whitening-PCA of the final-norm outputs (sv floor 1e-3 relative);
codebook ← fresh `normalize(randn)` with min-sep 45° (0 pushes needed);
W_s ← fp64 ridge on the base head's CE residual, α swept on the held-out
half and folded in. `revival_step24860.json`: K 256, D 256, seed 13,
sv0 3676.66, svD 15.90, α 8.0, ‖W_s‖ 481.53, fit_positions 65,536,
base_bpb 1.27682 → revived_bpb 1.23391 (Δ −0.04290 held-out),
liveness 0.01027× → 0.99642×, version 0.8.2, commit d6e1854. The
pre-revival resume was archived, Muon momentum zeroed for the three head
tensors, and the revived payload promoted on Phil's explicit go
(`repos/alephllm.md:1669`); thereafter `head_addr_frozen` freezes proj
and the head codebook by `requires_grad` alone, so optimizer groups and
resume state are unchanged and Muon skips them (`trainer.py:75-80`;
`optim.py:42-43`). `checkpoints/step_00024860.safetensors` is the
pre-revival bf16 weight; `step_00024860_revived.safetensors` (fp32) is
the post-write weight; neither the revived file nor the resume archives
appear in `manifest.json`'s checkpoint index.

**The anneal.** Activated 2026-08-31T09:54Z after s8; nochat done
14:42Z, chat done 19:31Z; the frame cost nothing in a bracketed phase
(probe mean .389 → .389; val 1.1153 → 1.1097).

**The 2s boundary ledgers** (`reports/v2/*.json`; bpb on the fineweb-edu
holdout; toggles = mechanism-off minus full):

| report | step | tokens | bpb_full | banks | hubs | head | doc / reset |
|---|---|---|---|---|---|---|---|
| warmup_wikitext | 1,145 | 0.300B | 2.36257 | +0.0365 | **+1.5042** | −2.3e-06 | 12.83 / 4.287 |
| session_cap | 8,320 | 2.181B | 1.20332 | +1.727 | +2.036 | −4.9e-07 | |
| fineweb_main | 20,219 | 5.300B | 1.11719 | +3.5527 | +2.2157 | −3.4e-07 | |
| curriculum_s0 | 22,890 | 6.000B | 1.2522 | | | ~0 (born null) | |
| curriculum_s1 | 25,561 | 6.700B | 1.2544 | | | **+0.0258** | |
| curriculum_s2 | 29,376 | 7.700B | 1.2177 | | | +0.0140 | |
| curriculum_s3 | 33,954 | 8.900B | 1.2318 | | | +0.0072 | |
| curriculum_s4 | 38,532 | 10.100B | 1.2356 | >3.0 | >3.0 | +0.0130 | |
| curriculum_s5 | 42,347 | 11.100B | 1.2504 | | | +0.0278 | |
| session_cap | 43,130 | 11.306B | 1.2614 | | | | |
| curriculum_s6 | 45,399 | 11.900B | 1.2683 | | | +0.0126 | |
| curriculum_s7 | 50,740 | 13.300B | 1.1883 | | | −0.0010 | |
| curriculum_s8 (STRUCK) | 53,792 | 14.100B | *1.0737* | *+3.396* | *+5.164* | | *0.843 / 1.889* |
| session_cap (true s8) | 53,792 | 14.100B | **1.19112** | +3.2144 | +3.3410 | +0.0149 | 5.052 / 2.469 |
| anneal_nochat | 57,607 | 15.100B | 1.1153 | | | +0.0101 | |
| anneal_mix | 61,422 | 16.101B | **1.10970** | +3.3115 | +3.3584 | +0.0123 | 3.862 / 2.130 |
Blank cells are not in the record consulted here; the head column is
complete. The warmup row is the first full-splat-under-AR ledger: with
all 20 hubs off the craft loses its entire attention (+1.50 at 0.3B),
which is what "load-bearing, not decorative" measures against the
mother's 3-hub form. `hidden_erank_floor` flags TRUE in every report from the 8,320 cap on (the
trainer's own TB census, on the first val batch, flags it from step 500)
(L19 pre-norm erank in single digits — the byte funnel; the post-norm
`nf(h)` erank reads 334/1024), `head_buried` FALSE from s1 on, den
floors 0.0 the entire run, zero loss spikes. Two "final val" readings
exist on the hub: the report ledger 1.10970 (the gauge of record) and
the manifest's checkpoint entry `val_bpb` 1.11403 at 61,422 — the
manifest column is the LAST PERIODIC EVAL's `bpb_full` (eval every
2,000; `trainer.py:444-447`), so it lags every boundary (fineweb 1.1156
vs 1.1172; s8 1.1833 vs 1.0737/1.1911); the manifest minimum is 1.1140
@60,000. Cite the report.

## The paired record (`mini-beatrix-2s-control/reports/v2/`)

**Derivation.** Every preset auto-generates a `-control` twin: the same
`AlephLMConfig` with `hub_layers = ()` (`presets.py:216-229`), so
`hub_K/hub_D/hub_const/hub_chunk` are carried but unused and every
block's attention is CausalSDPA. 0.8.7 (@b472815) gave twins their own
`TrainConfig` copy — the shared instance had let `head_addr_frozen=True`
leak into the control, a post-revival flag no birth recipe may carry —
and sets the 2s control to `head_addr_frozen = False` explicitly: the
birth recipe, born-null unfrozen head, which buries as the treatment's
did for its first 24,860 steps (`presets.py:221-235`). The control
inherits `governor="minsep"`; with no hub books it governs the head
codebook only (hits_cum 10 at warmup, fineweb and the 22,854 cap — the first
session — and 0 in every later session's report;
`crowd_check_extra_hits` 0). Same seed 1337: construction order embed →
blocks → norm → head after `torch.manual_seed` (`trainer.py:63`;
`alephlm.py:85-96`) makes the embedding draws identical between craft
and twin; the RNG streams diverge from block 0's attention onward
(a derivation from construction order, not a weight comparison).

**Protocol.** Declared at the control's warmup boundary, before the
outcome, while sdpa led (2.1056 vs 2.3626): the architecture verdict is
the paired boundary table across the full 16.1B matched budget; the
headline is endpoint val; secondaries are probe means at matched
boundaries and the doc/reset gauges; no verdict from any intermediate
lead in either direction (`repos/alephllm.md:1961-2002`; a control must
be able to fail). Completed unaltered through 13 paired rows. Budgets
are matched in tokens, not wall-clock: fused sdpa ran 71.9k tok/s on the
RTX PRO 6000 and 113k on the H100 against the eager scan's ~58k.

**The paired rows** (bpb_full, same step, same tokens, same gauge except
where struck; deficit = control − craft):
| boundary | step | tokens | 2s | control | deficit | control bank toggle | control grad-norm median |
|---|---|---|---|---|---|---|---|
| warmup | 1,145 | 0.30B | 2.3626 | 2.1056 | −0.257 | −0.008 | |
| fineweb | 20,219 | 5.30B | 1.1172 | 1.1763 | +0.059 | 1.62 | 0.31 |
| s0 | 22,890 | 6.00B | 1.2522 | 1.5047 | +0.253 | 3.44 | 6.47 |
| s1 | 25,561 | 6.70B | 1.2544 | 1.3917 | +0.137 | 3.61 | 2.43 |
| s2 | 29,376 | 7.70B | 1.2177 | 1.3938 | +0.176 | 4.13 | 11.9 |
| s3 | 33,954 | 8.90B | 1.2318 | 1.5317 | +0.300 | 4.73 | 12.7 |
| s4 | 38,532 | 10.10B | 1.2356 | 1.5689 | +0.333 | 5.09 | 24.5 |
| s5 | 42,347 | 11.10B | 1.2504 | 2.1762 | **+0.926** | 4.36 | 126.4 |
| s6 | 45,399 | 11.90B | 1.2683 | 3.1940 | **+1.926** | 3.89 | 471 |
| s7 | 50,740 | 13.30B | 1.1883 | 3.1201 | +1.932 | 4.17 | 162 |
| s8 | 53,792 | 14.10B | *1.0737* | 3.0810 | STRUCK (gauges differ) | 5.48 | 145 |
| anneal_nochat | 57,607 | 15.10B | 1.1153 | 2.9270 | +1.812 | 5.64 | 167 |
| anneal_mix | 61,422 | 16.10B | **1.1097** | **2.8846** | **+1.775** | 6.11 | 215 |
Eleven same-gauge rows after the crossover, all in the craft's favour;
the endpoint gap is +1.775 at 16,101,408,768 tokens on both. The
control's own trajectory is the finding: 1.1763 at 5.3B → 3.1940 at s6
→ 2.8846 at the end — a destabilised model at its endpoint, not a
trailing one. Its endpoint ledger: bank_off 8.99015 (toggle +6.1056),
head toggle −1.06e-05, `head_buried` TRUE at every row from fineweb on,
doc_bpb 5.422, reset_bpb 3.825; probes p0 .200 p1 .333 p2 .267 p3 .333
p4 .300 p5 .433 p6 .233 p7 .433 p8 .333 against the craft's .367 .367
.600 .200 .333 .567 .333 .333 .400 — ties at n = 30 (SE of a difference
.126). The control's bank dispatch entropy fell through the curriculum
(`collapse/dispatch_entropy_L5`: .94 at the s0 census, .75 @30,000, .48
@35,000, .22 at the s5 census; L0 .75 → .47 @30,000; layer minimum .14
@42,500), ahead of the >30 grad-norm-median onset at 38,000.

**The destabilisation facts** (from the hub tfevents, `train/grad_norm`
= pre-clip): the control first crossed the clip at step 17,600 (4.61B,
inside fineweb, on the RTX PRO 6000, while still leading); 813 of 824
logged curriculum+anneal steps clipped (98.7%); 38 steps above 1,000;
max 41,070 @44,600; the > 30 onset at bin 38,000 (late s4) and > 100 at
40,500 (mid-s5) inside one event file. The 2s craft: 0 of 1,225
logged steps from step 200 on above the clip (the only reading above 1.0 is
step 50, 1.063, inside the 200-step LR warmup); max 0.760 @1,300 after the
warmup boundary, and 0.619 @26,750 (s2) over the 824 curriculum+anneal steps. The spike
detector and the non-finite guard never fired on either craft. Under
Muon the clip braked nothing in the Muon group (qkv/o, the position
table, the head); the escalating norm is a sharpness signal, not a
throttled step.

**The grade and its caveats.** The instance holds: the all-sdpa twin is
a destabilised control under the splat-tuned recipe, one craft per arm
(S-1802 INSTANCE). The general claim — softmax attention is inadequate
for this model's attention requirement — is a CANDIDATE (S-1803) behind
confounds stated as measurements: (a) the recipe was tuned on the splat
craft and carries none of the field's softmax stabilisers (QK-norm,
softcap, z-loss, weight decay, decay), and the signature — clip-engaged
smooth degradation, entropy and rank collapse, no NaN, onset on plain
prose while leading — is the documented softmax logit-growth mode;
(b) n = 1, no bar, +20.3M parameters in the losing arm; (c) shape: the
mother null — mini-beatrix-1 (13 sdpa + 3 hubs, d768) under the
identical TrainConfig (git cc89a87..b472815; the manifests record no
train_config) through the same nine stages at the same budgets ran a
post-warmup grad-norm max of 0.883, 0 of 1,750 logged steps clipped
(`canon/splat_aleph_battery.md:955`; canon-cited, not re-verified against
the mini-beatrix-1 event files for this companion — her 88,508 steps /
26,102,071,296 tokens / 70.45 h do verify against
`mini-beatrix-1/manifest.json`) — with the nuance that v1 entered
s0 at 17.3B tokens after 15B prose and a 2B anneal, the control at 5.3B;
(d) the aleph-addressed bank sits in both crafts, its dispatch collapse
precedes the escalation, and no sdpa + dense-FFN arm exists at any
scale; (e) the 4096×1024 position table on WD-free Muon at 2e-2 forever;
(f) house Muon is not reference Muon; (g) host and row-order
asymmetries. One gauge (val bpb; probes are ties); the s8 pair struck;
three hosts. dist-e002 (vision, two seeds) remains the only Law-E
splat-vs-sdpa endpoint, and it is a tie.
**Provenance of the three hosts.** RTX PRO 6000 from birth to the 23.5 h
cap at 22,854 (event file 50–22,850); a Colab H100 80 GB from 22,854
(~2.3–2.4 s/step) through s0–s8 (22,900–53,750), 36 steps before the s0
boundary, with the shuffle buffers refilled; a session 53,800–56,000
follows the 53,792 cap in the H100 session's own host and process (event
file `…0a706d592fec.620.1`, 20:01–21:35Z, ~2.3 s/step, 114k tok/s); a third
host (2b1783ef9e94) opened 2026-09-03T01:41Z from the step-56,000
checkpoint at ~3.6 s/step and ran both anneal rows (56,050–61,400; two
88-byte event files are aborted starts) — a replay of up to 999 steps at
that restart is a hedge on the 1,000-step TB-upload cadence, not a hub
reading (the H100 process's last event is 21:35:20Z, the step-56,000
checkpoint landed 21:35:33Z and its manifest 21:36:06Z, the third host
opened 01:41:15Z; the hub can neither confirm nor refute a replay), and
it left no cap report.
The canon's "on the H100" lines for boundaries 11–12 are wrong by
erratum (`canon/splat_aleph_battery.md:1008`). The escalation began
pre-resume on the RTX PRO 6000; a 36-step data-order change cannot make
a 0.25 bpb gap on a fixed eval.
## The boundary report (`reports/v2/`) — schema
Written by the notebook's `boundary_report(run, tag)` cell to
`reports/v2/{tag}_step{step}.json`, tag = the phase that just finished
or `session_cap`:

- `tag`, `step`, `tokens` — `tokens` is `manifest.tokens_seen` (new
  in the v2 record).
- `probes` — `{p0_grounding, p1_self_other, p2_difference, p3_logic,
  p4_arithmetic, p5_causal, p6_learning, p7_composite,
  p8_articulation}`, each `{acc, families{…}, n: 30}`; byte-NLL argmin
  over the options, fp32, no sampling (`probes.py:191-221`); holdout
  families p3:d5 and p4:sub3 gauged from time zero (`probes.py:171`).
- `census_flags` — `hidden_erank_floor` (any block-output effective
  rank < 0.05·d = 51.2), `anchor_merge` (any pair |cos| > 0.99 in a
  bank/hub/head book), `dispatch_collapse` (bank dispatch entropy
  fraction < 0.10), `den_floor` (hub den floor_frac > 1e-3),
  `head_buried` (liveness < 0.1; added at 0.8.2 — absent from the 2s
  warmup_step1145, session_cap_step8320, fineweb_main_step20219 and
  curriculum_s0_step22890 reports, present from s1; present in every
  control report),
  `codebook_erank_floor` (bank codebook erank < 1.5)
  (`instruments.py:150-178`). The census sample is `(2, min(1024,
  ctx))` random bytes in 0..254 (`torch.randint(0, 255, …)`, so DOC never appears in the census sample) in the notebook writer; the trainer's own census
  uses the first val batch (`trainer.py:371-374`).
- `ledger` — `bpb_full`, `bpb_bank_off`, `bpb_head_aleph_off`,
  `bpb_hub_off` (only if any block is a hub — the control's ledger has
  five keys), and `toggle_<x> = bpb_<x> − bpb_full`; mean CE over the
  fixed val batches / ln 2 (`instruments.py:187-212`). No `w_s_norm`
  field (the mb1 schema had one).
- `special` — `{doc_count, doc_bpb, reset_bpb}` (`instruments.py:
  310-352`); None on BPE crafts.
- `governor_hits_cum` — the session's cumulative pair-pushes; the
  counter is a Trainer attribute defaulting to 0 (`trainer.py:266`),
  not persisted in the manifest or the resume payload, so it resets
  every session (103,951 @1,145; 1,304,497 @20,219; 70,747 at s1 after
  1,604,488 at s0; 0 at session_cap_53792; 776,450 @61,422).
- `crowd_check_extra_hits` — pushes from an on-demand `govern_model` at
  report time (415 @20,219; 709 @61,422; 0 on the control).

Other report kinds in the same directory: `revival_step24860.json` (the
op record above, no probes/ledger — chart code must skip it) and
`cultivation_{A_born_null,B_revival_birth,C_matured_transplant,
D_live_address_only}.json` — the C7 head-cultivation screen, 1,200
steps at seed 1337 with gauges at 300/600/900/1200 `{toggle, w_s_norm,
head_s_norm, head_s_chance, head_liveness_ratio, head_buried}` and,
for B and D, a `revival` dict (α 16.0, fit 16,384, base 8.336 → 5.389).
Endpoint toggles: A −5.3e-06 (buried, liveness 0.045); B +3.496
(liveness 1.125); C +0.0922 (‖W_s‖ 520.5); D −7.1e-05 (liveness 1.037,
not buried, no function) — a live address alone does not carry
function; a revived head at birth does.

**The manifest** (`manifest.py:16-110`): `preset`, `model_config` (the
full `AlephLMConfig`), `created_utc`, `updated_utc`, `steps`,
`tokens_seen`, `wall_hours`, `phases[{name, dataset, planned_tokens,
tokens_done, status ∈ planned|active|done|deferred}]`,
`checkpoints[{step, tokens, kind ∈ safetensors|resume|fp8, path,
val_bpb, utc}]`, `data_state{dataset, rows_consumed, epoch}`,
`notes[{utc, msg}]`. 2s: created 2026-08-27T00:35:31Z, updated
08-31T19:37:50Z, 142 entries (59 safetensors / 24 fp8 / 59 resume → 46
distinct bf16 paths). Control: created 08-31T21:26:32Z, updated
09-03T07:16:38Z, 136 entries → 44 bf16. Neither records a train_config.

**Checkpoint kinds** (`checkpoint.py:102-155,187-195`): bf16 safetensors
every 2,000 steps and at every boundary, every state-dict tensor
including the `addr.home` buffers (so the file's numel, 237,908,285,
exceeds the parameter count by 782,336); fp8-e4m3 every 5th checkpoint
or final, 2-D+ tensors with numel > 4,096 stored with a per-tensor fp32
scale `amax/448` under `<key>.scale`, small params kept bf16, metadata
"inference/testing only" — never train from these; `resume/latest.pt`
fp32 with the manifest embedded, overwritten at every checkpoint and
archived at boundaries. Hub uploads retry three times and never crash
training; downloads distinguish genuine absence from network trouble
and raise on the latter (`checkpoint.py:29-100`).

**TensorBoard tags** (`trainer.py:283-296`; `instruments.py:215-260`;
314 tags on the craft, 212 on the control): from the loop
`train/{loss,bpb,grad_norm,lr_scale,tokens,tokens_per_sec}`,
`collapse/loss_spike`, `governor/hits_cum`, `sys/vram_gb`; from the
census per layer i `erank/hidden_L{i}`, `erank/hub_consumed_L{i}`,
`erank/hub_codebook_L{i}`, `erank/bank_codebook_L{i}`,
`cv/bank_load_L{i}`, `cv/hub_usage_L{i}`, `drift/bank_L{i}`,
`drift/hub_L{i}`, `sign/bank_frac_neg_L{i}`,
`collapse/dispatch_entropy_L{i}`, `collapse/den_floor_L{i}`,
`gates/L{i}_e{k}`; head `head/{gamma,w_s_norm,s_norm,liveness_ratio,
consumed_erank}`, `erank/head_codebook`; `collapse/flag_{…}` for the six
flags; `toggles/{bpb_full,bpb_bank_off,bpb_head_aleph_off,bpb_hub_off,
toggle_bank_off,toggle_head_aleph_off,toggle_hub_off}`; from the full
eval `canary/{recall_acc,wrongpair_frac,noretrieval_frac}` and
`special/{doc_count,doc_bpb,reset_bpb}`. The control lacks every hub
family; the archived run0 file lacks `head/liveness_ratio`,
`head/s_norm`, `special/*` and `flag_head_buried` (pre-0.8.0 specials;
the `head_buried` flag arrived at 0.8.2 @d6e1854 and the `head/s_norm` +
`head/liveness_ratio` TB scalars at 0.8.4 @a690ea4).

**`archive/run0-nospecials/`.** The pre-specials attempt, archived whole
2026-08-27T00:16Z: its manifest (created 08-26T20:50:18Z, steps 2,537,
tokens 665,059,328, wall 3.22 h, warmup done at 1,145, fineweb_main in
progress), bf16 checkpoints 1,145/2,000/2,253/2,537, fp8 2,253/2,537,
`resume/{latest,boundary_warmup_wikitext_step1145}.pt`, one event
file, and a report at the doubled path
`archive/run0-nospecials/mini-beatrix-2s/reports/v2/warmup_wikitext_step2253.json`
— a late-tagged fineweb report written on interrupt (stale tag, doubled
path: the 0.8.1 driver-contract bug) with probes, five census flags and
governor fields but no `ledger`/`special` block. Code-wins note: the
manual's RIDER 9 says run0 "halted at step 2000"; the archived manifest
says 2,537 (train() swallowed the interrupt and the driver auto-resumed
~19 minutes; `repos/alephllm.md:1408`). The hub wins: 2,537.

## The AutoModel (AbstractPhil/mini-beatrix-2s)

Thirteen files: `README.md`, `config.json`, `modeling_minibeatrix.py`,
the vendored `address.py`, `alephlm.py`, `attention.py`, `bank.py`,
`embedding.py`, `governor.py`, `head.py`, `presets.py`,
`model.safetensors` (475,880,498 B; 593 bf16 tensors, numel
237,908,285, every key prefixed `model.`, metadata `{tokens: 16.101B,
step: 61422, format: pt}`), `.gitattributes`. No tokenizer files: input
ids are raw UTF-8 byte values. The vendored files are byte-identical to
`geolip/alephllm/model/*.py` and `presets.py` at 0.8.7 except one
import line (`alephlm.py:30`, `..presets` → `.presets`). `config.json`
is the craft's `model_config` plus `model_type "mini-beatrix"`,
`auto_map` to `MiniBeatrixConfig`/`MiniBeatrixForCausalLM`,
`torch_dtype "float32"` (the file is bf16; the loader upcasts on
request), `use_cache false`. The wrapper (`modeling_minibeatrix.py:
21-99`) holds `self.model = AlephLM(cfg)`, forces `hub_ckpt = 0`, makes
`_init_weights` a no-op ("AlephLM initializes itself; loader must not
re-init"), truncates to the last `context` positions, returns
`CausalLMOutputWithPast(loss, logits)` with HF label semantics, and
carries no KV cache — HF `generate()` recomputes the prefix each step;
the inner `AlephLM.generate(idx, max_new, temperature, top_p,
use_cache=True)` uses the constant-size hub state.
**The parity gate.** The ship gate — strict load, logits parity 0.0
against the geolip code path, `generate()` ok — is journal-recorded
(`repos/alephllm.md:1840-1846`; `history/timeline.md` 2026-08-31;
`MIRROR.md` step 2 in the AlephLLM repo), not hub-shipped: no parity
script exists on either hub repo or in the package. Re-verified on CPU
by the drafting workflow (torch 2.11.0, transformers 5.13.0, no geolip
installed; not re-run in the hub-side pass, which found the hub
consistent with these readings — TB `head/w_s_norm` 515.13 at step
61,000, 593 bf16 tensors, numel 237,908,285): 593 file keys == 593 model keys, parameter count
237,125,949, ‖W_s‖ 515.918, γ 1.0; the inner cached decode equals the
parallel path (`torch.equal` True); logits parity against geolip was
not recomputed. Two notes for the reader: the model card's lineage line
says alephllm 0.8.6 while the vendored files are the 0.8.7 mirror
(commit 90a06b75, 2026-08-31 21:11 — the `presets.py` carries the 0.8.7
twin-isolation block); and the card's "17 reports" counts the 16
boundary/session reports plus the revival op — the prefix holds 21
JSONs with the four cultivation arms.

**Load and evaluate** (the steps below were executed by the drafting
workflow; the numbers are from that run and were not re-run in the
hub-side pass):

```python
import math, torch
from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained("AbstractPhil/mini-beatrix-2s",
                                         trust_remote_code=True).eval()
ids = torch.tensor([list(text.encode("utf-8"))])          # bytes in
bpb = m(input_ids=ids, labels=ids).loss.item() / math.log(2)
# toggle ledger on the inner model (positive delta = mechanism carries function)
core = m.model
full = core(ids, labels=ids).loss / math.log(2)
bank_off = core(ids, labels=ids, disable_bank=True).loss / math.log(2)
hub_off = core(ids, labels=ids, disable_hub=True).loss / math.log(2)
head_off = core(ids, labels=ids, disable_head_aleph=True).loss / math.log(2)
# generation: HF path (no cache) or the inner cached decode
out = core.generate(ids, max_new=40, temperature=0.0, use_cache=True)
print(bytes(out[0].tolist()).decode("utf-8", errors="replace"))
```

On a 227-byte English snippet this read 0.9446 bpb full, +3.872 bank
off, +3.925 hub off, +0.0156 head off; greedy from "The history of
astronomy begins with the" continued " study of the sun and the
planets. The s". Chat: wrap raw bytes in 0xFB SYS / 0xFE USER / 0xFD
MODEL / 0xFC END (`special_tokens.py:112-136`). Training checkpoints
load into the same wrapper after prefixing every key with `model.`, or
into a bare `AlephLM(cfg.to_aleph())`; fp8 files must be rehydrated
first (`checkpoint.py:187-195`). What the AutoModel cannot reproduce is
the gauge of record: the fineweb-edu holdout needs
`geolip.alephllm.data.streams.build_stream(dataset, tok, 4096, 16, seed
= 1337 + 9999, role="val")` and 262,144 tokens (4 batches of 16 × 4096),
then `toggle_ledger` and `special_token_gauge` over those fixed batches
(`trainer.py:384-411`; `instruments.py:187-212,310-352`).

## The process that made 2s different (dated)

**The mother, measured (2026-08-12 → 08-27).** mini-beatrix-1 was built
with 13 sdpa blocks and three hubs at layers 4/9/14, each a single
512-anchor book in D = 32 (`git show cc89a87:presets.py`), full-width
banks holding 67% of the mass, and a 512 @ 32 head. Two post-mortems set
the 2s agenda: the 08-25 hub census — 16× the K ≤ 2D supply, 333/646/181
duplicate anchor pairs at |cos| ~1.0, consumed erank 2.2/3.7/10.8 — and
the 08-27 autopsy of the shipped weights, W_s exactly 0.0 after 8.4B:
the "head election" lore retracted, the head never carried function
(`repos/alephllm.md:1586`; `history/plans/2026-08-26_mini_beatrix_v2_shape.md`).
A third: the v1 anneal (2B, chat template inside, before the school)
baked the chat frame into the core irreversibly — the chat-in-anneal
law (2026-08-15) that later dictated the bracketed two-phase anneal.
**Why every block (08-23 → 08-26).** Phil's directive: "the entire model
to be splat attention". The record's support was dist-e002 — cured
splat in all 12 ViT-Ti blocks ties sdpa at matched params and matched
wall-clock on two seeds (R@1 .0545 vs .0562), late placement edging
(R@5 +.0128, outside the bars); the untested bet, flagged in the plan,
was that full-splat CAUSAL had never trained. The payoff row was the
warmup ledger (+1.50 with the hubs off at 0.3B) and the endpoint
(+3.36).
**The shape (08-24/25 → 08-26).** ROUND 5e's shape ladder: 8 × 256 lost
4 of 4 paired against 12 × 128, the wide model seed-fragile, its
crowding tail (79% of layer-1 heads over the 25° floor on one seed)
naming the chain width → crowding → positional ranking flips → deficit;
hence SUPPLY K ≤ 2D (`STATUTES_INDEX.md:53`) and depth over width. B2's
product code (independent books compose, .859 → .955 monotone in
members) gave the multi-book form; the rotary × anchor flip analysis
gave governability D ≥ 128 (γ* 32° at d512, 45° at 256, unreachable at
64). The arithmetic lands on 4 × 64 @ 128 (0.5× supply, 4× headroom
against v1's 16×), L20, d1024, ctx 4096 (splat's wall-clock crossover
~2k; below 4k a full-splat trunk pays attention's price without the
O(L) payoff), banks narrowed to ff 1024, head 256 @ 256. The 849M
sibling (L32, ctx 8192, 16 × 256 @ 256) was designed the same session
(`presets.py:161-183`).

**The governor from birth (08-24 → 08-26).** Phil's proposal —
"a middleman to control a min/max space that drift can occur to
guarantee there's always enough space between the anchor points" — was
confirmed in the math: for a crowded pair the static logit gap is
quadratic in separation while the positional oscillation is linear, so
small separation flips weight rankings along the sequence; separating
an already-crowded trained book leaves its flip pattern bit-identical,
so min-sep is preventive, not curative — run it from birth. Rung 1 ran
bit-identical to the ungoverned run (raw books never crowd; the
crowder was the frame W_a); gov25m fired on the fragile seed only,
harmlessly. Shipped as a post-step projection, 45°, every 8 steps
(`governor.py`, `presets.py:99-105`). In-mission it ran hot — ~120
pushes/step at step 1,000, ~84–105 sustained through fineweb, never
decaying — with every function gauge healthy; the θ ladder 35/40/45 for
K = 64 @ D128 is an open rung, and the preset comment's "45 at D=256"
does not itself justify 45° at D = 128.
**The kernel campaign and the pivot (08-26, one day).** 0.7.0 at 10:41
(multi-constellation hub, supply guard, governor wired in, the 849M
rescale, 2s as the screen craft). The Blackwell preflight OOMed
(`hub_ckpt`), then launched at 161.88 s/step; the per-op profiler
printed `qc torch.float32` under autocast — 0.7.2 at 12:27 (the einsum
promote trap; 4090 hub fwd+bwd 229.7 → 178.8 ms). The dtype fix bought
~6%; the binding constraint was kernel shape — 0.7.3 at 13:14 (one
stacked GEMM, one fused softmax, prefix as a triangular GEMM; 65.3 ms,
3.5×; peak 7.4 → 1.9 GB; Blackwell 149 → 69.7 s/step). Compile refuted
by measurement (bf16-backward NaN, and the pre-NaN projection 67 vs
69.7 s eager) — eager-only on Blackwell. Economics: 17.3B at 8k tok/s
is 24–26 days on the one card. Phil: "we can't train the large one
currently. Not until we work out a series of logistics problems." 0.7.5
at 13:43 (@73218bd): 2s is the mission, `hub_ckpt` 2 → 0; launched at
4.54 s/step, 58k tok/s. "Not a bad start. We'll train this one." The
849M is shelved, not retired (its value-bottleneck relaunch lever
quantified at ~46 s/step).

**The format restart (08-26 16:43).** Phil: "We'll want a practical
array of them for usage, nothing small. We'll halt this one and start
over with the correct formatting." 0.8.0: the thirteen unreachable ids,
the exhaustive proof at every launch, DOC from birth, the frame only in
anneal, the canary collision caught; run0 archived whole, run1 fresh on
the clean prefix (manifest 08-27T00:35Z). DOC cost nothing — run1 train
bpb 1.49 @1,100 against run0's 1.494 @1,000 — and doc_bpb fell 12.83 →
3.29 by 0.52B. 0.8.1 the same day: the driver contract that made every
later boundary row a real boundary.
**The head saga (08-13 → 08-29 → 08-31).** Born weight-zero by law
(gate-zero deadlock, `head.py:8-11`); buried inside 2,000 steps on run1
(0.032× chance at the first stored checkpoint, 0.014× @12k — journal
readings from checkpoint analysis, not TB scalars; learned
avoidance — proj rotated into codebook-orthogonality while W_s grew to
norm 31 multiplying ~0); LS-ignite alone refuted (0.0006 bpb); the
governor exonerated (the ungoverned v1 buried deeper; the 45° floor is
free at birth). Candidate law at 2/2 crafts, 3/3 with the control: a
born-null aleph head racing a strong same-target linear base
self-buries within ~2k steps; the bank's self-start does not transfer
across competitor types. Phil ruled activation; the BOUNDARY-WRITE law
was minted (`revival.py:26`; `STATUTES_INDEX.md:67`) — a third
intervention category outside both the governor statute and the
no-balance-machinery law, permitted only at run boundaries or halted
resume points, always logged — and the deterministic op executed at
24,860. The first nonzero head toggle in lineage history landed at s1
(+0.0258, after 6.5B tokens of exactly 0.000); the settled read is a
durable, small, diet-sensitive niche in the band −.001..+.028,
band-exits upward on the binding-adjacent diets (s5 causal +.0278),
touching zero at s7 as bpb_full fell 1.268 → 1.188.
**The recipe that never changed (08-12 → 09-04).** Muon on the trunk
because the 08-10 validity battery on the AR bed read muon .8081 >
sgd_mom .7995 > adam .7157 > lion .5638, with the address ~20× more
optimizer-fragile than sdpa (cross-optimizer spread .79 vs .037) and
schedules no-op-to-harmful (flat .714 > warmcos .596) — so the flat-LR
law; pure Adam on the rest by the 09-01 optimizer statute (Adam the
experimental baseline, Muon the trunk, AdamW never;
`STATUTES_INDEX.md:76`). `presets.py:80-84` is the same text at
cc89a87 (where it sits at `presets.py:65-69`) and at HEAD. The same recipe is the audit's confound (a).

**The diet (08-15 → 08-31).** The nine stages and budgets are v1's
(0.6.0), placed before the anneal rather than after it (Phil,
2026-08-30: "Let her really build up that internal knowledge first";
`history/plans/2026-08-26_mini_beatrix_2s_mission.md`), fineweb_extended
deferred; the two-phase anneal (Phil, 08-31: "enable the anneal mix
without chat and the anneal mix with chat, we'll run without first,
then just train chat on after"; `streams.py:63-77`), 0.8.6 at 02:48.
The s8 terminal-gauge hole appeared at 04:57Z and 0.8.5 had already
closed it in code (08-30 22:03); the true ledger was re-measured at
09:53Z. Mission complete 19:31Z: 61,422 steps, 16.101B tokens, 78.33 h,
zero spikes; the HF outage at the final push was recovered by a Drive
rescue and retry.
**The twin (08-31 → 09-04).** 0.8.7 at 14:10 (twin isolation, birth
recipe); the control born 21:26Z; the protocol declared at its warmup
boundary while sdpa led; the crossover at fineweb 5.3B and every row
after; the rank story as the mechanism claim (at step 4,000 the control
collapses early hidden erank to 5–9 of 1024 at L2–L4 (5.5 / 5.4 / 9.3)
while the craft reads 234 / 225 / 250 there and 150–396 across L0–L15
(`erank/hidden_L{i}`, TB census); screen scale reproduces it; rank ≠ function — the
control led val at 4k while running rank-5 layers); the H100 resume at
22,854; the grad-norm escalation rider at s5; Phil's claim held on
09-02 ("SDPA isn't an adequate representative of this model's
attention requirement. We can say that without question now."); the
third host from 56,000; the endpoint 09-03; the 12-agent audit on 09-04
(S-1802 INSTANCE, S-1803 CANDIDATE, the s8 strike, the mother null,
Muon clip-blind, the falsifier filed).

## Gauges, defined once

- **val bpb**: mean cross-entropy in bits/byte over 262,144 tokens
  (4 batches of 16 × 4096) of the fineweb-edu reserved head slice —
  the first 2,048 rows, fixed, unshuffled, seed 1337 + 9999 — for every
  phase from fineweb_main onward, both crafts (`trainer.py:384-411`;
  `streams.py:279-282,423-447`; gauge continuity law). The warmup row
  vals on wikitext-103's validation split. Pre- and post-0.8.0 gauges
  differ for v1-era crafts (DOC rides the val stream).
- **toggle ledger**: bpb with one mechanism disabled minus full bpb —
  the causal contribution of banks / hubs / head aleph
  (`instruments.py:187-212`). Positive = the mechanism carries function.
- **specials gauge**: `doc_bpb` and `reset_bpb` in bits, read against
  the same val batches (`instruments.py:310-352`); `doc_count` 70 on
  every 2s-era report on the fineweb-edu gauge (the struck 2s s8 file reads 1612).
- **head liveness**: mean ‖signed read‖ of real hidden states over the
  same book's mean on 512 random unit vectors — self-calibrating;
  `head_buried` = ratio < 0.1 (`instruments.py:144,176,356-382`).
  Both buried crafts sat at 0.003×–0.03× chance.
- **probe accuracy**: byte-NLL argmin over the item's options, nine
  suites × 30 items, holdout families d5 and sub3 gauged from time zero
  (`probes.py:171-235`). At n = 30 a between-craft difference has SE
  .126: the paired probe calls are ties.
- **train/grad_norm**: the pre-clip global norm returned by
  `clip_grad_norm_` (`trainer.py:244,287`), logged every 50 steps; under
  Muon it measures sharpness, not the applied step.
- **governor hits**: cumulative pair-pushes per session
  (`trainer.py:266`); a session-scoped counter, comparable within one
  event file only.
## Known caveats and open items
- The autocast einsum-promote trap voids every pre-0.7.2 splat speed
  verdict (fp32 scans under "bf16"); accuracy verdicts stand. Any
  autocast performance reading over einsum code is invalid until the
  dtypes are printed (`attention.py:167`).
- Muon is clip-blind (`optim.py:24`), and Adam's per-coordinate
  normalisation makes it nearly so: `grad_clip 1.0` shrank the applied
  step of neither group on the control, only the weighting of steps
  inside each optimizer's buffers; the recipe carries no softmax
  stabiliser.
- The 2s `curriculum_s8_step53792` ledger is on the synthetic fallback
  and is struck; the same-gauge pair is owed by a zero-training
  re-evaluation of `resume/boundary_curriculum_s8_step53792.pt` on the
  0.8.7 gauge (T0b in `history/plans/2026-09-04_sdpa_falsifier_screen.md`).
  Hub-vs-journal note: the file on the hub reads doc_count 1612,
  bpb_hub_off 6.2378 (toggle_hub_off 5.164), doc_bpb 0.843 — agreeing
  with the trainer comment and the manual (doc_count 1612, hub_off 5.16 =
  the toggle; `trainer.py:397-399`); the file carries no dataset field,
  so the strike rests on the same-step cap discrepancy (doc_count 70,
  bpb_full 1.1911 in `session_cap_step53792.json`) and on the 1612-document
  count itself. The hub file's numbers win for what the file says.
- The control's third host from 56,000: a replay of up to 999 steps is a
  TB-cadence hedge, not a hub reading; no cap report; the 53,800–56,000 segment ran in the H100
  session's own process (event file `0a706d592fec.620.1`); canon CB11/12 "on the H100" wrong by erratum.
- Every resume refills the shuffle buffer (not row-exact); the control
  resumed three times, the craft four; MixStream state carries no
  recipe fingerprint.
- The 2s preset at HEAD carries `head_addr_frozen = True`
  (`presets.py:199-201`), the post-revival state: a fresh birth of the
  treatment preset under 0.8.7 would freeze a random proj and codebook
  from step 0, which is not what the shipped run did (born-null unfrozen
  under 0.8.1 for 24,860 steps, revived, then frozen). The control's
  override (`presets.py:235`) is the birth recipe.
- Stale text, code wins: `presets.py:18` "~233M" (built 237,125,949);
  `presets.py:14,166` "~873.7M" for mini-beatrix-2 (built 849,023,425);
  the training README's ladder row "mini-beatrix-2 | 1024/20/2048 |
  249.1M" is the retired v1 shape; the hub driver `beatrix_v2_colab.ipynb`
  is headed "mini-beatrix-2 — the governed full-splat mission" and names
  `mini-beatrix-2-control` while its CRAFT is `mini-beatrix-2s` (no
  control notebook exists on the hub — only this file and
  `mini-beatrix-1/beatrix_curriculum.ipynb`); `AlephLMConfig.hub_chunk` defaults to
  128 while the module's own default is 256 (`presets.py:52`;
  `attention.py:83`) — the config governs, and 2s sets 256.
- `governor_hits_cum` resets per session; the manifests record no
  train_config or package version — the mother null's "identical
  TrainConfig" is a git fact, and the craft's birth version 0.8.1 is
  inferred from commit dates (run1 created 08-27T00:35Z between 0.8.1
  on 08-26 and 0.8.2 on 08-29; the revival report records 0.8.2; the hub
  driver `beatrix_v2_colab.ipynb` pins `geolip-alephllm @2a902b7`, the
  0.8.1 commit).
- Owed instruments: the per-layer hidden-erank census of both crafts at
  the full-scale boundaries; the falsifier S-1803-F (Tier 0 on the 4090;
  a 40M twin screen with sdpa + QK-norm / LR/3 / dense-FFN / positions
  on Adam arms; pivot = resume the control at 20,219 through s0 at LR/3
  with QK-norm). Without it the general claim stays CANDIDATE.
- `article_assets_ft2/make_charts_ft2.py` sources chart 01 from the hub
  but charts 02–05 from local scratch paths, and chart 01 pairs the
  struck s8 rows unless filtered.
## Provenance discipline
Both crafts are resume-first: the manifest is embedded in the resume
payload (atomic consistency), a crash never overwrites
`resume/latest.pt` (`trainer.py:334-368`: any exception other than a
manual stop leaves the hub state untouched), an interrupted session
uploads before exit, and every phase boundary archives its own
immutable resume. The revival is auditable from the prefix alone: the
pre-write state (`resume/pre_revival_step24860.pt`), the op's
provenance (`reports/v2/revival_step24860.json`, with version and
commit), the pre- and post-write weights (`step_00024860` and
`step_00024860_revived`), and the first live row after it
(`curriculum_s1_step25561.json`). The struck s8 ledger stays public
beside the cap that supersedes it, as run0 stays public under
`archive/`, for the same reason the mother's rewind checkpoints do. The
recipe is pinned from source, not from the manifests. The AutoModel
vendors the model files verbatim and ships the mission-final weights
re-prefixed; the training checkpoints, not the card, are the control's
weights. Every refuted arm of the head-cultivation screen ships.
## Pointers
- Canon: `canon/splat_aleph_battery.md` — CONTROL BOUNDARY 1–12
  (522–889), ENDPOINT VERDICT (891), CLAIM AUDIT (901–1006: the s8
  strike 930–937, MUON IS CLIP-BLIND 936, THE MOTHER NULL 955–968,
  PROVENANCE RIDER 1008), ROUND 5a/5d/5e, B2/B4/B5.
- Statutes (`STATUTES_INDEX.md`): S-701 a control must be able to fail;
  S-801 supply law K ≤ 2D; S-1001 gate-zero deadlock; S-1002 born-null
  head self-burial 3/3; S-1003 BOUNDARY-WRITE; S-1201 optimizer family;
  S-1202 autocast einsum-promote; S-1801 the crossover; S-1802 the
  destabilised control (INSTANCE); S-1803 sdpa-inadequacy (CANDIDATE);
  S-1901 the specials plane.
- Manual: `repos/alephllm.md` — the 0.7.0 era (1180–1330) and RIDERS
  7–21b (1333–2081): pivot, launch health, specials and the run0
  archive, driver contract, the dead head, the revival trial, the
  boundary-write and promote, the head trend 14–14l, curriculum complete
  and the s8 artifact, the pre-chat baseline, mission complete, the
  ship-out, twin isolation, the pre-registered protocol.
- Plans: `history/plans/2026-08-26_mini_beatrix_v2_shape.md` (the
  shape arithmetic), `2026-08-26_mini_beatrix_2s_mission.md` (the pivot,
  the anneal-timing ruling, the two-phase amendment),
  `2026-08-24_covariant_address_program.md` (the governor),
  `2026-09-02_sdpa_inadequacy_claim_held.md`,
  `2026-09-04_sdpa_falsifier_screen.md`.
- Dockets: `history/dockets/2026-09-04_sdpa_claim_audit/`,
  `2026-09-01_ft2_article/` and `2026-09-01_article/tools/`
  (`writeback_cb*.py`, `writeback_endpoint.py`).
- Article: `articles/article_beatrix_ft2.md`; charts under
  `article_assets_ft2/` on the training repo.
- Code: AlephLLM HEAD 4f4b7d6 (0.8.7); the era's commits a63a29b
  (0.7.0) · 2be28dd (0.7.2) · 60ee496 (0.7.3) · 73218bd (0.7.5, the
  pivot) · 3ea3103 (0.8.0) · 2a902b7 (0.8.1) · d6e1854 (0.8.2) ·
  7d48328 (0.8.5) · af9319c (0.8.6) · b472815 (0.8.7). No git tags
  exist; versions are commit subjects.
- The mother: `TECHNICAL.md` on the training repo (byte-identical local
  copy `articles/TECHNICAL_training_repo_beatrix.md`).