Hugging Face's logo

AbstractPhil
/
mini-beatrix-2s

Text Generation
Safetensors
English
mini-beatrix
byte-level
linear-attention
splat-attention
aleph
custom_code
Model card Files
xet
 Community
mini-beatrix-2s / TECHNICAL.md
AbstractPhil's picture
AbstractPhil
TECHNICAL companion for the 2s era (verified code-side + hub-side, 2026-09-04)
907da49 verified
preview code
|
Raw History Blame Contribute Delete
65.9 kB

alephllm-mini-beatrix-training β€” Technical Companion (the 2s era: the full-splat craft and its softmax twin)

Companion to Twinning Beatrix: A Full-Splat Byte Model, Its Softmax Control, and What Reaches an Image Generator (AbstractPhil with Claude Fable 5 and Claude Fable 5.1, September 2026). It covers two crafts and one packaged model: mini-beatrix-2s/ and mini-beatrix-2s-control/ under AbstractPhil/alephllm-mini-beatrix-training, and the AutoModel at AbstractPhil/mini-beatrix-2s; the dates run from the first launch (run0, 2026-08-26) through the control's endpoint (2026-09-03) and the claim audit that graded it (2026-09-04). It is written for a reader who will load the checkpoints and read the reports. It is not the mini-beatrix-1 life record (TECHNICAL.md, which stays as it is), not the article's argument, and not a manual for the side campaigns that share the repo root (reports/v2/{battery,wide_moe,aflow,bcd} appear only as a catalogue line). Every number carries its pointer: file:line means E:/mirel/AlephLLM/geolip/alephllm/ at HEAD 4f4b7d6 (package 0.8.7, the last commit of the 2s era); hub paths are relative to the craft prefix; journal paths are relative to the claude-mind repo. Where the journal and the code disagree, the code wins and the disagreement is reported in place. Stack-side detail lives in the AlephLLM companion, which covers the 0.7.0 mechanisms only; the 0.8.x organs (specials, driver contract, revival, gauge fallback, twin isolation) are carried here.

The two crafts at a glance

mini-beatrix-2s mini-beatrix-2s-control mini-beatrix-1
d_model / layers / heads 1024 / 20 / 16 1024 / 20 / 16 768 / 16 / 12
attention CausalSplatHUB in all 20 blocks CausalSDPA (16 Γ— 64) in all 20 13 sdpa + 3 hubs (layers 4, 9, 14)
hub books Γ— anchors @ D 4 Γ— 64 @ 128 (supply 0.5Γ—) β€” 1 Γ— 512 @ 32 (supply 16Γ—)
banks 3 experts @ ff 1024 3 @ 1024 3 @ 768 (full width)
head (K @ D) 256 @ 256, revived at 24,860, address frozen after 256 @ 256, born null, never elected 512 @ 32, never elected (W_s = 0.0 at ship)
context / vocab 4096 / 256 byte-trigram 4096 / 256 2048 / 256
params 237,125,949 257,442,109 (+20,316,160, all attention) 112,524,593
tokens per step 16 Γ— 4 Γ— 4096 = 262,144 262,144 48 Γ— 3 Γ— 2048 = 294,912
steps / tokens 61,422 / 16,101,408,768 61,422 / 16,101,408,768 88,508 / 26,102,071,296
wall-hours / hosts 78.33 on one RTX PRO 6000 49.98 on three hosts 70.45
final val bpb (report ledger) 1.1097 2.8846 1.121
optimizer Muon 2e-2 + pure Adam 3e-4, flat after 200 same same
governor minsep 45Β°, every 8 steps, from birth inherited (head book only) off
package at birth β†’ end 0.8.1 β†’ 0.8.6 (revival under 0.8.2) 0.8.7 throughout cc89a87 β†’ 0.6.0 (pre-specials)

Shape and knobs: presets.py:191-203 (2s), :214-235 (the -control derivation and the head_addr_frozen override), :155-160 (mb1). Parameter counts were computed by constructing each preset on the meta device (alephlm.py:218-224 param_count): 2s = embed 4,982,784 + 20 blocks Γ— 11,574,275 (hub attention 3,178,496 + bank 8,391,683 + norms 4,096) + final norm 2,048 + head 655,617; the control's block swaps the hub's 3,178,496 for sdpa's 4,194,304 (qkv 3072Γ—1024 + o 1024Γ—1024, no biases; attention.py:24-32). Tokens per step is micro_batch Γ— grad_accum Γ— context (trainer.py:186); 61,422 Γ— 262,144 lands on both manifests' tokens_seen exactly. The mother's numbers are the exemplar's (TECHNICAL.md, mini-beatrix-1/manifest.json).

Layout 

mini-beatrix-2s/
  manifest.json                  # 14 phases, 142 checkpoint entries, notes
  checkpoints/step_XXXXXXXX.safetensors   # 46 bf16 (2000-cadence + every boundary and cap)
  checkpoints/step_00024860_revived.safetensors   # fp32, the post-BOUNDARY-WRITE head
  checkpoints/fp8/               # 20 e4m3 ship-only copies
  resume/latest.pt               # fp32 model + Muon/Adam + stream + RNG + manifest
  resume/boundary_<phase>_step<N>.pt      # 13 immutable boundary archives
  resume/pre_revival_step24860.pt         # the state the revival overwrote
  reports/v2/                    # 13 boundary + 3 session_cap + revival + 4 cultivation
  runs/                          # 5 TensorBoard event files (one per session)
  archive/run0-nospecials/       # the pre-specials first attempt, whole
  beatrix_v2_colab.ipynb         # the driver

mini-beatrix-2s-control/
  manifest.json                  # identical phases and budgets; 136 entries
  checkpoints/step_XXXXXXXX.safetensors   # 44 bf16 (incl. the 22,854 cap)
  checkpoints/fp8/               # 20 e4m3 ship-only copies
  resume/latest.pt  resume/boundary_*.pt  # latest + 13 boundary archives
  reports/v2/                    # 13 boundary + 2 session_cap (22,854; 53,792)
  runs/                          # 6 event files (two are 88-byte aborted starts)

The control has no notebook file and no AutoModel packaging; these checkpoints are its weights. Files: bf16 475,876,922 B (2s) and 515,164,234 B (control); fp8 238,219,518 / 257,813,142 B; resume archives ~1.904 / ~2.064 GB each; the revived fp32 file 951,692,996 B (hub headers read by ranged GET; checkpoint.py:102-128 for the two export rules).

Architecture in detail

The block. Pre-norm, two residual branches: x = x + attn(n1(x)), x = x + bank(n2(x)) (alephlm.py:37-70). attn is CausalSplatHUB when the layer index is in hub_layers, else CausalSDPA (alephlm.py:41-50); every block carries an AnchoredBank (alephlm.py:51). hub_ckpt (0 on 2s: the retained scan fits the 96 GB card at 237M, presets.py:184-198) would wrap the branches in activation checkpointing (alephlm.py:60-68). The model is embedding β†’ 20 blocks β†’ final LayerNorm β†’ DualHead (alephlm.py:81-96); forward takes pre-shifted targets (ours) or HF-style labels (shifted internally) and ignores attention_mask deliberately (alephlm.py:98-124).

Trigram embedding and positions. e_t = emb0[x_t] + emb1[x_{t-1}] + emb2[x_{t-2}] + pos[t]; the shift tables have a 257th PAD_ROW so the two positions shifted in at the sequence start never read a legal byte (embedding.py:19-34; the pad law β€” padding with a real byte starved address consumption, measured +.05..+.11 on repair). pos is a learned absolute table (1, 4096, 1024) (embedding.py:28); there is no RoPE.

CausalSplatHUB β€” the address. Each book is an AlephAddress: K unit anchors in D dims (address.py:24-36, with a home buffer holding the birth copy for drift). The read is a cosine at temperature, u_k = cos(xΜ‚, Γ’_k)/Ο„ with Ο„ = 0.1 (address.py:38-40); the oriented form is the 2K-softmax over the axes (+a_k, βˆ’a_k), and the signed read is its half-difference w_k = sinh(u_k)/Ξ£_j cosh(u_j) β€” inhibition (negative w) is first-class; there is no softmax over choices, no argmax, no top-k (address.py:1-13,42-66). The hub's q and k frames are Linear(d, D) per book, orthogonal-init; v and o are shared Linear(d, d) (attention.py:64-79,106-109). Construction warns when K > 2D β€” the supply law from ROUND 5e and the mini-beatrix-1 hub census (attention.py:85-92); 2s runs 64 @ 128, half of the lawful ceiling.

CausalSplatHUB β€” books and budget. With n_const = 4 the four books each own a codebook and q/k frames (attention.py:64-79); their numerators and agreement masses are summed across books before the one divide β€” composition by budget, never softmax over books (attention.py:245-254; B4 measured comparative composition at βˆ’.10). n_const = 1 is the v1 layout with the same state-dict keys (attention.py:93-101), which is how mini-beatrix-1 checkpoints still load.

CausalSplatHUB β€” the exact chunked scan (prefix memory). _code_cat_qk computes q and k for all books in one stacked GEMM, folds the row-norm and Ο„ into one post-GEMM scale, and takes ONE fused softmax over cat([u, βˆ’u]) β€” the module's docstring names this a kernel substitution, not a mechanism change: the softmax that exists is over the 2K oriented axes of a single query, never over positions (attention.py:154-212). _scan_cat splits the sequence into chunks of C = 256 (16 chunks at ctx 4096), forms per-chunk (2K Γ— d) sums, takes the exclusive prefix over chunks as a strictly-lower-triangular GEMM with fp32 accumulation, adds the within-chunk causal affinity, and divides the numerator by the scalar agreement mass den, floored at dtype_floor (attention.py:139-151,229-281; address.py:16-22: fp16 flushes 1e-12 to zero, the measured landmine). Exactness is chunk-independent (forward_naive is the oracle, attention.py:320; fp reorder ~1.5e-06). Decode carries a constant-size state per book β€” {Ξ£ kβΊβŠ—v, Ξ£ kβ»βŠ—v, Ξ£ k⁺, Ξ£ k⁻} β€” regardless of length (attention.py:284-318); the generate path falls back to a sliding window once the position table is exhausted because "the hub's constant-size state cannot evict" (alephlm.py:180-201).

The autocast trap. torch.einsum sits in autocast's PROMOTE category and F.normalize/exp/softmax on its fp32 list, so one fp32 operand dragged the whole scan to fp32 (attention.py:167); 0.7.2 (@2be28dd) cast the operands explicitly. Every earlier "bf16" splat run had silently run fp32 scans; sdpa was unaffected β€” so every pre-0.7.2 splat-vs-sdpa wall-clock comparison favoured sdpa, and accuracy verdicts stand (autocast einsum-promote law).

The governor. minsep_project_ bounds |cos(a_i, a_j)| ≀ cos(ΞΈ_min) for every anchor pair by an exact symmetric geodesic push-apart, largest violation first, projected to ΞΈ_min + 0.5Β° (governor.py:35-80); govern_model collects hub and head codebooks, runs one batched bmm slack check with one device sync, and projects only the violating books (governor.py:92-137). It is a post-optimizer-step projection, never an auxiliary loss (governor.py:11); the trainer calls it every 8 steps after opt.step and accumulates pair-pushes into governor/hits_cum (trainer.py:263-267,294-296; presets.py:99-105: ΞΈ = 45Β°). Banks are excluded by default (a 3-anchor book never crowds). The docstring records the small-rig verdict it shipped on (gov25m: fires only when needed, census tail cut 5.4Γ—, fragile-seed +.007, stable-seed +.002; governor.py:19). Code-wins note: the journal's covariance program describes a metric governor through the frame W_a; alephllm's is the plain projective form on the D-space books, justified in place β€” "Hub and head codebooks live natively in their own D-dim address space" (governor.py:14), since the q/k frames project d β†’ D before the cosine read (address.py:38-40).

CausalSDPA in the twin. qkv = Linear(d, 3d), o = Linear(d, d), both orthogonal-init and bias-free; F.scaled_dot_product_attention(q, k, v, is_causal=True) over 16 heads of 64 (attention.py:24-42); the KV cache grows O(n) (attention.py:43-61). No QK-norm, no logit softcap.

AnchoredBank. Bank(x) = trunk(x) + Ξ£_k w_k(x)Β·Οƒ(g_k)Β·expert_k(x) (bank.py:3): a dense trunk t_out(GELU(t_in(x))), three experts (3, 1024, 1024) with w_out zero-initialised (the null path), gates at βˆ’3.0, and dispatch w = addr.signed(x) through a 3-anchor book in d dims. disable_dispatch returns the trunk alone β€” bit-identical to the dense control at birth (the C6 null); "No balance machinery of any kind" (bank.py:10,41-50).

DualHead. logits = W_h h + Ξ³Β·W_s s(h), s(h) = addr.signed(proj(h)) with proj Linear(1024, 256), a 256 @ 256 book, W_s zero-initialised ("weight-zero: self-starting") and Ξ³ = 1 frozen as an ablation knob (head.py:23-51). The docstring carries the law that set it: on mini-beatrix-1 over 8.4B tokens a gamma-gated branch DEADLOCKS while a weight-zero bank elected into +2 bpb (head.py:8-11; gate-zero deadlock law). disable_aleph returns W_h h alone β€” the head toggle.

The special-token control plane. Thirteen ids that no UTF-8 encoder can emit β€” {0xC0, 0xC1} βˆͺ [0xF5, 0xFF], the overlong leads and the leads past U+10FFFF β€” carry structure inside vocab 256: DOC 0xFF, USER 0xFE, MODEL 0xFD, END 0xFC, SYS 0xFB, THINK 0xFA, DATA 0xF9, SEP 0xF8, MODE 0xF7 (+1 ASCII payload), CUE 0xF6, ESC 0xF5 (+payload = 256 extended slots; 0x00 the null guard, 0x01 CANARY), RES0 0xC0, RES1 0xC1 (special_tokens.py:20-34,50-71; assert len(SPECIALS) == 13). The proof is exhaustive β€” every codepoint 0..0x10FFFF encoded and checked β€” and the Trainer runs it at every byte-craft launch (trainer.py:72-73). Doctrine: DOC is appended by the packer after every document in every phase from birth, identity-gated to the byte-trigram tokenizer (streams.py:59-62,209,355-356); the chat frame [SYS] text [END] [USER] text [END] [MODEL] text [END] (special_tokens.py:112-136) is unforgeable (content is UTF-8) and appears only in the anneal (special_tokens.py:36-44; chat-in-anneal law). The canary frame moved to CUE/SEP because the old QRY=255 collided with DOC (canaries.py:20-29); recall_acc re-baselines at 0.8.0. The trigram embedding feeds each special into the next two positions' input representations by construction.

The recipe (pinned from source; unchanged cc89a87 β†’ b472815)

  • Split: every parameter with ndim ∈ {2, 3} that is not an nn.Embedding weight rides Muon; everything else β€” the three byte tables, LayerNorms, bank gates, Ξ³, the head bias β€” rides torch.optim.Adam(weight_decay=0.0) (optim.py:54-79; "pure Adam, wd=0 β€” never AdamW", presets.py:82). Measured split for 2s: Muon 385 tensors / 236,253,184 numel, Adam 107 / 872,765; the control's Muon set is 145 / 256,569,344 with the same Adam set. Code-wins note: the optim.py:4-5,55-58 docstrings say positions ride Adam; the code sends embed.pos (shape (1, 4096, 1024), ndim 3, not an nn.Embedding) to Muon (optim.py:68), verified on the meta device. The journal already records the code's behaviour.
  • Muon: 5-step Newton–Schulz (a, b, c = 3.4445, βˆ’4.7750, 2.0315) on the momentum buffer after whole-matrix Frobenius normalisation X/(β€–Xβ€–_F + 1e-7); plain momentum 0.95, no Nesterov, no shape scaling; 3-D stacks orthogonalised per slice (optim.py:20-51).
  • Learning rates: Muon 2e-2, Adam 3e-4; lr_scale = (step+1)/200 for 200 steps, then 1.0 forever (optim.py:82-93, presets.py:80-83). The tfevents read train/lr_scale 0.25/0.50/0.75 at steps 50/100/150 and 1.000 from step 200 to the last log on both crafts, through both anneal phases: the anneal is a diet change, not a decay.
  • Clip: clip_grad_norm_(…, 1.0) over all parameters; the logged train/grad_norm is the PRE-clip norm it returns (trainer.py:244, 287). Because Muon normalises the momentum buffer before orthogonalising, its update magnitude is independent of gradient scale: clipping never changes the magnitude of a Muon step; it only reweights steps inside the momentum buffer. The Adam set (torch.optim.Adam, wd 0, default eps 1e-8; optim.py:79) is per-coordinate scale-normalised (m/sqrt(v)), so under a persistent clip its step magnitude is likewise unchanged and the clip only reweights the two moment averages (Muon is clip-blind; Adam is nearly so).
  • Precision and batch: fp32 masters, bf16 autocast on CUDA, TF32 on (trainer.py:14-15,65-66,239); micro-batch 16 Γ— accum 4 Γ— ctx 4096; seed 1337 (presets.py:97,199); each micro-batch row is 4097 ids scored at 4096 positions.
  • Guards: a non-finite loss or gradient raises before any optimizer step, weights untouched and the resume state not overwritten (trainer.py:250); the spike flag loss > 2Β·EMA + 0.5 is logged only (trainer.py:273-275,293). Neither fired on either craft.
  • Cadences: log 50, health census 500, eval and checkpoint 2000, fp8 every 5th checkpoint, TB upload 1000, val 262,144 tokens, 128 canary episodes (presets.py:88-96).
  • Absent by design: no QK-norm, no logit softcap, no z-loss, no weight decay, no dropout, no RoPE, no LR decay in any phase.

The diet (all on one gauge: fineweb-edu holdout bpb)

phase dataset budget β†’ step 2s bpb control bpb note
warmup_wikitext wikitext-103 0.30B 1,145 2.3626 2.1056 sdpa led by 0.257; protocol pre-registered here
fineweb_main fineweb-edu 5.0B 20,219 1.1172 1.1763 the crossover; hubs +2.22 / banks +3.55
fineweb_extended fineweb-edu 10B β€” β€” β€” DEFERRED, unspent on both
curriculum_s0 curriculum-s0 0.7B 22,890 1.2522 1.5047
REVIVAL β€” β€” 24,860 1.2768 β†’ 1.2339 (op gauge) β€” BOUNDARY-WRITE; pre_revival_step24860.pt archived
curriculum_s1 curriculum-s1 0.7B 25,561 1.2544 1.3917 first live head toggle in the lineage, +0.0258
curriculum_s2 curriculum-s2 1.0B 29,376 1.2177 1.3938 2s grad-norm curriculum+anneal max 0.619 @26,750
curriculum_s3 curriculum-s3 1.2B 33,954 1.2318 1.5317
curriculum_s4 curriculum-s4 1.2B 38,532 1.2356 1.5689 both 2s mechanisms above +3.0
curriculum_s5 curriculum-s5 1.0B 42,347 1.2504 2.1762 control grad-norm median 126
curriculum_s6 curriculum-s6 0.8B 45,399 1.2683 3.1940 control max 41,070 @44,600
curriculum_s7 curriculum-s7 1.4B 50,740 1.1883 3.1201
curriculum_s8 curriculum-s8 0.8B 53,792 1.0737 (synthetic gauge) / 1.1911 (cap) 3.0810 2s row STRUCK; true ledger = session_cap_step53792
anneal_nochat anneal-nochat 1.0B 57,607 1.1153 2.9270 the pre-chat baseline; probe mean .389
anneal_mix anneal-mix 1.0B 61,422 1.1097 2.8846 run best; probe mean .389 == pre-chat

Phases and budgets: presets.py:132-146 (the preset plan carries anneal_mix at 2B deferred; the notebook's C6 cell rewrote it to 1B and inserted anneal_nochat before it β€” "two-phase anneal activated: nochat 1B then chat 1B (plan of record 2026-08-31)" in both manifests' notes; the hub copy of beatrix_v2_colab.ipynb, uploaded 2026-08-28T23:00Z with cells C1–C5 and pinned to geolip-alephllm @2a902b7 (0.8.1), predates the amendment and carries no C6 cell), curriculum.py:657-663 (STAGE_TOKENS), curriculum.py:666-689 (append at runtime). Manifest order is not chronology: the S0–S8 phases are listed after the anneals but ran before them, because current_phase() activates the first planned phase and deferred phases are invisible until flipped (manifest.py:33-41); count back from the totals (16,101,408,768 βˆ’ anneal_mix's 1,000,079,360 lands on step 57,607) rather than reading the list.

The mixes (curriculum.py:614-655; weights sum to 1): s0 tinystories .74, simple-wiki .20, aochildes .01, recall-synth .03, counting-synth .02 Β· s1 perspective-synth .30, tinystories .28, simple-wiki .20, cosmo-young .12, concept-synth .04, recall-synth .03, siqa-narrative .02, aochildes .01 Β· s2 concept-synth .30, cosmo-young .30, simple-wiki .12, tinystories .10, definitions .08, wikipedia-en .07, recall-synth .03 Β· s3 rulechain-synth .32, proofwriter-prose .22, simple-wiki .12, tinystories .10, wikipedia-en .08, cosmo-young .08, concept-synth .04, recall-synth .03, babi-prose .01 Β· s4 arith-synth .35, cosmo-young .32, simple-wiki .12, tinystories .08, rulechain-synth .06, counting-synth .04, recall-synth .03 Β· s5 causal-synth .32, atomic-flicker .25, tinystories .12, simple-wiki .10, gutenberg .08, cosmo-young .07, recall-synth .05, siqa-narrative .01 Β· s6 tryfail-synth .32, tinystories .22, arith-synth .13, cosmo-young .10, simple-wiki .10, causal-synth .08, recall-synth .05 Β· s7 tinystories .10, cosmo-young .10, rulechain-synth .10, arith-synth .10, fineweb-good .10, simple-wiki .08, atomic-flicker .08, causal-synth .08, tryfail-synth .08, proofwriter-prose .08, recall-synth .05, perspective-synth .04, babi-prose .01 Β· s8 definitions .25, register-synth .25, simple-wiki .15, cosmo-young .12, wikipedia-en .10, tinystories .09, recall-synth .03, siqa-narrative .01. Rails audited at append time: MAX_EPOCHS 4.0, MAX_GENERATOR_SHARE 0.35, MIN_BALLAST 0.30 natural text (curriculum.py:548-551,565-596); holdouts enforced at the generator β€” bAbI tasks 16/19, ProofWriter depth 5, rule-chain depth 5, three-digit subtraction (curriculum.py:26-31). No chat template in s0–s8.

The anneal recipes (streams.py:68-77): ANNEAL_MIX = fineweb-edu .45, cosmopedia .20, tinystories .15, soda-dialogue .12, beatrix-texture-sp .05 (the specials-native SYS/USER/MODEL/END identity texture), recall-synth .03; ANNEAL_NOCHAT is the same list minus beatrix-texture-sp, renormalised by MixStream β€” so the nochat boundary is the exact pre-chat baseline on the same distribution. MixStream draws one component per batch from a counter-seeded RNG (streams.py:375-404): reproducible and resumable, but its state carries no recipe fingerprint (fresh starts only across recipe changes).

The DOC plane on the diet. The packer appends DOC after every document (streams.py:192-372); the specials gauge reads doc_bpb (bits at positions whose target is DOC) and reset_bpb (mean bits over the 16 targets after each DOC) on the fixed val batches (instruments.py:310-352). doc_bpb fell 12.83 β†’ 3.29 by 0.52B and sits at 3.862 at the endpoint (control 5.422); reset_bpb 4.287 β†’ 2.130 (control 3.825). doc_count is 70 in every 2s and control report except the struck 2s curriculum_s8_step53792.json, which reads doc_count 1612 (the synthetic fallback stream) β€” the same 70 documents every time on the fineweb-edu gauge.

The val gauge. _val() builds 4 batches of (16, 4097) from build_stream(dataset, …, seed = 1337 + 9999, role="val") (trainer.py:384-411); for fineweb-edu the val region is the first 2,048 unshuffled rows, which the training stream skips (streams.py:56,279-282), and build_stream routes every curriculum-* and anneal-* val request to that same fineweb-edu head holdout β€” "gauge continuity" in the source (streams.py:423-447). So from fineweb_main onward every row of both crafts is one gauge; the canon's CB2/CB3 "own slice" wording is struck by erratum (canon/splat_aleph_battery.md:583). The s8 fallback caveat: at the terminal curriculum boundary current_phase() is None (everything done or deferred) and pre-0.8.5 _val fell to the synthetic stream, so the 2s curriculum_s8_step53792 ledger is an incomparable artifact; 0.8.5 (@7d48328) makes _val fall back to the last DONE phase's dataset (trainer.py:397-401), and the true end-of-curriculum ledger is session_cap_step53792, taken at the same step on the restored gauge (1.1911; banks +3.214, hubs +3.341, head +0.0149). The control, born under 0.8.7, measured its s8 row on the repaired gauge; the s8 pair is therefore not same-gauge and is struck from the paired count.

The life (reports/v2/, resume/, runs/)

Sessions. The 2s craft ran five sessions on the one RTX PRO 6000 (Blackwell, 96 GB; 4.54 s/step, ~58k tok/s), each its own event file: steps 50–8,300 (host 4d130c0e7351), 8,350–24,850 (91b9f3518593), 24,900–43,100 (e935248f0778), 43,150–53,750 (68304a2b8506), 53,800–61,400 (c2dc3d694638). The breaks are the session caps at 8,320 (fineweb, 2.181B), 43,130 (inside s6, 11.306B) and 53,792 (the s8 boundary itself, the resumed session's first action) β€” session_cap_step*.json is written when a session hits MAX_HOURS mid-phase β€” and the revival halt at 24,860. Sessions are boundary-exact: the driver calls train(max_hours, stop_at_boundary=True), which returns after each boundary checkpoint with _last_boundary set, and never auto-resumes a manual stop (_interrupted; trainer.py:170-184,328-335; the 0.8.1 driver contract, minted after run0 rolled through its boundaries and swallowed a KeyboardInterrupt).

Resume. _restore loads resume/latest.pt β€” fp32 model, both optimizer states, the stream position, torch and CUDA RNG, and the manifest snapshot embedded in the payload, which wins over the hub's manifest.json (trainer.py:110-153; checkpoint.py:131-175). A missing payload over a manifest with steps > 0 refuses to start fresh (trainer.py:110-120). Every phase boundary archives an immutable copy resume/boundary_<phase>_step<N>.pt (trainer.py:469; checkpoint.py:134-154). Resume is not row-exact: the datasets shuffle buffer (10,000 rows, streams.py:292-293) is not part of the saved state, so each resume refills it (streams.py:12, the honesty note).

The revival at 24,860. The born-null head buried itself inside the first 2,000 steps (0.032Γ— chance liveness at the first stored checkpoint; β€–sβ€– ~6e-4; proj cosines ~60Γ— below chance β€” learned avoidance; repos/alephllm.md:1564-1643 β€” journal readings from checkpoint analysis: the craft's TB carries no head/liveness_ratio scalar before 0.8.4; the control's TB reads 0.052Γ— @2,000, 0.014Γ— @12,000, 0.010Γ— @24,000). The BOUNDARY-WRITE op (revival.py:44-104) ran on the halted run: proj ← uncentered whitening-PCA of the final-norm outputs (sv floor 1e-3 relative); codebook ← fresh normalize(randn) with min-sep 45Β° (0 pushes needed); W_s ← fp64 ridge on the base head's CE residual, Ξ± swept on the held-out half and folded in. revival_step24860.json: K 256, D 256, seed 13, sv0 3676.66, svD 15.90, Ξ± 8.0, β€–W_sβ€– 481.53, fit_positions 65,536, base_bpb 1.27682 β†’ revived_bpb 1.23391 (Ξ” βˆ’0.04290 held-out), liveness 0.01027Γ— β†’ 0.99642Γ—, version 0.8.2, commit d6e1854. The pre-revival resume was archived, Muon momentum zeroed for the three head tensors, and the revived payload promoted on Phil's explicit go (repos/alephllm.md:1669); thereafter head_addr_frozen freezes proj and the head codebook by requires_grad alone, so optimizer groups and resume state are unchanged and Muon skips them (trainer.py:75-80; optim.py:42-43). checkpoints/step_00024860.safetensors is the pre-revival bf16 weight; step_00024860_revived.safetensors (fp32) is the post-write weight; neither the revived file nor the resume archives appear in manifest.json's checkpoint index.

The anneal. Activated 2026-08-31T09:54Z after s8; nochat done 14:42Z, chat done 19:31Z; the frame cost nothing in a bracketed phase (probe mean .389 β†’ .389; val 1.1153 β†’ 1.1097).

The 2s boundary ledgers (reports/v2/*.json; bpb on the fineweb-edu holdout; toggles = mechanism-off minus full):

report step tokens bpb_full banks hubs head doc / reset
warmup_wikitext 1,145 0.300B 2.36257 +0.0365 +1.5042 βˆ’2.3e-06 12.83 / 4.287
session_cap 8,320 2.181B 1.20332 +1.727 +2.036 βˆ’4.9e-07
fineweb_main 20,219 5.300B 1.11719 +3.5527 +2.2157 βˆ’3.4e-07
curriculum_s0 22,890 6.000B 1.2522 ~0 (born null)
curriculum_s1 25,561 6.700B 1.2544 +0.0258
curriculum_s2 29,376 7.700B 1.2177 +0.0140
curriculum_s3 33,954 8.900B 1.2318 +0.0072
curriculum_s4 38,532 10.100B 1.2356 >3.0 >3.0 +0.0130
curriculum_s5 42,347 11.100B 1.2504 +0.0278
session_cap 43,130 11.306B 1.2614
curriculum_s6 45,399 11.900B 1.2683 +0.0126
curriculum_s7 50,740 13.300B 1.1883 βˆ’0.0010
curriculum_s8 (STRUCK) 53,792 14.100B 1.0737 +3.396 +5.164 0.843 / 1.889
session_cap (true s8) 53,792 14.100B 1.19112 +3.2144 +3.3410 +0.0149 5.052 / 2.469
anneal_nochat 57,607 15.100B 1.1153 +0.0101
anneal_mix 61,422 16.101B 1.10970 +3.3115 +3.3584 +0.0123 3.862 / 2.130

Blank cells are not in the record consulted here; the head column is complete. The warmup row is the first full-splat-under-AR ledger: with all 20 hubs off the craft loses its entire attention (+1.50 at 0.3B), which is what "load-bearing, not decorative" measures against the mother's 3-hub form. hidden_erank_floor flags TRUE in every report from the 8,320 cap on (the trainer's own TB census, on the first val batch, flags it from step 500) (L19 pre-norm erank in single digits β€” the byte funnel; the post-norm nf(h) erank reads 334/1024), head_buried FALSE from s1 on, den floors 0.0 the entire run, zero loss spikes. Two "final val" readings exist on the hub: the report ledger 1.10970 (the gauge of record) and the manifest's checkpoint entry val_bpb 1.11403 at 61,422 β€” the manifest column is the LAST PERIODIC EVAL's bpb_full (eval every 2,000; trainer.py:444-447), so it lags every boundary (fineweb 1.1156 vs 1.1172; s8 1.1833 vs 1.0737/1.1911); the manifest minimum is 1.1140 @60,000. Cite the report.

The paired record (mini-beatrix-2s-control/reports/v2/)

Derivation. Every preset auto-generates a -control twin: the same AlephLMConfig with hub_layers = () (presets.py:216-229), so hub_K/hub_D/hub_const/hub_chunk are carried but unused and every block's attention is CausalSDPA. 0.8.7 (@b472815) gave twins their own TrainConfig copy β€” the shared instance had let head_addr_frozen=True leak into the control, a post-revival flag no birth recipe may carry β€” and sets the 2s control to head_addr_frozen = False explicitly: the birth recipe, born-null unfrozen head, which buries as the treatment's did for its first 24,860 steps (presets.py:221-235). The control inherits governor="minsep"; with no hub books it governs the head codebook only (hits_cum 10 at warmup, fineweb and the 22,854 cap β€” the first session β€” and 0 in every later session's report; crowd_check_extra_hits 0). Same seed 1337: construction order embed β†’ blocks β†’ norm β†’ head after torch.manual_seed (trainer.py:63; alephlm.py:85-96) makes the embedding draws identical between craft and twin; the RNG streams diverge from block 0's attention onward (a derivation from construction order, not a weight comparison).

Protocol. Declared at the control's warmup boundary, before the outcome, while sdpa led (2.1056 vs 2.3626): the architecture verdict is the paired boundary table across the full 16.1B matched budget; the headline is endpoint val; secondaries are probe means at matched boundaries and the doc/reset gauges; no verdict from any intermediate lead in either direction (repos/alephllm.md:1961-2002; a control must be able to fail). Completed unaltered through 13 paired rows. Budgets are matched in tokens, not wall-clock: fused sdpa ran 71.9k tok/s on the RTX PRO 6000 and 113k on the H100 against the eager scan's ~58k.

The paired rows (bpb_full, same step, same tokens, same gauge except where struck; deficit = control βˆ’ craft):

boundary step tokens 2s control deficit control bank toggle control grad-norm median
warmup 1,145 0.30B 2.3626 2.1056 βˆ’0.257 βˆ’0.008
fineweb 20,219 5.30B 1.1172 1.1763 +0.059 1.62 0.31
s0 22,890 6.00B 1.2522 1.5047 +0.253 3.44 6.47
s1 25,561 6.70B 1.2544 1.3917 +0.137 3.61 2.43
s2 29,376 7.70B 1.2177 1.3938 +0.176 4.13 11.9
s3 33,954 8.90B 1.2318 1.5317 +0.300 4.73 12.7
s4 38,532 10.10B 1.2356 1.5689 +0.333 5.09 24.5
s5 42,347 11.10B 1.2504 2.1762 +0.926 4.36 126.4
s6 45,399 11.90B 1.2683 3.1940 +1.926 3.89 471
s7 50,740 13.30B 1.1883 3.1201 +1.932 4.17 162
s8 53,792 14.10B 1.0737 3.0810 STRUCK (gauges differ) 5.48 145
anneal_nochat 57,607 15.10B 1.1153 2.9270 +1.812 5.64 167
anneal_mix 61,422 16.10B 1.1097 2.8846 +1.775 6.11 215

Eleven same-gauge rows after the crossover, all in the craft's favour; the endpoint gap is +1.775 at 16,101,408,768 tokens on both. The control's own trajectory is the finding: 1.1763 at 5.3B β†’ 3.1940 at s6 β†’ 2.8846 at the end β€” a destabilised model at its endpoint, not a trailing one. Its endpoint ledger: bank_off 8.99015 (toggle +6.1056), head toggle βˆ’1.06e-05, head_buried TRUE at every row from fineweb on, doc_bpb 5.422, reset_bpb 3.825; probes p0 .200 p1 .333 p2 .267 p3 .333 p4 .300 p5 .433 p6 .233 p7 .433 p8 .333 against the craft's .367 .367 .600 .200 .333 .567 .333 .333 .400 β€” ties at n = 30 (SE of a difference .126). The control's bank dispatch entropy fell through the curriculum (collapse/dispatch_entropy_L5: .94 at the s0 census, .75 @30,000, .48 @35,000, .22 at the s5 census; L0 .75 β†’ .47 @30,000; layer minimum .14 @42,500), ahead of the >30 grad-norm-median onset at 38,000.

The destabilisation facts (from the hub tfevents, train/grad_norm = pre-clip): the control first crossed the clip at step 17,600 (4.61B, inside fineweb, on the RTX PRO 6000, while still leading); 813 of 824 logged curriculum+anneal steps clipped (98.7%); 38 steps above 1,000; max 41,070 @44,600; the > 30 onset at bin 38,000 (late s4) and > 100 at 40,500 (mid-s5) inside one event file. The 2s craft: 0 of 1,225 logged steps from step 200 on above the clip (the only reading above 1.0 is step 50, 1.063, inside the 200-step LR warmup); max 0.760 @1,300 after the warmup boundary, and 0.619 @26,750 (s2) over the 824 curriculum+anneal steps. The spike detector and the non-finite guard never fired on either craft. Under Muon the clip braked nothing in the Muon group (qkv/o, the position table, the head); the escalating norm is a sharpness signal, not a throttled step.

The grade and its caveats. The instance holds: the all-sdpa twin is a destabilised control under the splat-tuned recipe, one craft per arm (S-1802 INSTANCE). The general claim β€” softmax attention is inadequate for this model's attention requirement β€” is a CANDIDATE (S-1803) behind confounds stated as measurements: (a) the recipe was tuned on the splat craft and carries none of the field's softmax stabilisers (QK-norm, softcap, z-loss, weight decay, decay), and the signature β€” clip-engaged smooth degradation, entropy and rank collapse, no NaN, onset on plain prose while leading β€” is the documented softmax logit-growth mode; (b) n = 1, no bar, +20.3M parameters in the losing arm; (c) shape: the mother null β€” mini-beatrix-1 (13 sdpa + 3 hubs, d768) under the identical TrainConfig (git cc89a87..b472815; the manifests record no train_config) through the same nine stages at the same budgets ran a post-warmup grad-norm max of 0.883, 0 of 1,750 logged steps clipped (canon/splat_aleph_battery.md:955; canon-cited, not re-verified against the mini-beatrix-1 event files for this companion β€” her 88,508 steps / 26,102,071,296 tokens / 70.45 h do verify against mini-beatrix-1/manifest.json) β€” with the nuance that v1 entered s0 at 17.3B tokens after 15B prose and a 2B anneal, the control at 5.3B; (d) the aleph-addressed bank sits in both crafts, its dispatch collapse precedes the escalation, and no sdpa + dense-FFN arm exists at any scale; (e) the 4096Γ—1024 position table on WD-free Muon at 2e-2 forever; (f) house Muon is not reference Muon; (g) host and row-order asymmetries. One gauge (val bpb; probes are ties); the s8 pair struck; three hosts. dist-e002 (vision, two seeds) remains the only Law-E splat-vs-sdpa endpoint, and it is a tie.

Provenance of the three hosts. RTX PRO 6000 from birth to the 23.5 h cap at 22,854 (event file 50–22,850); a Colab H100 80 GB from 22,854 (~2.3–2.4 s/step) through s0–s8 (22,900–53,750), 36 steps before the s0 boundary, with the shuffle buffers refilled; a session 53,800–56,000 follows the 53,792 cap in the H100 session's own host and process (event file …0a706d592fec.620.1, 20:01–21:35Z, ~2.3 s/step, 114k tok/s); a third host (2b1783ef9e94) opened 2026-09-03T01:41Z from the step-56,000 checkpoint at ~3.6 s/step and ran both anneal rows (56,050–61,400; two 88-byte event files are aborted starts) β€” a replay of up to 999 steps at that restart is a hedge on the 1,000-step TB-upload cadence, not a hub reading (the H100 process's last event is 21:35:20Z, the step-56,000 checkpoint landed 21:35:33Z and its manifest 21:36:06Z, the third host opened 01:41:15Z; the hub can neither confirm nor refute a replay), and it left no cap report. The canon's "on the H100" lines for boundaries 11–12 are wrong by erratum (canon/splat_aleph_battery.md:1008). The escalation began pre-resume on the RTX PRO 6000; a 36-step data-order change cannot make a 0.25 bpb gap on a fixed eval.

The boundary report (reports/v2/) β€” schema

Written by the notebook's boundary_report(run, tag) cell to reports/v2/{tag}_step{step}.json, tag = the phase that just finished or session_cap:

  • tag, step, tokens β€” tokens is manifest.tokens_seen (new in the v2 record).
  • probes β€” {p0_grounding, p1_self_other, p2_difference, p3_logic, p4_arithmetic, p5_causal, p6_learning, p7_composite, p8_articulation}, each {acc, families{…}, n: 30}; byte-NLL argmin over the options, fp32, no sampling (probes.py:191-221); holdout families p3:d5 and p4:sub3 gauged from time zero (probes.py:171).
  • census_flags β€” hidden_erank_floor (any block-output effective rank < 0.05Β·d = 51.2), anchor_merge (any pair |cos| > 0.99 in a bank/hub/head book), dispatch_collapse (bank dispatch entropy fraction < 0.10), den_floor (hub den floor_frac > 1e-3), head_buried (liveness < 0.1; added at 0.8.2 β€” absent from the 2s warmup_step1145, session_cap_step8320, fineweb_main_step20219 and curriculum_s0_step22890 reports, present from s1; present in every control report), codebook_erank_floor (bank codebook erank < 1.5) (instruments.py:150-178). The census sample is (2, min(1024, ctx)) random bytes in 0..254 (torch.randint(0, 255, …), so DOC never appears in the census sample) in the notebook writer; the trainer's own census uses the first val batch (trainer.py:371-374).
  • ledger β€” bpb_full, bpb_bank_off, bpb_head_aleph_off, bpb_hub_off (only if any block is a hub β€” the control's ledger has five keys), and toggle_<x> = bpb_<x> βˆ’ bpb_full; mean CE over the fixed val batches / ln 2 (instruments.py:187-212). No w_s_norm field (the mb1 schema had one).
  • special β€” {doc_count, doc_bpb, reset_bpb} (instruments.py: 310-352); None on BPE crafts.
  • governor_hits_cum β€” the session's cumulative pair-pushes; the counter is a Trainer attribute defaulting to 0 (trainer.py:266), not persisted in the manifest or the resume payload, so it resets every session (103,951 @1,145; 1,304,497 @20,219; 70,747 at s1 after 1,604,488 at s0; 0 at session_cap_53792; 776,450 @61,422).
  • crowd_check_extra_hits β€” pushes from an on-demand govern_model at report time (415 @20,219; 709 @61,422; 0 on the control).

Other report kinds in the same directory: revival_step24860.json (the op record above, no probes/ledger β€” chart code must skip it) and cultivation_{A_born_null,B_revival_birth,C_matured_transplant, D_live_address_only}.json β€” the C7 head-cultivation screen, 1,200 steps at seed 1337 with gauges at 300/600/900/1200 {toggle, w_s_norm, head_s_norm, head_s_chance, head_liveness_ratio, head_buried} and, for B and D, a revival dict (Ξ± 16.0, fit 16,384, base 8.336 β†’ 5.389). Endpoint toggles: A βˆ’5.3e-06 (buried, liveness 0.045); B +3.496 (liveness 1.125); C +0.0922 (β€–W_sβ€– 520.5); D βˆ’7.1e-05 (liveness 1.037, not buried, no function) β€” a live address alone does not carry function; a revived head at birth does.

The manifest (manifest.py:16-110): preset, model_config (the full AlephLMConfig), created_utc, updated_utc, steps, tokens_seen, wall_hours, phases[{name, dataset, planned_tokens, tokens_done, status ∈ planned|active|done|deferred}], checkpoints[{step, tokens, kind ∈ safetensors|resume|fp8, path, val_bpb, utc}], data_state{dataset, rows_consumed, epoch}, notes[{utc, msg}]. 2s: created 2026-08-27T00:35:31Z, updated 08-31T19:37:50Z, 142 entries (59 safetensors / 24 fp8 / 59 resume β†’ 46 distinct bf16 paths). Control: created 08-31T21:26:32Z, updated 09-03T07:16:38Z, 136 entries β†’ 44 bf16. Neither records a train_config.

Checkpoint kinds (checkpoint.py:102-155,187-195): bf16 safetensors every 2,000 steps and at every boundary, every state-dict tensor including the addr.home buffers (so the file's numel, 237,908,285, exceeds the parameter count by 782,336); fp8-e4m3 every 5th checkpoint or final, 2-D+ tensors with numel > 4,096 stored with a per-tensor fp32 scale amax/448 under <key>.scale, small params kept bf16, metadata "inference/testing only" β€” never train from these; resume/latest.pt fp32 with the manifest embedded, overwritten at every checkpoint and archived at boundaries. Hub uploads retry three times and never crash training; downloads distinguish genuine absence from network trouble and raise on the latter (checkpoint.py:29-100).

TensorBoard tags (trainer.py:283-296; instruments.py:215-260; 314 tags on the craft, 212 on the control): from the loop train/{loss,bpb,grad_norm,lr_scale,tokens,tokens_per_sec}, collapse/loss_spike, governor/hits_cum, sys/vram_gb; from the census per layer i erank/hidden_L{i}, erank/hub_consumed_L{i}, erank/hub_codebook_L{i}, erank/bank_codebook_L{i}, cv/bank_load_L{i}, cv/hub_usage_L{i}, drift/bank_L{i}, drift/hub_L{i}, sign/bank_frac_neg_L{i}, collapse/dispatch_entropy_L{i}, collapse/den_floor_L{i}, gates/L{i}_e{k}; head head/{gamma,w_s_norm,s_norm,liveness_ratio, consumed_erank}, erank/head_codebook; collapse/flag_{…} for the six flags; toggles/{bpb_full,bpb_bank_off,bpb_head_aleph_off,bpb_hub_off, toggle_bank_off,toggle_head_aleph_off,toggle_hub_off}; from the full eval canary/{recall_acc,wrongpair_frac,noretrieval_frac} and special/{doc_count,doc_bpb,reset_bpb}. The control lacks every hub family; the archived run0 file lacks head/liveness_ratio, head/s_norm, special/* and flag_head_buried (pre-0.8.0 specials; the head_buried flag arrived at 0.8.2 @d6e1854 and the head/s_norm + head/liveness_ratio TB scalars at 0.8.4 @a690ea4).

archive/run0-nospecials/. The pre-specials attempt, archived whole 2026-08-27T00:16Z: its manifest (created 08-26T20:50:18Z, steps 2,537, tokens 665,059,328, wall 3.22 h, warmup done at 1,145, fineweb_main in progress), bf16 checkpoints 1,145/2,000/2,253/2,537, fp8 2,253/2,537, resume/{latest,boundary_warmup_wikitext_step1145}.pt, one event file, and a report at the doubled path archive/run0-nospecials/mini-beatrix-2s/reports/v2/warmup_wikitext_step2253.json β€” a late-tagged fineweb report written on interrupt (stale tag, doubled path: the 0.8.1 driver-contract bug) with probes, five census flags and governor fields but no ledger/special block. Code-wins note: the manual's RIDER 9 says run0 "halted at step 2000"; the archived manifest says 2,537 (train() swallowed the interrupt and the driver auto-resumed ~19 minutes; repos/alephllm.md:1408). The hub wins: 2,537.

The AutoModel (AbstractPhil/mini-beatrix-2s)

Thirteen files: README.md, config.json, modeling_minibeatrix.py, the vendored address.py, alephlm.py, attention.py, bank.py, embedding.py, governor.py, head.py, presets.py, model.safetensors (475,880,498 B; 593 bf16 tensors, numel 237,908,285, every key prefixed model., metadata {tokens: 16.101B, step: 61422, format: pt}), .gitattributes. No tokenizer files: input ids are raw UTF-8 byte values. The vendored files are byte-identical to geolip/alephllm/model/*.py and presets.py at 0.8.7 except one import line (alephlm.py:30, ..presets β†’ .presets). config.json is the craft's model_config plus model_type "mini-beatrix", auto_map to MiniBeatrixConfig/MiniBeatrixForCausalLM, torch_dtype "float32" (the file is bf16; the loader upcasts on request), use_cache false. The wrapper (modeling_minibeatrix.py: 21-99) holds self.model = AlephLM(cfg), forces hub_ckpt = 0, makes _init_weights a no-op ("AlephLM initializes itself; loader must not re-init"), truncates to the last context positions, returns CausalLMOutputWithPast(loss, logits) with HF label semantics, and carries no KV cache β€” HF generate() recomputes the prefix each step; the inner AlephLM.generate(idx, max_new, temperature, top_p, use_cache=True) uses the constant-size hub state.

The parity gate. The ship gate β€” strict load, logits parity 0.0 against the geolip code path, generate() ok β€” is journal-recorded (repos/alephllm.md:1840-1846; history/timeline.md 2026-08-31; MIRROR.md step 2 in the AlephLLM repo), not hub-shipped: no parity script exists on either hub repo or in the package. Re-verified on CPU by the drafting workflow (torch 2.11.0, transformers 5.13.0, no geolip installed; not re-run in the hub-side pass, which found the hub consistent with these readings β€” TB head/w_s_norm 515.13 at step 61,000, 593 bf16 tensors, numel 237,908,285): 593 file keys == 593 model keys, parameter count 237,125,949, β€–W_sβ€– 515.918, Ξ³ 1.0; the inner cached decode equals the parallel path (torch.equal True); logits parity against geolip was not recomputed. Two notes for the reader: the model card's lineage line says alephllm 0.8.6 while the vendored files are the 0.8.7 mirror (commit 90a06b75, 2026-08-31 21:11 β€” the presets.py carries the 0.8.7 twin-isolation block); and the card's "17 reports" counts the 16 boundary/session reports plus the revival op β€” the prefix holds 21 JSONs with the four cultivation arms.

Load and evaluate (the steps below were executed by the drafting workflow; the numbers are from that run and were not re-run in the hub-side pass):

import math, torch
from transformers import AutoModelForCausalLM
m = AutoModelForCausalLM.from_pretrained("AbstractPhil/mini-beatrix-2s",
                                         trust_remote_code=True).eval()
ids = torch.tensor([list(text.encode("utf-8"))])          # bytes in
bpb = m(input_ids=ids, labels=ids).loss.item() / math.log(2)
# toggle ledger on the inner model (positive delta = mechanism carries function)
core = m.model
full = core(ids, labels=ids).loss / math.log(2)
bank_off = core(ids, labels=ids, disable_bank=True).loss / math.log(2)
hub_off = core(ids, labels=ids, disable_hub=True).loss / math.log(2)
head_off = core(ids, labels=ids, disable_head_aleph=True).loss / math.log(2)
# generation: HF path (no cache) or the inner cached decode
out = core.generate(ids, max_new=40, temperature=0.0, use_cache=True)
print(bytes(out[0].tolist()).decode("utf-8", errors="replace"))

On a 227-byte English snippet this read 0.9446 bpb full, +3.872 bank off, +3.925 hub off, +0.0156 head off; greedy from "The history of astronomy begins with the" continued " study of the sun and the planets. The s". Chat: wrap raw bytes in 0xFB SYS / 0xFE USER / 0xFD MODEL / 0xFC END (special_tokens.py:112-136). Training checkpoints load into the same wrapper after prefixing every key with model., or into a bare AlephLM(cfg.to_aleph()); fp8 files must be rehydrated first (checkpoint.py:187-195). What the AutoModel cannot reproduce is the gauge of record: the fineweb-edu holdout needs geolip.alephllm.data.streams.build_stream(dataset, tok, 4096, 16, seed = 1337 + 9999, role="val") and 262,144 tokens (4 batches of 16 Γ— 4096), then toggle_ledger and special_token_gauge over those fixed batches (trainer.py:384-411; instruments.py:187-212,310-352).

The process that made 2s different (dated)

The mother, measured (2026-08-12 β†’ 08-27). mini-beatrix-1 was built with 13 sdpa blocks and three hubs at layers 4/9/14, each a single 512-anchor book in D = 32 (git show cc89a87:presets.py), full-width banks holding 67% of the mass, and a 512 @ 32 head. Two post-mortems set the 2s agenda: the 08-25 hub census β€” 16Γ— the K ≀ 2D supply, 333/646/181 duplicate anchor pairs at |cos| ~1.0, consumed erank 2.2/3.7/10.8 β€” and the 08-27 autopsy of the shipped weights, W_s exactly 0.0 after 8.4B: the "head election" lore retracted, the head never carried function (repos/alephllm.md:1586; history/plans/2026-08-26_mini_beatrix_v2_shape.md). A third: the v1 anneal (2B, chat template inside, before the school) baked the chat frame into the core irreversibly β€” the chat-in-anneal law (2026-08-15) that later dictated the bracketed two-phase anneal.

Why every block (08-23 β†’ 08-26). Phil's directive: "the entire model to be splat attention". The record's support was dist-e002 β€” cured splat in all 12 ViT-Ti blocks ties sdpa at matched params and matched wall-clock on two seeds (R@1 .0545 vs .0562), late placement edging (R@5 +.0128, outside the bars); the untested bet, flagged in the plan, was that full-splat CAUSAL had never trained. The payoff row was the warmup ledger (+1.50 with the hubs off at 0.3B) and the endpoint (+3.36).

The shape (08-24/25 β†’ 08-26). ROUND 5e's shape ladder: 8 Γ— 256 lost 4 of 4 paired against 12 Γ— 128, the wide model seed-fragile, its crowding tail (79% of layer-1 heads over the 25Β° floor on one seed) naming the chain width β†’ crowding β†’ positional ranking flips β†’ deficit; hence SUPPLY K ≀ 2D (STATUTES_INDEX.md:53) and depth over width. B2's product code (independent books compose, .859 β†’ .955 monotone in members) gave the multi-book form; the rotary Γ— anchor flip analysis gave governability D β‰₯ 128 (Ξ³* 32Β° at d512, 45Β° at 256, unreachable at 64). The arithmetic lands on 4 Γ— 64 @ 128 (0.5Γ— supply, 4Γ— headroom against v1's 16Γ—), L20, d1024, ctx 4096 (splat's wall-clock crossover ~2k; below 4k a full-splat trunk pays attention's price without the O(L) payoff), banks narrowed to ff 1024, head 256 @ 256. The 849M sibling (L32, ctx 8192, 16 Γ— 256 @ 256) was designed the same session (presets.py:161-183).

The governor from birth (08-24 β†’ 08-26). Phil's proposal β€” "a middleman to control a min/max space that drift can occur to guarantee there's always enough space between the anchor points" β€” was confirmed in the math: for a crowded pair the static logit gap is quadratic in separation while the positional oscillation is linear, so small separation flips weight rankings along the sequence; separating an already-crowded trained book leaves its flip pattern bit-identical, so min-sep is preventive, not curative β€” run it from birth. Rung 1 ran bit-identical to the ungoverned run (raw books never crowd; the crowder was the frame W_a); gov25m fired on the fragile seed only, harmlessly. Shipped as a post-step projection, 45Β°, every 8 steps (governor.py, presets.py:99-105). In-mission it ran hot β€” ~120 pushes/step at step 1,000, ~84–105 sustained through fineweb, never decaying β€” with every function gauge healthy; the ΞΈ ladder 35/40/45 for K = 64 @ D128 is an open rung, and the preset comment's "45 at D=256" does not itself justify 45Β° at D = 128.

The kernel campaign and the pivot (08-26, one day). 0.7.0 at 10:41 (multi-constellation hub, supply guard, governor wired in, the 849M rescale, 2s as the screen craft). The Blackwell preflight OOMed (hub_ckpt), then launched at 161.88 s/step; the per-op profiler printed qc torch.float32 under autocast β€” 0.7.2 at 12:27 (the einsum promote trap; 4090 hub fwd+bwd 229.7 β†’ 178.8 ms). The dtype fix bought ~6%; the binding constraint was kernel shape β€” 0.7.3 at 13:14 (one stacked GEMM, one fused softmax, prefix as a triangular GEMM; 65.3 ms, 3.5Γ—; peak 7.4 β†’ 1.9 GB; Blackwell 149 β†’ 69.7 s/step). Compile refuted by measurement (bf16-backward NaN, and the pre-NaN projection 67 vs 69.7 s eager) β€” eager-only on Blackwell. Economics: 17.3B at 8k tok/s is 24–26 days on the one card. Phil: "we can't train the large one currently. Not until we work out a series of logistics problems." 0.7.5 at 13:43 (@73218bd): 2s is the mission, hub_ckpt 2 β†’ 0; launched at 4.54 s/step, 58k tok/s. "Not a bad start. We'll train this one." The 849M is shelved, not retired (its value-bottleneck relaunch lever quantified at ~46 s/step).

The format restart (08-26 16:43). Phil: "We'll want a practical array of them for usage, nothing small. We'll halt this one and start over with the correct formatting." 0.8.0: the thirteen unreachable ids, the exhaustive proof at every launch, DOC from birth, the frame only in anneal, the canary collision caught; run0 archived whole, run1 fresh on the clean prefix (manifest 08-27T00:35Z). DOC cost nothing β€” run1 train bpb 1.49 @1,100 against run0's 1.494 @1,000 β€” and doc_bpb fell 12.83 β†’ 3.29 by 0.52B. 0.8.1 the same day: the driver contract that made every later boundary row a real boundary.

The head saga (08-13 β†’ 08-29 β†’ 08-31). Born weight-zero by law (gate-zero deadlock, head.py:8-11); buried inside 2,000 steps on run1 (0.032Γ— chance at the first stored checkpoint, 0.014Γ— @12k β€” journal readings from checkpoint analysis, not TB scalars; learned avoidance β€” proj rotated into codebook-orthogonality while W_s grew to norm 31 multiplying ~0); LS-ignite alone refuted (0.0006 bpb); the governor exonerated (the ungoverned v1 buried deeper; the 45Β° floor is free at birth). Candidate law at 2/2 crafts, 3/3 with the control: a born-null aleph head racing a strong same-target linear base self-buries within ~2k steps; the bank's self-start does not transfer across competitor types. Phil ruled activation; the BOUNDARY-WRITE law was minted (revival.py:26; STATUTES_INDEX.md:67) β€” a third intervention category outside both the governor statute and the no-balance-machinery law, permitted only at run boundaries or halted resume points, always logged β€” and the deterministic op executed at 24,860. The first nonzero head toggle in lineage history landed at s1 (+0.0258, after 6.5B tokens of exactly 0.000); the settled read is a durable, small, diet-sensitive niche in the band βˆ’.001..+.028, band-exits upward on the binding-adjacent diets (s5 causal +.0278), touching zero at s7 as bpb_full fell 1.268 β†’ 1.188.

The recipe that never changed (08-12 β†’ 09-04). Muon on the trunk because the 08-10 validity battery on the AR bed read muon .8081 > sgd_mom .7995 > adam .7157 > lion .5638, with the address ~20Γ— more optimizer-fragile than sdpa (cross-optimizer spread .79 vs .037) and schedules no-op-to-harmful (flat .714 > warmcos .596) β€” so the flat-LR law; pure Adam on the rest by the 09-01 optimizer statute (Adam the experimental baseline, Muon the trunk, AdamW never; STATUTES_INDEX.md:76). presets.py:80-84 is the same text at cc89a87 (where it sits at presets.py:65-69) and at HEAD. The same recipe is the audit's confound (a).

The diet (08-15 β†’ 08-31). The nine stages and budgets are v1's (0.6.0), placed before the anneal rather than after it (Phil, 2026-08-30: "Let her really build up that internal knowledge first"; history/plans/2026-08-26_mini_beatrix_2s_mission.md), fineweb_extended deferred; the two-phase anneal (Phil, 08-31: "enable the anneal mix without chat and the anneal mix with chat, we'll run without first, then just train chat on after"; streams.py:63-77), 0.8.6 at 02:48. The s8 terminal-gauge hole appeared at 04:57Z and 0.8.5 had already closed it in code (08-30 22:03); the true ledger was re-measured at 09:53Z. Mission complete 19:31Z: 61,422 steps, 16.101B tokens, 78.33 h, zero spikes; the HF outage at the final push was recovered by a Drive rescue and retry.

The twin (08-31 β†’ 09-04). 0.8.7 at 14:10 (twin isolation, birth recipe); the control born 21:26Z; the protocol declared at its warmup boundary while sdpa led; the crossover at fineweb 5.3B and every row after; the rank story as the mechanism claim (at step 4,000 the control collapses early hidden erank to 5–9 of 1024 at L2–L4 (5.5 / 5.4 / 9.3) while the craft reads 234 / 225 / 250 there and 150–396 across L0–L15 (erank/hidden_L{i}, TB census); screen scale reproduces it; rank β‰  function β€” the control led val at 4k while running rank-5 layers); the H100 resume at 22,854; the grad-norm escalation rider at s5; Phil's claim held on 09-02 ("SDPA isn't an adequate representative of this model's attention requirement. We can say that without question now."); the third host from 56,000; the endpoint 09-03; the 12-agent audit on 09-04 (S-1802 INSTANCE, S-1803 CANDIDATE, the s8 strike, the mother null, Muon clip-blind, the falsifier filed).

Gauges, defined once

  • val bpb: mean cross-entropy in bits/byte over 262,144 tokens (4 batches of 16 Γ— 4096) of the fineweb-edu reserved head slice β€” the first 2,048 rows, fixed, unshuffled, seed 1337 + 9999 β€” for every phase from fineweb_main onward, both crafts (trainer.py:384-411; streams.py:279-282,423-447; gauge continuity law). The warmup row vals on wikitext-103's validation split. Pre- and post-0.8.0 gauges differ for v1-era crafts (DOC rides the val stream).
  • toggle ledger: bpb with one mechanism disabled minus full bpb β€” the causal contribution of banks / hubs / head aleph (instruments.py:187-212). Positive = the mechanism carries function.
  • specials gauge: doc_bpb and reset_bpb in bits, read against the same val batches (instruments.py:310-352); doc_count 70 on every 2s-era report on the fineweb-edu gauge (the struck 2s s8 file reads 1612).
  • head liveness: mean β€–signed readβ€– of real hidden states over the same book's mean on 512 random unit vectors β€” self-calibrating; head_buried = ratio < 0.1 (instruments.py:144,176,356-382). Both buried crafts sat at 0.003×–0.03Γ— chance.
  • probe accuracy: byte-NLL argmin over the item's options, nine suites Γ— 30 items, holdout families d5 and sub3 gauged from time zero (probes.py:171-235). At n = 30 a between-craft difference has SE .126: the paired probe calls are ties.
  • train/grad_norm: the pre-clip global norm returned by clip_grad_norm_ (trainer.py:244,287), logged every 50 steps; under Muon it measures sharpness, not the applied step.
  • governor hits: cumulative pair-pushes per session (trainer.py:266); a session-scoped counter, comparable within one event file only.

Known caveats and open items

  • The autocast einsum-promote trap voids every pre-0.7.2 splat speed verdict (fp32 scans under "bf16"); accuracy verdicts stand. Any autocast performance reading over einsum code is invalid until the dtypes are printed (attention.py:167).
  • Muon is clip-blind (optim.py:24), and Adam's per-coordinate normalisation makes it nearly so: grad_clip 1.0 shrank the applied step of neither group on the control, only the weighting of steps inside each optimizer's buffers; the recipe carries no softmax stabiliser.
  • The 2s curriculum_s8_step53792 ledger is on the synthetic fallback and is struck; the same-gauge pair is owed by a zero-training re-evaluation of resume/boundary_curriculum_s8_step53792.pt on the 0.8.7 gauge (T0b in history/plans/2026-09-04_sdpa_falsifier_screen.md). Hub-vs-journal note: the file on the hub reads doc_count 1612, bpb_hub_off 6.2378 (toggle_hub_off 5.164), doc_bpb 0.843 β€” agreeing with the trainer comment and the manual (doc_count 1612, hub_off 5.16 = the toggle; trainer.py:397-399); the file carries no dataset field, so the strike rests on the same-step cap discrepancy (doc_count 70, bpb_full 1.1911 in session_cap_step53792.json) and on the 1612-document count itself. The hub file's numbers win for what the file says.
  • The control's third host from 56,000: a replay of up to 999 steps is a TB-cadence hedge, not a hub reading; no cap report; the 53,800–56,000 segment ran in the H100 session's own process (event file 0a706d592fec.620.1); canon CB11/12 "on the H100" wrong by erratum.
  • Every resume refills the shuffle buffer (not row-exact); the control resumed three times, the craft four; MixStream state carries no recipe fingerprint.
  • The 2s preset at HEAD carries head_addr_frozen = True (presets.py:199-201), the post-revival state: a fresh birth of the treatment preset under 0.8.7 would freeze a random proj and codebook from step 0, which is not what the shipped run did (born-null unfrozen under 0.8.1 for 24,860 steps, revived, then frozen). The control's override (presets.py:235) is the birth recipe.
  • Stale text, code wins: presets.py:18 "233M" (built 237,125,949); presets.py:14,166 "873.7M" for mini-beatrix-2 (built 849,023,425); the training README's ladder row "mini-beatrix-2 | 1024/20/2048 | 249.1M" is the retired v1 shape; the hub driver beatrix_v2_colab.ipynb is headed "mini-beatrix-2 β€” the governed full-splat mission" and names mini-beatrix-2-control while its CRAFT is mini-beatrix-2s (no control notebook exists on the hub β€” only this file and mini-beatrix-1/beatrix_curriculum.ipynb); AlephLMConfig.hub_chunk defaults to 128 while the module's own default is 256 (presets.py:52; attention.py:83) β€” the config governs, and 2s sets 256.
  • governor_hits_cum resets per session; the manifests record no train_config or package version β€” the mother null's "identical TrainConfig" is a git fact, and the craft's birth version 0.8.1 is inferred from commit dates (run1 created 08-27T00:35Z between 0.8.1 on 08-26 and 0.8.2 on 08-29; the revival report records 0.8.2; the hub driver beatrix_v2_colab.ipynb pins geolip-alephllm @2a902b7, the 0.8.1 commit).
  • Owed instruments: the per-layer hidden-erank census of both crafts at the full-scale boundaries; the falsifier S-1803-F (Tier 0 on the 4090; a 40M twin screen with sdpa + QK-norm / LR/3 / dense-FFN / positions on Adam arms; pivot = resume the control at 20,219 through s0 at LR/3 with QK-norm). Without it the general claim stays CANDIDATE.
  • article_assets_ft2/make_charts_ft2.py sources chart 01 from the hub but charts 02–05 from local scratch paths, and chart 01 pairs the struck s8 rows unless filtered.

Provenance discipline

Both crafts are resume-first: the manifest is embedded in the resume payload (atomic consistency), a crash never overwrites resume/latest.pt (trainer.py:334-368: any exception other than a manual stop leaves the hub state untouched), an interrupted session uploads before exit, and every phase boundary archives its own immutable resume. The revival is auditable from the prefix alone: the pre-write state (resume/pre_revival_step24860.pt), the op's provenance (reports/v2/revival_step24860.json, with version and commit), the pre- and post-write weights (step_00024860 and step_00024860_revived), and the first live row after it (curriculum_s1_step25561.json). The struck s8 ledger stays public beside the cap that supersedes it, as run0 stays public under archive/, for the same reason the mother's rewind checkpoints do. The recipe is pinned from source, not from the manifests. The AutoModel vendors the model files verbatim and ships the mission-final weights re-prefixed; the training checkpoints, not the card, are the control's weights. Every refuted arm of the head-cultivation screen ships.

Pointers

  • Canon: canon/splat_aleph_battery.md β€” CONTROL BOUNDARY 1–12 (522–889), ENDPOINT VERDICT (891), CLAIM AUDIT (901–1006: the s8 strike 930–937, MUON IS CLIP-BLIND 936, THE MOTHER NULL 955–968, PROVENANCE RIDER 1008), ROUND 5a/5d/5e, B2/B4/B5.
  • Statutes (STATUTES_INDEX.md): S-701 a control must be able to fail; S-801 supply law K ≀ 2D; S-1001 gate-zero deadlock; S-1002 born-null head self-burial 3/3; S-1003 BOUNDARY-WRITE; S-1201 optimizer family; S-1202 autocast einsum-promote; S-1801 the crossover; S-1802 the destabilised control (INSTANCE); S-1803 sdpa-inadequacy (CANDIDATE); S-1901 the specials plane.
  • Manual: repos/alephllm.md β€” the 0.7.0 era (1180–1330) and RIDERS 7–21b (1333–2081): pivot, launch health, specials and the run0 archive, driver contract, the dead head, the revival trial, the boundary-write and promote, the head trend 14–14l, curriculum complete and the s8 artifact, the pre-chat baseline, mission complete, the ship-out, twin isolation, the pre-registered protocol.
  • Plans: history/plans/2026-08-26_mini_beatrix_v2_shape.md (the shape arithmetic), 2026-08-26_mini_beatrix_2s_mission.md (the pivot, the anneal-timing ruling, the two-phase amendment), 2026-08-24_covariant_address_program.md (the governor), 2026-09-02_sdpa_inadequacy_claim_held.md, 2026-09-04_sdpa_falsifier_screen.md.
  • Dockets: history/dockets/2026-09-04_sdpa_claim_audit/, 2026-09-01_ft2_article/ and 2026-09-01_article/tools/ (writeback_cb*.py, writeback_endpoint.py).
  • Article: articles/article_beatrix_ft2.md; charts under article_assets_ft2/ on the training repo.
  • Code: AlephLLM HEAD 4f4b7d6 (0.8.7); the era's commits a63a29b (0.7.0) Β· 2be28dd (0.7.2) Β· 60ee496 (0.7.3) Β· 73218bd (0.7.5, the pivot) Β· 3ea3103 (0.8.0) Β· 2a902b7 (0.8.1) Β· d6e1854 (0.8.2) Β· 7d48328 (0.8.5) Β· af9319c (0.8.6) Β· b472815 (0.8.7). No git tags exist; versions are commit subjects.
  • The mother: TECHNICAL.md on the training repo (byte-identical local copy articles/TECHNICAL_training_repo_beatrix.md).