TECHNICAL.md — the life record companion (eras, exam record, arm record, gauges, provenance discipline)
Browse files- TECHNICAL.md +106 -0
TECHNICAL.md
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# alephllm-mini-beatrix-training — Technical Companion (the life record)
|
| 2 |
+
|
| 3 |
+
Companion to *Raising Beatrix: A Byte-Level Model's Measured Childhood*
|
| 4 |
+
(AbstractPhil with Claude Fable 5 & Claude Opus 5, August 2026). This
|
| 5 |
+
repository is the primary artifact: every checkpoint of every era —
|
| 6 |
+
including the checkpoints a rewind discarded — every arm (refuted arms
|
| 7 |
+
included), every exam and boundary report, and the manifest that makes
|
| 8 |
+
the run resumable from any machine. Stack-side detail lives in the
|
| 9 |
+
[AlephLLM companion](https://github.com/AbstractEyes/alephllm);
|
| 10 |
+
arm-system detail in
|
| 11 |
+
[amoe-lora](https://github.com/AbstractEyes/amoe-lora).
|
| 12 |
+
|
| 13 |
+
## Layout
|
| 14 |
+
|
| 15 |
+
```
|
| 16 |
+
mini-beatrix-1/
|
| 17 |
+
manifest.json # phases, checkpoints, token accounting
|
| 18 |
+
checkpoints/step_XXXXXXXX.safetensors # bf16 weights (72 with val gauge)
|
| 19 |
+
checkpoints/fp8/ # e4m3 ship-only copies
|
| 20 |
+
resume/latest.pt # optimizer + stream state (resume-first)
|
| 21 |
+
runs/ # TensorBoard event files
|
| 22 |
+
arms/ # chat arms (provenance-matched to cores)
|
| 23 |
+
arms/night/ arms/day1/ arms/bdist/ # the arm-program waves
|
| 24 |
+
reports/curriculum/ # baseline, gate_*, boundary_* JSON
|
| 25 |
+
reports/arms/ # wave ledgers
|
| 26 |
+
reports/lexicon/ # lexicon census editions
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
## The eras (all on one gauge: fineweb-edu holdout bpb)
|
| 30 |
+
|
| 31 |
+
| era | steps | bytes | val bpb | note |
|
| 32 |
+
|---|---|---|---|---|
|
| 33 |
+
| wikitext warmup | 0 → ~1k | 0.30B | 1.32 → ~1.2 | |
|
| 34 |
+
| fineweb main+extended | → 51,882 | 15.30B | **1.033** | the LOCK; core of record |
|
| 35 |
+
| anneal (contaminated) | → 58,664 | 17.30B | 1.045 | chat template + identity leaked into core; law recorded |
|
| 36 |
+
| head fold | 58,664 | — | 1.0479 (retro) | γ folded into W_s; semantic no-op 2.4e-07 |
|
| 37 |
+
| curriculum S0–S2 | → 66,803 | +2.4B | 1.13–1.14 | |
|
| 38 |
+
| S3 first attempt | (discarded) | ~+1.2B | **1.71–2.05** | CONVICTED: template colonization; checkpoints retained |
|
| 39 |
+
| REWIND | 66,803 | — | — | resume state restored; corrected mixes |
|
| 40 |
+
| S3–S6 corrected | → 81,047 | | 1.14 → 1.17 | logic/arith/causal/learning stages |
|
| 41 |
+
| S7 spiral | → 85,795 | | **1.111** | interleaved review repays the drift |
|
| 42 |
+
| S8 articulation | → 88,508 | 26.102B | 1.121 | graduation |
|
| 43 |
+
|
| 44 |
+
**Benchmark-best checkpoint: 83,903** (mid-spiral; 270-item exam mean
|
| 45 |
+
~0.43 vs graduate ~0.39). The graduate is the *continuation* core; the
|
| 46 |
+
mid-spiral checkpoint is the *evaluation* core. Both public.
|
| 47 |
+
|
| 48 |
+
## The exam record (`reports/curriculum/`)
|
| 49 |
+
|
| 50 |
+
- `baseline.json` — the 45-item era batteries on both t0 cores (51,882
|
| 51 |
+
and 58,664): the pre-curriculum report card, including the anneal's
|
| 52 |
+
measured probe cost (clean core beat annealed on 4 of 9 suites).
|
| 53 |
+
- `boundary_curriculum_s*.json` — automatic per-stage instruments:
|
| 54 |
+
probe suites, fineweb gauge, hub/bank toggle deltas, head ‖W_s‖.
|
| 55 |
+
Boundaries S0–S6 used the era batteries; S7–S8 the 270-item exams.
|
| 56 |
+
- `gate_step*.json` — on-demand exam runs of the newest checkpoint
|
| 57 |
+
(the notebook's instrument gate resolves the newest checkpoint at run
|
| 58 |
+
time; `baseline.json` is reserved for the historical t0 pair).
|
| 59 |
+
|
| 60 |
+
Headline exam-era facts: in-template logic/arithmetic 1.00 collapsed to
|
| 61 |
+
~chance on surface-disjoint items (both cores — template competence,
|
| 62 |
+
not transfer); S2 difference/category is the one stage that
|
| 63 |
+
generalized (0.50 → 0.77); the spiral produced the only broad rise
|
| 64 |
+
(~0.35 → ~0.43 mean) and the final blocked stage receded part of it
|
| 65 |
+
(graduation ~0.39).
|
| 66 |
+
|
| 67 |
+
## The arm record (`arms/`, `reports/arms/`)
|
| 68 |
+
|
| 69 |
+
Chat arms: `chat-s1200@step51882` (arm A — the space's exhibit pair
|
| 70 |
+
with the clean lock) and `chat-s1200@step58664` (arm B — trained on the
|
| 71 |
+
annealed core; its near-zero deltas are the anneal-confound
|
| 72 |
+
measurement). Night/day waves: stop arms (`stopnl` working, `stop` NUL
|
| 73 |
+
control refuted), capability arms (d5 default/wide, sub3 ×4 variants
|
| 74 |
+
all refuted, defs, identity, poly generalist), each with ledgers.
|
| 75 |
+
`arms/bdist/` receives the distillation lane (pre-registered; teacher
|
| 76 |
+
gate T0 = Qwen2.5-3B at 0.96 after 1.5B rejected at 0.86; pushforward
|
| 77 |
+
gate A0 = 1.0000).
|
| 78 |
+
|
| 79 |
+
Every refuted anchor ships. The ledger census equals the anchor census
|
| 80 |
+
(weights-completeness law).
|
| 81 |
+
|
| 82 |
+
## Gauges, defined once
|
| 83 |
+
|
| 84 |
+
- **val bpb**: mean cross-entropy in bits/byte on the fineweb-edu
|
| 85 |
+
reserved head slice (fixed, unshuffled; measured fact: 100%
|
| 86 |
+
crawl-year 2013), identical across all eras (gauge continuity law).
|
| 87 |
+
- **toggle ledger**: bpb with one mechanism disabled minus full bpb —
|
| 88 |
+
the causal contribution of banks / hubs / head aleph.
|
| 89 |
+
- **exam accuracy**: byte-NLL argmin over 3 options, 270 items,
|
| 90 |
+
surface-disjoint from every training generator; family sub-scores
|
| 91 |
+
including the reserved holdout families (rule-chain depth-5,
|
| 92 |
+
three-digit subtraction) that no training row has ever contained.
|
| 93 |
+
- **strict generative accuracy** (arm program): order-aware full-
|
| 94 |
+
sequence match — substring matching measured as inflating (1.00 vs
|
| 95 |
+
0.70) and retired from verdicts.
|
| 96 |
+
|
| 97 |
+
## Provenance discipline
|
| 98 |
+
|
| 99 |
+
The chat space serves only provenance-matched pairs (numeric step
|
| 100 |
+
match; anchor `base_model_id` verified at load) after an adversarial
|
| 101 |
+
review demonstrated silent foreign-arm attachment. The manifest is
|
| 102 |
+
embedded in the resume payload (atomic consistency); crashes never
|
| 103 |
+
overwrite `resume/latest.pt`; interrupted sessions upload before exit.
|
| 104 |
+
The rewind's discarded checkpoints remain public deliberately: the
|
| 105 |
+
conviction, the rewind target, and the corrected path are all
|
| 106 |
+
auditable from this repository alone.
|