AbstractPhil commited on
Commit
a2d9e07
·
verified ·
1 Parent(s): abaabb7

TECHNICAL.md — the life record companion (eras, exam record, arm record, gauges, provenance discipline)

Browse files
Files changed (1) hide show
  1. TECHNICAL.md +106 -0
TECHNICAL.md ADDED
@@ -0,0 +1,106 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # alephllm-mini-beatrix-training — Technical Companion (the life record)
2
+
3
+ Companion to *Raising Beatrix: A Byte-Level Model's Measured Childhood*
4
+ (AbstractPhil with Claude Fable 5 & Claude Opus 5, August 2026). This
5
+ repository is the primary artifact: every checkpoint of every era —
6
+ including the checkpoints a rewind discarded — every arm (refuted arms
7
+ included), every exam and boundary report, and the manifest that makes
8
+ the run resumable from any machine. Stack-side detail lives in the
9
+ [AlephLLM companion](https://github.com/AbstractEyes/alephllm);
10
+ arm-system detail in
11
+ [amoe-lora](https://github.com/AbstractEyes/amoe-lora).
12
+
13
+ ## Layout
14
+
15
+ ```
16
+ mini-beatrix-1/
17
+ manifest.json # phases, checkpoints, token accounting
18
+ checkpoints/step_XXXXXXXX.safetensors # bf16 weights (72 with val gauge)
19
+ checkpoints/fp8/ # e4m3 ship-only copies
20
+ resume/latest.pt # optimizer + stream state (resume-first)
21
+ runs/ # TensorBoard event files
22
+ arms/ # chat arms (provenance-matched to cores)
23
+ arms/night/ arms/day1/ arms/bdist/ # the arm-program waves
24
+ reports/curriculum/ # baseline, gate_*, boundary_* JSON
25
+ reports/arms/ # wave ledgers
26
+ reports/lexicon/ # lexicon census editions
27
+ ```
28
+
29
+ ## The eras (all on one gauge: fineweb-edu holdout bpb)
30
+
31
+ | era | steps | bytes | val bpb | note |
32
+ |---|---|---|---|---|
33
+ | wikitext warmup | 0 → ~1k | 0.30B | 1.32 → ~1.2 | |
34
+ | fineweb main+extended | → 51,882 | 15.30B | **1.033** | the LOCK; core of record |
35
+ | anneal (contaminated) | → 58,664 | 17.30B | 1.045 | chat template + identity leaked into core; law recorded |
36
+ | head fold | 58,664 | — | 1.0479 (retro) | γ folded into W_s; semantic no-op 2.4e-07 |
37
+ | curriculum S0–S2 | → 66,803 | +2.4B | 1.13–1.14 | |
38
+ | S3 first attempt | (discarded) | ~+1.2B | **1.71–2.05** | CONVICTED: template colonization; checkpoints retained |
39
+ | REWIND | 66,803 | — | — | resume state restored; corrected mixes |
40
+ | S3–S6 corrected | → 81,047 | | 1.14 → 1.17 | logic/arith/causal/learning stages |
41
+ | S7 spiral | → 85,795 | | **1.111** | interleaved review repays the drift |
42
+ | S8 articulation | → 88,508 | 26.102B | 1.121 | graduation |
43
+
44
+ **Benchmark-best checkpoint: 83,903** (mid-spiral; 270-item exam mean
45
+ ~0.43 vs graduate ~0.39). The graduate is the *continuation* core; the
46
+ mid-spiral checkpoint is the *evaluation* core. Both public.
47
+
48
+ ## The exam record (`reports/curriculum/`)
49
+
50
+ - `baseline.json` — the 45-item era batteries on both t0 cores (51,882
51
+ and 58,664): the pre-curriculum report card, including the anneal's
52
+ measured probe cost (clean core beat annealed on 4 of 9 suites).
53
+ - `boundary_curriculum_s*.json` — automatic per-stage instruments:
54
+ probe suites, fineweb gauge, hub/bank toggle deltas, head ‖W_s‖.
55
+ Boundaries S0–S6 used the era batteries; S7–S8 the 270-item exams.
56
+ - `gate_step*.json` — on-demand exam runs of the newest checkpoint
57
+ (the notebook's instrument gate resolves the newest checkpoint at run
58
+ time; `baseline.json` is reserved for the historical t0 pair).
59
+
60
+ Headline exam-era facts: in-template logic/arithmetic 1.00 collapsed to
61
+ ~chance on surface-disjoint items (both cores — template competence,
62
+ not transfer); S2 difference/category is the one stage that
63
+ generalized (0.50 → 0.77); the spiral produced the only broad rise
64
+ (~0.35 → ~0.43 mean) and the final blocked stage receded part of it
65
+ (graduation ~0.39).
66
+
67
+ ## The arm record (`arms/`, `reports/arms/`)
68
+
69
+ Chat arms: `chat-s1200@step51882` (arm A — the space's exhibit pair
70
+ with the clean lock) and `chat-s1200@step58664` (arm B — trained on the
71
+ annealed core; its near-zero deltas are the anneal-confound
72
+ measurement). Night/day waves: stop arms (`stopnl` working, `stop` NUL
73
+ control refuted), capability arms (d5 default/wide, sub3 ×4 variants
74
+ all refuted, defs, identity, poly generalist), each with ledgers.
75
+ `arms/bdist/` receives the distillation lane (pre-registered; teacher
76
+ gate T0 = Qwen2.5-3B at 0.96 after 1.5B rejected at 0.86; pushforward
77
+ gate A0 = 1.0000).
78
+
79
+ Every refuted anchor ships. The ledger census equals the anchor census
80
+ (weights-completeness law).
81
+
82
+ ## Gauges, defined once
83
+
84
+ - **val bpb**: mean cross-entropy in bits/byte on the fineweb-edu
85
+ reserved head slice (fixed, unshuffled; measured fact: 100%
86
+ crawl-year 2013), identical across all eras (gauge continuity law).
87
+ - **toggle ledger**: bpb with one mechanism disabled minus full bpb —
88
+ the causal contribution of banks / hubs / head aleph.
89
+ - **exam accuracy**: byte-NLL argmin over 3 options, 270 items,
90
+ surface-disjoint from every training generator; family sub-scores
91
+ including the reserved holdout families (rule-chain depth-5,
92
+ three-digit subtraction) that no training row has ever contained.
93
+ - **strict generative accuracy** (arm program): order-aware full-
94
+ sequence match — substring matching measured as inflating (1.00 vs
95
+ 0.70) and retired from verdicts.
96
+
97
+ ## Provenance discipline
98
+
99
+ The chat space serves only provenance-matched pairs (numeric step
100
+ match; anchor `base_model_id` verified at load) after an adversarial
101
+ review demonstrated silent foreign-arm attachment. The manifest is
102
+ embedded in the resume payload (atomic consistency); crashes never
103
+ overwrite `resume/latest.pt`; interrupted sessions upload before exit.
104
+ The rewind's discarded checkpoints remain public deliberately: the
105
+ conviction, the rewind target, and the corrected path are all
106
+ auditable from this repository alone.