ounce100m-code / train /train_ounce100m.py

Commit History

trainer: the post-stop assertions are rank-0 facts (E-044: hub_cb.steps is empty on rank 1 by construction, so the guard killed every correctly-checkpointed segment)
e26f3f1
verified

Cion-lab commited on

trainer: per-rank marks through the post-training sequence (v6's rank-1 death was unlocalisable because every line there is rank-0-only)
296186e
verified

Cion-lab commited on

trainer: an unverified checkpoint never rolls latest.json and never prunes (E-042)
a19fe02
verified

Cion-lab commited on

train: a segment that stopped short of the horizon skips the validation pass instead of entering a collective the two ranks can reach a batch apart (probe 1 v1/v4 found rank 1 dying there with no traceback, while every horizon leg passed the same call)
b6a124b
verified

Cion-lab commited on

review round 3 (probe/model-card/eval): rank-visible eval failure + cross-rank peak memory + complete run_summary; card derived-and-asserted from config; harness install --no-deps and task ids confirmed; torchrun redirects 3 + tee 3
52c7981
verified

Cion-lab commited on

train: escape the literal percent signs in two --help strings (argparse interpolates help text, so '+52 %' crashed --help with 'must be real number, not dict' -- caught by the CPU rehearsal's stage 3)
76fbbb7
verified

Cion-lab commited on

perm_sha: identify the visit order by hashing the permutation, not just its seed (E-035 residual) -- cursor gains shuffle_perm_sha, asserted on resume; session repins the two files
ef0c95d
verified

Cion-lab commited on

review round: pointer written+read-back before prune, verify retried, latest rolled to final, stale-pointer scan on resume, ddp_timeout 4h, chunked batched commits, manifest self-consistency gate, full content verification after upload
492bf86
verified

Cion-lab commited on

pin: train/train_ounce100m.py (RepoMissing resume branch, disk gate, PHASE4_PREP_ONLY)
7dc409a
verified

Cion-lab commited on

Upload train/train_ounce100m.py with huggingface_hub
212e0b7
verified

Cion-lab commited on

Upload train/train_ounce100m.py with huggingface_hub
6ed0dbb
verified

Cion-lab commited on

Upload train/train_ounce100m.py with huggingface_hub
e11176d
verified

Cion-lab commited on

Upload train/train_ounce100m.py with huggingface_hub
18d95d5
verified

Cion-lab commited on

Upload train/train_ounce100m.py with huggingface_hub
c3592b7
verified

Cion-lab commited on

train: --grad-ckpt, and record attn/grad_ckpt in every result line
a477d8d
verified

Cion-lab commited on

train: --benchmark N runs the same step with no Trainer, to attribute the P2 throughput gap
182a8d9
verified

Cion-lab commited on

train: explicit attention kernel (default eager, per p0c on Turing); report requested vs effective
3880d41
verified

Cion-lab commited on

train: train_ounce100m -- static-review fixes: absolute-step cursor, hub_listing pagination, cursor required on resume, repo_type everywhere, rank-0 repo create
11456a3
verified

Cion-lab commited on

train: frozen-width keeps GQA 3:1; report peak_gpu_gb and per-run tok/s
db65aa1
verified

Cion-lab commited on

train: train_ounce100m -- readback counts as verification; GQA kv must divide heads; pargs reports before optional probe
9bf38c0
verified

Cion-lab commited on

train: train_ounce100m -- download_checkpoint prefix/landing-dir fix (found by P1 v2)
4a89fe7
verified

Cion-lab commited on

train: fix SwiGLU width in build_config variants (was 2x)
b09a370
verified

Cion-lab commited on

train: train_ounce100m -- hubckpt.ensure_repo writes LFS attributes before the first checkpoint push (found by P1)
7a667d0
verified

Cion-lab commited on

train: state which tokenizer files shipped with the export
38d1f55
verified

Cion-lab commited on

train: create the ckpt repo if absent before the first push
0538d57
verified

Cion-lab commited on

train: add train_ounce100m (Phase 3/4 glue)
6ddf1d2
verified

Cion-lab commited on