pretrain: default pack-len 0 (per-example+compile = validated 57% MFU; packing needs Blackwell flash-attn) 069cb1a ceselder commited on 4 days ago
pretrain: sequence packing (block-diag attn, per-marker injection) — no-leak + inject proofs pass; MFU bench pending e6dc020 ceselder Claude Fable 5 commited on 4 days ago
pretrain: round padded L to mult-of-64 (bounds torch.compile recompiles to ~2 shapes) fe46de0 ceselder commited on 4 days ago
pretrain: MFU meter + length-bucketing + big-batch + optional compile c6f935b ceselder Claude Fable 5 commited on 4 days ago
scripts: attn sdpa (no flash-attn sm_103 build on Blackwell) af8715e ceselder Claude Fable 5 commited on 4 days ago
maxact-fast: hardening fixes (6 bugs) + Dr.GRPO vllm-lens rl.py + sbatch; Probe = universal direction 48db85f ceselder Claude Fable 5 commited on 4 days ago
maxact-fast: clean repo — DESIGN + pretraining pipeline (embed/cluster/probe-cache/build-data/pretrain), Dr. GRPO RL to follow 8505f8e ceselder Claude Fable 5 commited on 4 days ago