maxact-fast / scripts /pretrain.py

Commit History

pretrain: default pack-len 0 (per-example+compile = validated 57% MFU; packing needs Blackwell flash-attn)
069cb1a

ceselder commited on

pretrain: sequence packing (block-diag attn, per-marker injection) — no-leak + inject proofs pass; MFU bench pending
e6dc020

ceselder Claude Fable 5 commited on

pretrain: round padded L to mult-of-64 (bounds torch.compile recompiles to ~2 shapes)
fe46de0

ceselder commited on

pretrain: MFU meter + length-bucketing + big-batch + optional compile
c6f935b

ceselder Claude Fable 5 commited on

scripts: attn sdpa (no flash-attn sm_103 build on Blackwell)
af8715e

ceselder Claude Fable 5 commited on

maxact-fast: hardening fixes (6 bugs) + Dr.GRPO vllm-lens rl.py + sbatch; Probe = universal direction
48db85f

ceselder Claude Fable 5 commited on

maxact-fast: clean repo — DESIGN + pretraining pipeline (embed/cluster/probe-cache/build-data/pretrain), Dr. GRPO RL to follow
8505f8e

ceselder Claude Fable 5 commited on