Commit History
-
Wave 13: serverless DiLoCo + replaysim normalization + 3 distillation losses + PRIME-RL + Monarch
Expanded the brief mid-deep-work-loop to address the user's request for serverless training-system support, replaysim dataset normalization, deeper self-distillation paper coverage, and Meta's PyTorch agentic stack tie-ins. NEW MODULES (35 tests passing): - composer_replication.distillation: SimPO (arXiv:2405.14734), TAID (arXiv:2501.16937), Entropy-Aware OPD (ICLR 2026 spotlight). 17 unit tests covering scalar/differentiable/scheduler-monotonicity and boundary-condition correctness against paper formulas. - composer_replication.diloco.serverless: ServerlessExecutor Protocol + ObjectStoreAllReduce (fsspec-backed; works with file:// + s3:// + hf:// + gs:// + az://) + LocalProcessExecutor (working) + ModalExecutor / HFJobsExecutor (skeletons, raise NotImplementedError). 9 tests including 3 multi-process tests pinning the allreduce barrier with mean-of-{0,1,2}=1 + mean-of-{0,100,200}=100 across two consecutive rounds. - composer_replication.replaysim: data-juicer adapte…b266c31