SFT Cold Start Subsampling

open-athena 's Collections

Chat-Aligned Bases

Releases

5.7T Mixed RLVR

10T Mixed RLVR

2.7T Math RL

Snowball LCE Issue 8977

updated 2 days ago

H100 cold-start SFT reproduction: base, six checkpoints and original 10-05 reference. Subsampling findings are provisional.

Upvote

open-athena/snowball-67b-a2b-base-262k-qk175-skew8

Text Generation • 67B • Updated 12 days ago • 156

Note Base initializer.
open-athena/Grug-67B-A2B-Datakit-SFT-Repro-262K-Step222-2026.10.25

Text Generation • 67B • Updated 2 days ago • 54

Note Reproduction checkpoint; corrected EOS generation config. Eval requires explicit thinking. Results provisional.
open-athena/Grug-67B-A2B-Datakit-SFT-Repro-262K-Step471-2026.10.25

Text Generation • 67B • Updated 2 days ago • 39

Note Reproduction checkpoint; corrected EOS generation config. Eval requires explicit thinking. Results provisional.
open-athena/Grug-67B-A2B-Datakit-SFT-Repro-262K-Step1024-2026.10.28

Text Generation • 67B • Updated 2 days ago • 50

Note Reproduction checkpoint; corrected EOS generation config. Eval requires explicit thinking. Results provisional.
open-athena/Grug-67B-A2B-Datakit-SFT-Repro-262K-Step1536-2026.10.28

Text Generation • 67B • Updated 2 days ago • 2

Note Reproduction checkpoint; corrected EOS generation config. Eval requires explicit thinking. Results provisional.
open-athena/Grug-67B-A2B-Datakit-SFT-Repro-262K-Step2048-2026.10.29

Text Generation • 67B • Updated 2 days ago • 41

Note Reproduction checkpoint; corrected EOS generation config. Eval requires explicit thinking. Results provisional.
open-athena/Grug-67B-A2B-Datakit-SFT-Repro-262K-Step2338-2026.10.29

Text Generation • 67B • Updated 2 days ago • 1

Note Reproduction checkpoint; corrected EOS generation config. Eval requires explicit thinking. Results provisional.
open-athena/Grug-67B-A2B-Datakit-SFT-262K-2026.10.05

Text Generation • 67B • Updated 12 days ago • 1.56k

Note Original 10-05 reference; separate full cold-start run.

Upvote