osoblanco commited on
Commit
0ef4714
·
verified ·
1 Parent(s): d156370

Card: full-corpus control epochs 1.3

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -41,7 +41,7 @@ Armenian SFT.
41
  token seen ~2× and every ArmSTEM token 7–9× (Gemma-tokenizer accounting).
42
  The ArmSTEM stream is a 109,885-item subset of the corpus (~30%, sampled
43
  at random with balanced stratification across the math and science
44
- pools). A control run trained on the full 373K corpus (~1.2 epochs
45
  instead of 7–9) scores within noise of the released model (paper §5), so
46
  the repetition is costless.
47
  - **Details**: global batch 512 sequences (~2.1M tokens/step), 4,770 steps,
 
41
  token seen ~2× and every ArmSTEM token 7–9× (Gemma-tokenizer accounting).
42
  The ArmSTEM stream is a 109,885-item subset of the corpus (~30%, sampled
43
  at random with balanced stratification across the math and science
44
+ pools). A control run trained on the full 373K corpus (~1.3 epochs
45
  instead of 7–9) scores within noise of the released model (paper §5), so
46
  the repetition is costless.
47
  - **Details**: global batch 512 sequences (~2.1M tokens/step), 4,770 steps,