Card: full-corpus control epochs 1.3
Browse files
README.md
CHANGED
|
@@ -41,7 +41,7 @@ Armenian SFT.
|
|
| 41 |
token seen ~2× and every ArmSTEM token 7–9× (Gemma-tokenizer accounting).
|
| 42 |
The ArmSTEM stream is a 109,885-item subset of the corpus (~30%, sampled
|
| 43 |
at random with balanced stratification across the math and science
|
| 44 |
-
pools). A control run trained on the full 373K corpus (~1.
|
| 45 |
instead of 7–9) scores within noise of the released model (paper §5), so
|
| 46 |
the repetition is costless.
|
| 47 |
- **Details**: global batch 512 sequences (~2.1M tokens/step), 4,770 steps,
|
|
|
|
| 41 |
token seen ~2× and every ArmSTEM token 7–9× (Gemma-tokenizer accounting).
|
| 42 |
The ArmSTEM stream is a 109,885-item subset of the corpus (~30%, sampled
|
| 43 |
at random with balanced stratification across the math and science
|
| 44 |
+
pools). A control run trained on the full 373K corpus (~1.3 epochs
|
| 45 |
instead of 7–9) scores within noise of the released model (paper §5), so
|
| 46 |
the repetition is costless.
|
| 47 |
- **Details**: global batch 512 sequences (~2.1M tokens/step), 4,770 steps,
|