Model card (run in progress)

README.md

@@ -0,0 +1,46 @@
+---
+license: apache-2.0
+library_name: pytorch
+tags:
+- polish
+- checkpoints
+- training-history
+---
+
+# Slayer1B 1.02 — training checkpoints
+
+Full trainer checkpoints of the run that produces Slayer1B 1.02: continued training of
+[Slayer1B 1.01](https://huggingface.co/SlayerLab/Slayer1B-1.01) from its last checkpoint before the learning-rate decay
+([`ckpt_2250.pt`](https://huggingface.co/SlayerLab/Slayer1B-1.01-checkpoints), sha256 `aeddfb02…`), with a new total
+budget of twice the tokens of 1.01. Each file holds the model weights **and the optimizer state**.
+
+| | |
+|---|---|
+| Start | 1.01 `ckpt_2250.pt` (step 2250, constant lr 6e-4) |
+| Steps | 5720 in total × 1 048 576 tokens = 5 997 854 720 tokens (≈ 5.6 tokens per parameter; 1.01: 2860 steps) |
+| Schedule | constant lr 6e-4 through step 4577, then 1 − sqrt decay over steps 4578–5720 to lr / 10 |
+| Data | steps 1–1801: pack-00 stream (as 1.00 and 1.01); 1802–3594: pack-01 stream (as 1.01, continued); 3595–5389: pack-02; 5390–5720: pack-03 — all from [SlayerLab/slayer-pl-8x3b](https://huggingface.co/datasets/SlayerLab/slayer-pl-8x3b) revision `d50df04a`; shared components of each pack disjoint from those already used |
+| Checkpoints | every 150 steps from 2400, **step 4576** (end of the constant-lr phase, starting point for further continued training) and the final step 5720 |
+| Trainer | `train_gpt_ref_r24.py` = the 1.00/1.01 trainer plus a chain of data streams and extra checkpoint steps; batches up to step 3594 are identical to those of 1.01's data order |
+| Training curve and evaluations during training | https://track.fabryka.ai/run/21ae1234-82c2-42b7-887b-cbcc69901ae1 |
+
+The run is in progress; checkpoints are added as they are written. Model card with evaluation results:
+SlayerLab/Slayer1B-1.02 (after the run).
+
+## File format
+
+`torch.save` dictionary of the trainer: `model`, `opt` (Muon + AdamW), `step`, `config`. Load with
+`torch.load(path, map_location="cpu", weights_only=True)`. Model class: `train_gpt_ref_r23.py` in the 1.00/1.01 model
+repositories (the architecture is unchanged).
+
+## SHA-256
+
+Recorded off the training node at the moment each checkpoint is written, before upload; files are downloaded back and
+checked against these sums. The list is extended as the run progresses.
+
+```
+```
+
+## License
+
+Apache-2.0.