Model card: 15 checkpoints listed (21977 latest)
Browse files
README.md
CHANGED
|
@@ -9,21 +9,22 @@ tags:
|
|
| 9 |
|
| 10 |
# Slayer1B 1.07 — training checkpoints
|
| 11 |
|
| 12 |
-
Native training checkpoints of the run that
|
| 13 |
for resuming or studying training (full trainer state, not a Transformers/safetensors package): pretraining continued
|
| 14 |
from the final 1.06 checkpoint `ckpt_13848.pt` (sha256 `1fc7396c…`, published in
|
| 15 |
[Slayer1B-1.06-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.06-checkpoints)). Each file holds the model
|
| 16 |
weights **and the optimizer state**.
|
| 17 |
|
| 18 |
-
**Status: training
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
the trainer: the LFS object id (sha256)
|
|
|
|
| 22 |
|
| 23 |
| | |
|
| 24 |
|---|---|
|
| 25 |
| Start | `ckpt_13848.pt` (final state of 1.06, constant lr 6e-4) |
|
| 26 |
-
| Steps | 13849 → 22327
|
| 27 |
| Schedule | **constant lr 6e-4**, no decay |
|
| 28 |
| Data | [SlayerLab/slayer-1.07-pretrain](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain), 8 891 005 707 tokens, one pass |
|
| 29 |
| Hardware | 4 × NVIDIA RTX 4090, DDP, gradient accumulation 32; global batch identical to the 8-GPU runs |
|
|
|
|
| 9 |
|
| 10 |
# Slayer1B 1.07 — training checkpoints
|
| 11 |
|
| 12 |
+
Native training checkpoints of the run that produced [Slayer1B 1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07),
|
| 13 |
for resuming or studying training (full trainer state, not a Transformers/safetensors package): pretraining continued
|
| 14 |
from the final 1.06 checkpoint `ckpt_13848.pt` (sha256 `1fc7396c…`, published in
|
| 15 |
[Slayer1B-1.06-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.06-checkpoints)). Each file holds the model
|
| 16 |
weights **and the optimizer state**.
|
| 17 |
|
| 18 |
+
**Status: training complete at step 22327 (6 November 2026).** The table below lists only the files that are in this
|
| 19 |
+
repository now; the remaining saved checkpoints up to 22327 are added one by one. Publishing these native training
|
| 20 |
+
checkpoints is separate from a 1.07 model release (converted weights and model card), which has not been published.
|
| 21 |
+
Every file listed was compared after upload with the checksum list written by the trainer: the LFS object id (sha256)
|
| 22 |
+
and the size match.
|
| 23 |
|
| 24 |
| | |
|
| 25 |
|---|---|
|
| 26 |
| Start | `ckpt_13848.pt` (final state of 1.06, constant lr 6e-4) |
|
| 27 |
+
| Steps | 13849 → 22327 (8 479 steps × 1 048 576 tokens), 256 sequences × 4096 tokens per step |
|
| 28 |
| Schedule | **constant lr 6e-4**, no decay |
|
| 29 |
| Data | [SlayerLab/slayer-1.07-pretrain](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain), 8 891 005 707 tokens, one pass |
|
| 30 |
| Hardware | 4 × NVIDIA RTX 4090, DDP, gradient accumulation 32; global batch identical to the 8-GPU runs |
|