Maggio33 commited on
Commit
7e05da0
·
verified ·
1 Parent(s): 920525c

Model card: 15 checkpoints listed (21977 latest)

Browse files
Files changed (1) hide show
  1. README.md +7 -6
README.md CHANGED
@@ -9,21 +9,22 @@ tags:
9
 
10
  # Slayer1B 1.07 — training checkpoints
11
 
12
- Native training checkpoints of the run that is producing [Slayer1B 1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07),
13
  for resuming or studying training (full trainer state, not a Transformers/safetensors package): pretraining continued
14
  from the final 1.06 checkpoint `ckpt_13848.pt` (sha256 `1fc7396c…`, published in
15
  [Slayer1B-1.06-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.06-checkpoints)). Each file holds the model
16
  weights **and the optimizer state**.
17
 
18
- **Status: training in progress.** The table below lists only the files that are in this repository now; further
19
- checkpoints are added as training saves them (the planned end is step 22327, which will also be released as the final
20
- 1.07 weights in the model repository). Every file listed was compared after upload with the checksum list written by
21
- the trainer: the LFS object id (sha256) and the size match.
 
22
 
23
  | | |
24
  |---|---|
25
  | Start | `ckpt_13848.pt` (final state of 1.06, constant lr 6e-4) |
26
- | Steps | 13849 → 22327 planned (8 479 steps × 1 048 576 tokens), 256 sequences × 4096 tokens per step |
27
  | Schedule | **constant lr 6e-4**, no decay |
28
  | Data | [SlayerLab/slayer-1.07-pretrain](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain), 8 891 005 707 tokens, one pass |
29
  | Hardware | 4 × NVIDIA RTX 4090, DDP, gradient accumulation 32; global batch identical to the 8-GPU runs |
 
9
 
10
  # Slayer1B 1.07 — training checkpoints
11
 
12
+ Native training checkpoints of the run that produced [Slayer1B 1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07),
13
  for resuming or studying training (full trainer state, not a Transformers/safetensors package): pretraining continued
14
  from the final 1.06 checkpoint `ckpt_13848.pt` (sha256 `1fc7396c…`, published in
15
  [Slayer1B-1.06-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.06-checkpoints)). Each file holds the model
16
  weights **and the optimizer state**.
17
 
18
+ **Status: training complete at step 22327 (6 November 2026).** The table below lists only the files that are in this
19
+ repository now; the remaining saved checkpoints up to 22327 are added one by one. Publishing these native training
20
+ checkpoints is separate from a 1.07 model release (converted weights and model card), which has not been published.
21
+ Every file listed was compared after upload with the checksum list written by the trainer: the LFS object id (sha256)
22
+ and the size match.
23
 
24
  | | |
25
  |---|---|
26
  | Start | `ckpt_13848.pt` (final state of 1.06, constant lr 6e-4) |
27
+ | Steps | 13849 → 22327 (8 479 steps × 1 048 576 tokens), 256 sequences × 4096 tokens per step |
28
  | Schedule | **constant lr 6e-4**, no decay |
29
  | Data | [SlayerLab/slayer-1.07-pretrain](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain), 8 891 005 707 tokens, one pass |
30
  | Hardware | 4 × NVIDIA RTX 4090, DDP, gradient accumulation 32; global batch identical to the 8-GPU runs |