Text Generation
PyTorch
Safetensors
Polish
English
polish
causal-lm
from-scratch
Maggio33 commited on
Commit
117a801
·
verified ·
1 Parent(s): 9134eb9

Add final step 22327 to the evaluation table

Browse files
Files changed (2) hide show
  1. README.md +15 -13
  2. slayer1b_107_training.png +2 -2
README.md CHANGED
@@ -20,9 +20,9 @@ Decoder-only language model, 1 073 912 851 parameters, Polish-dominant. **Slayer
20
  Slayer1B 1.06** from its final checkpoint (step 13848) on 8 891 005 707 training tokens (about 7.0 B new and 1.9 B
21
  replayed from the 1.06 data), at the same constant learning rate and global batch.
22
 
23
- **Status: training complete at step 22327 (6 November 2026).** The training-time evaluation table below currently
24
- ends at step 19000; the checkpoints after it (20000, 21000, 21450, 22000, 22327) are being evaluated with the same
25
- protocol, and each result is added here when it is ready. A 1.07 model release (converted weights) has not been
26
  published in this repository yet. Native training checkpoints (raw PyTorch training state `ckpt_N.pt`, not a
27
  ready-to-use model) are published file by file in
28
  [SlayerLab/Slayer1B-1.07-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.07-checkpoints), with sha256 sums.
@@ -81,18 +81,20 @@ ARC-Easy-PL eu20). 1.07 checkpoints are evaluated on CPU; the 1.06 checkpoint (s
81
  setup matches the 1.06 card values at card rounding. These are **not** the lm-evaluation-harness benchmark used in the
82
  Results table of 1.00–1.03.
83
 
84
- | metric | 1.06 (13848) | 13998 | 14000 | 15000 | 15730 | 16000 | 17000 | 18000 | 18590 | **19000** |
85
- |---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
86
- | BLiMP (EN), accuracy % | 76.12 | 76.29 | 76.33 | 77.94 | 77.32 | 77.13 | 77.61 | 77.88 | 77.46 | **77.74** |
87
- | ARC-Easy (EN), accuracy % | 52.99 | 54.34 | 54.21 | 54.59 | 55.64 | 56.73 | 56.48 | 56.52 | 56.52 | **56.65** |
88
- | WikiText-2, byte perplexity ↓ | 2.2849 | 2.2618 | 2.2601 | 2.2321 | 2.2237 | 2.2205 | 2.2138 | 2.2042 | 2.2145 | **2.2029** |
89
- | WikiText-2, token perplexity ↓ | 30.03 | 28.80 | 28.71 | 27.28 | 26.85 | 26.70 | 26.37 | 25.90 | 26.40 | **25.84** |
90
- | MultiBLiMP (PL), accuracy % | 98.41 | 98.66 | 98.59 | 98.62 | 98.62 | 98.81 | 98.87 | 98.84 | 98.62 | **98.53** |
91
- | ARC-Easy (PL, eu20), accuracy % | 45.03 | 45.37 | 44.91 | 45.92 | 45.75 | 46.25 | 45.41 | 46.21 | 47.14 | **46.80** |
92
 
93
  ARC-Easy has 2 376 questions per language, so differences below about one percentage point between neighbouring
94
- checkpoints are within noise. WikiText-2 byte perplexity at 18590 was 0.010 higher than at 18000; at 19000 it is the
95
- lowest so far (2.2029). Each point is a single measurement of a constant-learning-rate run. The training loss in the figure is measured
 
 
96
  on the 1.07 data mixture (44 % English), so it is not directly comparable with the training loss of 1.00–1.06.
97
 
98
  The benchmark caveat of the 1.03 card applies to the 1.00–1.06 part of the training. For 1.07, all nine data files
 
20
  Slayer1B 1.06** from its final checkpoint (step 13848) on 8 891 005 707 training tokens (about 7.0 B new and 1.9 B
21
  replayed from the 1.06 data), at the same constant learning rate and global batch.
22
 
23
+ **Status: training complete at step 22327 (6 November 2026).** The training-time evaluation table below includes the
24
+ final checkpoint (22327); the checkpoints between 19000 and 22327 (20000, 21000, 21450, 22000) are being evaluated
25
+ with the same protocol, and each result is added here when it is ready. A 1.07 model release (converted weights) has not been
26
  published in this repository yet. Native training checkpoints (raw PyTorch training state `ckpt_N.pt`, not a
27
  ready-to-use model) are published file by file in
28
  [SlayerLab/Slayer1B-1.07-checkpoints](https://huggingface.co/SlayerLab/Slayer1B-1.07-checkpoints), with sha256 sums.
 
81
  setup matches the 1.06 card values at card rounding. These are **not** the lm-evaluation-harness benchmark used in the
82
  Results table of 1.00–1.03.
83
 
84
+ | metric | 1.06 (13848) | 13998 | 14000 | 15000 | 15730 | 16000 | 17000 | 18000 | 18590 | 19000 | **22327 (final)** |
85
+ |---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
86
+ | BLiMP (EN), accuracy % | 76.12 | 76.29 | 76.33 | 77.94 | 77.32 | 77.13 | 77.61 | 77.88 | 77.46 | 77.74 | **77.90** |
87
+ | ARC-Easy (EN), accuracy % | 52.99 | 54.34 | 54.21 | 54.59 | 55.64 | 56.73 | 56.48 | 56.52 | 56.52 | 56.65 | **58.67** |
88
+ | WikiText-2, byte perplexity ↓ | 2.2849 | 2.2618 | 2.2601 | 2.2321 | 2.2237 | 2.2205 | 2.2138 | 2.2042 | 2.2145 | 2.2029 | **2.1837** |
89
+ | WikiText-2, token perplexity ↓ | 30.03 | 28.80 | 28.71 | 27.28 | 26.85 | 26.70 | 26.37 | 25.90 | 26.40 | 25.84 | **24.92** |
90
+ | MultiBLiMP (PL), accuracy % | 98.41 | 98.66 | 98.59 | 98.62 | 98.62 | 98.81 | 98.87 | 98.84 | 98.62 | 98.53 | **98.84** |
91
+ | ARC-Easy (PL, eu20), accuracy % | 45.03 | 45.37 | 44.91 | 45.92 | 45.75 | 46.25 | 45.41 | 46.21 | 47.14 | 46.80 | **46.93** |
92
 
93
  ARC-Easy has 2 376 questions per language, so differences below about one percentage point between neighbouring
94
+ checkpoints are within noise. WikiText-2 byte perplexity at 18590 was 0.010 higher than at 18000. The final checkpoint
95
+ (22327) has the lowest WikiText-2 perplexity and the highest English ARC-Easy accuracy of the run (+2.0 points over
96
+ 19000, together with small gains on all other metrics); the checkpoints in between are not measured yet, so the chart
97
+ joins 19000 and 22327 directly. Each point is a single measurement of a constant-learning-rate run. The training loss in the figure is measured
98
  on the 1.07 data mixture (44 % English), so it is not directly comparable with the training loss of 1.00–1.06.
99
 
100
  The benchmark caveat of the 1.03 card applies to the 1.00–1.06 part of the training. For 1.07, all nine data files
slayer1b_107_training.png CHANGED

Git LFS Details

  • SHA256: b809f9e1896b9c0fefe09765c36db28a2d99b6bcf9c6cd9a8c485d2fe3c7a02a
  • Pointer size: 131 Bytes
  • Size of remote file: 181 kB

Git LFS Details

  • SHA256: 6a2c6bfa65e6ab37f8fa354c8acc927df73e852af385b5fab008a6cd4332786c
  • Pointer size: 131 Bytes
  • Size of remote file: 179 kB