Maggio33 commited on
Commit
517aeda
·
verified ·
1 Parent(s): 50030d6

Slayer1B 1.09: update cards and training files

Browse files
Files changed (3) hide show
  1. DATA_CARD.md +8 -5
  2. README.md +28 -13
  3. TRAINING_CONFIGURATION.md +14 -7
DATA_CARD.md CHANGED
@@ -7,14 +7,14 @@ task_categories:
7
  - text-generation
8
  tags:
9
  - pretraining
10
- - training-paused
11
  ---
12
 
13
  # Slayer1B 1.09 — staged training data
14
 
15
  **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
16
 
17
- **CPT data status, 10 November 2026: one fresh reference shard, c01, is byte-approved.** The requested model continues 1.08 with diverse new data; a ready reference is not evidence of a training-host copy or actual CPT execution. The earlier from-scratch attempt is abandoned, and its s01–s03 approvals do not make those replay-containing inputs eligible. The complete future corpus and achieved over-100B content-unique exposure are not established.
18
 
19
  ## Target, units and exposure
20
 
@@ -36,6 +36,9 @@ TRAIN SHA256: `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1`
36
 
37
  **Runtime validation is a separate fresh CL split**: expected `val.bin` SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, 2,004,526 positions. The construction manifest's legacy Logika validation (`d27c5764…`) must **not** become the runtime validation set. Validation/held-out binaries and exclusion lists are private and are not included in this repository.
38
 
 
 
 
39
  The abandoned schedule was 95,000 updates × 1,048,576 positions = **99,614,720,000**, not exactly 100B. It is not a newly approved CPT length or a count of distinct data already seen by the starting model.
40
 
41
  The earlier random-initialization attempt used s01 v2. The references below are **historical inputs, not approved fresh CPT shards**. s01 contains ancestry data; s02/s03 need replacement Polish inputs and complete ancestral deduplication checks on reusable English material. No consumed language ratio or recovered checkpoint is established by their capacities.
@@ -46,7 +49,7 @@ The earlier random-initialization attempt used s01 v2. The references below are
46
  | s02; rebuild/checks required for CPT | 5,302,913,370 | `b4df1406b88f131ac14a431108e33a68450fbd783a748a4978dc5a99a5886bb3` |
47
  | s03; rebuild/checks required for CPT | 5,284,760,292 | `b1091fbc7bcb1a677c829dd07b288f18c353c6b923d99dd0a85338deb5f727be` |
48
 
49
- Their **15,904,047,726 positions** are historical reference capacity, **not ready fresh CPT data**. Queue `6e06f0979f27c60e524852d89660bb2febef59a8b5f21f846e0618af1235715d` belongs to the abandoned attempt. Its [old Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) last received update 450 / 471,859,200 positions; these must not be relabelled as CPT from 1.08. The proper continuation needs a separate history.
50
 
51
  The reported ancestry inventory starts 1.00 from random weights, then continues through 1.06 on `slayer-pl-8x3b@d50df04a`, 1.07 on `slayer-1.07-pretrain@6a337bbf`, and 1.08 on R1 TRAIN `b73a69ea` (including Logika and English Technical). The conservative exclusion reference covers the union of these inputs, **not a claim that every reference document was actually trained on**. Known gaps include unavailable shared-history train digests for pack-04–07 and uncertain consumed tails/partial 1.08 coverage. Used-document exclusion, exact token deduplication and normalized-text fingerprints do not prove semantic uniqueness or a lower count of inherited exposure.
52
 
@@ -79,6 +82,6 @@ The old s01 training exception does **not** authorize replay in this revised CPT
79
 
80
  GoLLeM reports LLMzSzŁ variants for 1,662/18,820 items (8.83%); exclude affected items if evaluated. Its card conflicts about PoQuAD exclusion versus partition inclusion: do not use PoQuAD without decontamination. The ancestry's reported near-deduplication and EWoK omissions remain inherited concerns; filtering new data does not erase them from the starting weights.
81
 
82
- Whole-document PII DROP changes coverage; document-drop fraction is not token-loss fraction. Independent residual checks, post-filter document/native-token counts, deduplication losses and final pins remain pending. Identifiers, extraction damage, bias and incorrect reasoning may survive.
83
 
84
- Content/code approval is **not byte approval**. c01 has the specific byte approval above; it does not approve later tranches, certify all historical content as unique or prove training has begun. Old s01 SHA256 `2ab70797649947e0c14b8067161ed8a2541585428ca52c5ac8c9a744669bbb06` remains historical, not approval for reuse. Each new tranche needs its own config, byte-derived digests, manifest and sampler identity. Tokenizer v6 SHA256 remains `0640d3bd3674d7a6e59c540d945a2ad275a12e1bcaeb6007c37a90751f9815dd`. Rights, attribution and egress are separate publication conditions.
 
7
  - text-generation
8
  tags:
9
  - pretraining
10
+ - training-in-progress
11
  ---
12
 
13
  # Slayer1B 1.09 — staged training data
14
 
15
  **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
16
 
17
+ **CPT data status, 10 November 2026: training from 1.08 started at approximately 04:50 UTC on fresh c01.** Operators reproduced the byte-approved reference on the training host and verified its identity before launch. This card describes that initial input snapshot, not the complete future corpus or achieved over-100B content-unique exposure. The earlier from-scratch attempt is abandoned; its replay-containing s01–s03 approvals do not make those inputs eligible for this continuation.
18
 
19
  ## Target, units and exposure
20
 
 
36
 
37
  **Runtime validation is a separate fresh CL split**: expected `val.bin` SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, 2,004,526 positions. The construction manifest's legacy Logika validation (`d27c5764…`) must **not** become the runtime validation set. Validation/held-out binaries and exclusion lists are private and are not included in this repository.
38
 
39
+ The separate [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) contains real positive-step training aggregates. At the first checked sample, update 20 corresponds to **20,971,520 new processed positions**. This is neither consumption of the whole c01 file nor a completed 5,000-update slot. The run's registration timestamp is not its actual training-start time; input-source ratios do not establish the language ratio of consumed windows or a globally content-unique history.
40
+
41
+
42
  The abandoned schedule was 95,000 updates × 1,048,576 positions = **99,614,720,000**, not exactly 100B. It is not a newly approved CPT length or a count of distinct data already seen by the starting model.
43
 
44
  The earlier random-initialization attempt used s01 v2. The references below are **historical inputs, not approved fresh CPT shards**. s01 contains ancestry data; s02/s03 need replacement Polish inputs and complete ancestral deduplication checks on reusable English material. No consumed language ratio or recovered checkpoint is established by their capacities.
 
49
  | s02; rebuild/checks required for CPT | 5,302,913,370 | `b4df1406b88f131ac14a431108e33a68450fbd783a748a4978dc5a99a5886bb3` |
50
  | s03; rebuild/checks required for CPT | 5,284,760,292 | `b1091fbc7bcb1a677c829dd07b288f18c353c6b923d99dd0a85338deb5f727be` |
51
 
52
+ Their **15,904,047,726 positions** are historical reference capacity, **not ready fresh CPT data**. Queue `6e06f0979f27c60e524852d89660bb2febef59a8b5f21f846e0618af1235715d` belongs to the abandoned attempt. Its [old Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) last received update 450 / 471,859,200 positions; these have not been relabelled as CPT from 1.08. The running continuation uses the distinct Track history linked above.
53
 
54
  The reported ancestry inventory starts 1.00 from random weights, then continues through 1.06 on `slayer-pl-8x3b@d50df04a`, 1.07 on `slayer-1.07-pretrain@6a337bbf`, and 1.08 on R1 TRAIN `b73a69ea` (including Logika and English Technical). The conservative exclusion reference covers the union of these inputs, **not a claim that every reference document was actually trained on**. Known gaps include unavailable shared-history train digests for pack-04–07 and uncertain consumed tails/partial 1.08 coverage. Used-document exclusion, exact token deduplication and normalized-text fingerprints do not prove semantic uniqueness or a lower count of inherited exposure.
55
 
 
82
 
83
  GoLLeM reports LLMzSzŁ variants for 1,662/18,820 items (8.83%); exclude affected items if evaluated. Its card conflicts about PoQuAD exclusion versus partition inclusion: do not use PoQuAD without decontamination. The ancestry's reported near-deduplication and EWoK omissions remain inherited concerns; filtering new data does not erase them from the starting weights.
84
 
85
+ Whole-document PII DROP changes coverage; document-drop fraction is not token-loss fraction. Independent residual checks, post-filter document/native-token counts, deduplication losses and final pins are required for each later tranche and the final corpus. The c01 evidence above does not approve those future inputs. Identifiers, extraction damage, bias and incorrect reasoning may survive.
86
 
87
+ Content/code approval is **not byte approval**. c01 has the specific byte approval above; it does not approve later tranches or certify all historical content as unique. Current execution is established separately by operator preflight and the new Track history, not by source capacity. Old s01 SHA256 `2ab70797649947e0c14b8067161ed8a2541585428ca52c5ac8c9a744669bbb06` remains historical, not approval for reuse. Each new tranche needs its own config, byte-derived digests, manifest and sampler identity. Tokenizer v6 SHA256 remains `0640d3bd3674d7a6e59c540d945a2ad275a12e1bcaeb6007c37a90751f9815dd`. Rights, attribution and egress are separate publication conditions.
README.md CHANGED
@@ -9,39 +9,54 @@ tags:
9
  - causal-lm
10
  - pretraining
11
  - continued-pretraining
12
- - training-paused
13
  ---
14
 
15
  # Slayer1B 1.09
16
 
17
  **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
18
 
19
- **Corrected plan: continued pretraining from 1.08; not started.** The 9 November 2026 decision at 23:23–23:24 UTC supersedes the earlier from-scratch interpretation. That initial attempt was interrupted and abandoned; it is not the intended 1.09 continuation. No completed 1.09 model, recovered checkpoint, new quality results or inference-ready weight release is claimed.
20
 
21
  ## Objective and starting point
22
 
23
- The requested 1.09 continues the existing 1B model with **diverse, new, deduplicated data**, aiming for **more than 100B unique tokens after continuation**. The inherited unique-data contribution, additional exposure needed and PL/EN balance must be recomputed; neither replayed training exposure nor the old 95,000-update default proves this target. The earlier fixed 21.37% addition is no longer a requirement. The deadline and approved budget remain constraints, not achieved results.
24
 
25
- The intended starting weights are **[SlayerLab/Slayer1B-1.08](https://huggingface.co/SlayerLab/Slayer1B-1.08/tree/5f854a0cc7160594d54d1788d37a44e8ebabe16c)**, not random initialization and not the abandoned attempt's checkpoint. The selected v5 launcher explicitly initializes those weights with fresh optimizer state and a new output history. Its code review is complete; clean-data bytes, hardware control and actual execution remain prerequisites.
26
 
27
  The unchanged architecture is specified as **1,073,912,851 parameters**, 20 layers, width 2,048, 16 heads, SwiGLU, RMSNorm, RoPE base 500,000, QK normalization and value residuals. The reused v6 tokenizer has vocabulary 32,768 and pin `0640d3bd3674d7a6e59c540d945a2ad275a12e1bcaeb6007c37a90751f9815dd`.
28
 
29
  Architecture and tokenizer lineage includes [SlayerLab/Slayer1B-1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07/blob/a090b54e658fb57dc5166c1fcc90cc6bf5c80106/README.md). The selected 1.08 base was published as an experiment with an **ARC-Easy retention-guard FAIL**; choosing it for further training does not erase that verdict or establish a successful 1.09.
30
 
31
- The [training configuration](TRAINING_CONFIGURATION.md) preserves exact evidence for completed 1.08 and the abandoned first attempt, explicitly separated from the revised CPT plan.
32
 
33
- ## Selected CPT plan — reviewed, not executed
34
 
35
- | Setting | Operator-selected plan |
36
  |---|---|
37
- | New training | 80,000 updates × 1,048,576 = **83,886,080,000 positions**, sixteen 5,000-update slots |
38
  | New-source allocation | **20% PL / 80% EN target**; not a measured cumulative model-language ratio |
39
- | Optimizer / schedule | Muon 0.02 plus AdamW peak 0.0006; 2,000-update warmup; final 20% WSD decay to 0.00006 |
40
- | Checkpoints | Full state every 200 updates; slot-end weight milestones; local workstation copies |
41
  | Code | Launcher v5 SHA256 `796c71016c1e167bfd3ca98b78518f91c3502a96eac233dd7186ef6639fcc656`; exact command and converter pins in the configuration document |
42
 
43
  The lineage-based planning count is **at least about 18.86B inherited positions disjoint by source files**, with an upper estimate of about 23B. The lower count, **18,863,257,682**, was recomputed from deterministic sampler coverage and step logs and independently repeated in review; it does not multiply a source share by an epoch fraction. If all 80,000 updates consume new deduplicated positions once, the corresponding planned source-position total is at least **102,749,337,682**, approximately **102.7–107B**. **Historical content-level near-duplicates across sources have not been deducted**; this source-position calculation alone does not certify the requested globally content-unique total or actual completion.
44
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  ## Archived from-scratch attempt — not the CPT recipe
46
 
47
  | Setting | Abandoned first-attempt configuration |
@@ -55,7 +70,7 @@ The lineage-based planning count is **at least about 18.86B inherited positions
55
 
56
  The abandoned plan's **95,000 × 1,048,576 = 99,614,720,000 positions** was not exactly 100B and is **not the approved length of the revised continuation**. Its nineteen planned 5,000-update slots are historical configuration, not a measured unique-history count or evidence that those updates completed.
57
 
58
- Slot-1 GPU execution was observed before the interruption. **Do not resume that from-scratch state as the requested CPT.** The continuation needs a separate output history and Track run; historical points must not be relabelled as training from 1.08. Any future queue still requires approved byte ranges, manifest digests and preserved sampler identity within that particular experiment.
59
 
60
  ## Archived first-attempt evidence — not CPT progress
61
 
@@ -63,13 +78,13 @@ The [public Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cb
63
 
64
  The last received public sample before the pause is update **450**, training loss **4.3910**, with **471,859,200 processed positions**. Its ingestion timestamp is 22:36:50 UTC. It is not a live GPU reading, a final exposure count or proof of a checkpoint at update 500. The trainer's recoverable step must be checked from the saved state, not inferred from elapsed time or the tracker label.
65
 
66
- Numerical control on the resumed hardware remains pending and is required before the new CPT launch. The earlier permission to start the abandoned slot does not establish that control, approve initialization from 1.08 or approve new data bytes.
67
 
68
  Checkpoint copies are currently directed to a trusted local workstation, **not Hugging Face**. The retention plan is slot-end weight milestones plus the latest full optimizer checkpoint. That plan does not prove a successful local copy or recovery; publication of documents and code does not imply publication of model weights.
69
 
70
  ## Data and exposure
71
 
72
- The first fresh CPT reference, **c01**, is now byte-approved: **5,347,737,900 input positions**, about 80% English-source FineWeb-Edu and 20% Polish-source GoLLeM web. Its TRAIN SHA256 is `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1`; approval of the reference is not proof that it is present on the training host or that CPT has started. A 5,000-update slot processes 5,242,880,000 positions, not the entire input capacity. The older s01–s03 references remain historical, not fresh CPT inputs; see [the data card](DATA_CARD.md).
73
 
74
  The PL/EN balance will follow the approved fresh-data inventory. The user explicitly allows a different ratio; source buckets still do not establish the language share of actually consumed windows. Missing ancestral data must be reported as a limitation, not silently counted as verified unique exposure.
75
 
 
9
  - causal-lm
10
  - pretraining
11
  - continued-pretraining
12
+ - training-in-progress
13
  ---
14
 
15
  # Slayer1B 1.09
16
 
17
  **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
18
 
19
+ **Continued pretraining from 1.08 started on 10 November 2026 at approximately 04:50 UTC, on eight B200 GPUs.** This is the intended 1.09 continuation, not the earlier interrupted from-scratch attempt. The 9 November decision at 23:23–23:24 UTC supersedes that earlier interpretation. Training is in progress; no completed 1.09 model, final quality verdict, verified local restore or inference-ready weight release is claimed.
20
 
21
  ## Objective and starting point
22
 
23
+ The requested 1.09 continues the existing 1B model with **diverse, new, deduplicated data**, aiming for **more than 100B unique tokens after continuation**. Inherited content-unique history and the cumulative PL/EN balance remain unresolved; the reviewed source/file-position planning count below does not certify them. Neither replayed exposure nor the old 95,000-update default proves the target. The earlier fixed 21.37% addition is no longer a requirement. The deadline and approved budget remain constraints, not achieved results.
24
 
25
+ The starting weights are **[SlayerLab/Slayer1B-1.08](https://huggingface.co/SlayerLab/Slayer1B-1.08/tree/5f854a0cc7160594d54d1788d37a44e8ebabe16c)**, not random initialization or the abandoned attempt's checkpoint. Operators verified step-zero `INIT-FROM`, fresh optimizer state, approved c01 bytes, fresh numerical control and successful preflight before the observed updates. The selected public v5 launcher uses a separate output history for this continuation.
26
 
27
  The unchanged architecture is specified as **1,073,912,851 parameters**, 20 layers, width 2,048, 16 heads, SwiGLU, RMSNorm, RoPE base 500,000, QK normalization and value residuals. The reused v6 tokenizer has vocabulary 32,768 and pin `0640d3bd3674d7a6e59c540d945a2ad275a12e1bcaeb6007c37a90751f9815dd`.
28
 
29
  Architecture and tokenizer lineage includes [SlayerLab/Slayer1B-1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07/blob/a090b54e658fb57dc5166c1fcc90cc6bf5c80106/README.md). The selected 1.08 base was published as an experiment with an **ARC-Easy retention-guard FAIL**; choosing it for further training does not erase that verdict or establish a successful 1.09.
30
 
31
+ The [training configuration](TRAINING_CONFIGURATION.md) records the running CPT recipe and preserves completed 1.08 and the abandoned first attempt as separate historical evidence.
32
 
33
+ ## Selected CPT recipe — execution started
34
 
35
+ | Setting | Operator-selected recipe |
36
  |---|---|
37
+ | Planned new training | 80,000 updates × 1,048,576 = **83,886,080,000 positions**, sixteen 5,000-update slots; not completed exposure |
38
  | New-source allocation | **20% PL / 80% EN target**; not a measured cumulative model-language ratio |
39
+ | Optimizer / schedule | Muon peak 0.02 plus AdamW peak 0.0006; 2,000-update warmup; final 20% WSD decay to 0.00006 |
40
+ | Checkpoints | Full state every 200 updates; slot-end weight milestones; local workstation copies are a plan, not verified here |
41
  | Code | Launcher v5 SHA256 `796c71016c1e167bfd3ca98b78518f91c3502a96eac233dd7186ef6639fcc656`; exact command and converter pins in the configuration document |
42
 
43
  The lineage-based planning count is **at least about 18.86B inherited positions disjoint by source files**, with an upper estimate of about 23B. The lower count, **18,863,257,682**, was recomputed from deterministic sampler coverage and step logs and independently repeated in review; it does not multiply a source share by an epoch fraction. If all 80,000 updates consume new deduplicated positions once, the corresponding planned source-position total is at least **102,749,337,682**, approximately **102.7–107B**. **Historical content-level near-duplicates across sources have not been deducted**; this source-position calculation alone does not certify the requested globally content-unique total or actual completion.
44
 
45
+ ## Observed CPT start — not a completed model
46
+
47
+ The active [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) has a new identity and real training aggregates. Its 02:27 UTC registration timestamp is **not** the 04:50 UTC training start reported by operators. Fresh hardware control, initialization from 1.08, world size eight and preflight were independently checked by the operators.
48
+
49
+ | First checked public sample | Value |
50
+ |---|---:|
51
+ | Update | 20 |
52
+ | Training loss | 2.486659 |
53
+ | Interval throughput | 578,293 positions/s |
54
+ | New processed positions | 20,971,520 |
55
+ | AdamW learning rate during warmup | 0.000006 |
56
+
57
+ This is an early sample, not the latest step; use Track for current progress. Interval throughput is not full-window or billed throughput, and training loss on the new mixture is not a quality evaluation or directly comparable with 1.08 loss on different data. No completed slot, global content-unique total or checkpoint copy/restore is established by this sample.
58
+
59
+
60
  ## Archived from-scratch attempt — not the CPT recipe
61
 
62
  | Setting | Abandoned first-attempt configuration |
 
70
 
71
  The abandoned plan's **95,000 × 1,048,576 = 99,614,720,000 positions** was not exactly 100B and is **not the approved length of the revised continuation**. Its nineteen planned 5,000-update slots are historical configuration, not a measured unique-history count or evidence that those updates completed.
72
 
73
+ Slot-1 GPU execution was observed before that historical interruption. **Do not resume the from-scratch state as the requested CPT.** The running continuation uses its own output history and Track identity; historical points have not been relabelled as training from 1.08. Each queue extension still requires approved byte ranges, manifest digests and preserved sampler identity within its experiment.
74
 
75
  ## Archived first-attempt evidence — not CPT progress
76
 
 
78
 
79
  The last received public sample before the pause is update **450**, training loss **4.3910**, with **471,859,200 processed positions**. Its ingestion timestamp is 22:36:50 UTC. It is not a live GPU reading, a final exposure count or proof of a checkpoint at update 500. The trainer's recoverable step must be checked from the saved state, not inferred from elapsed time or the tracker label.
80
 
81
+ Fresh numerical control and step-zero `INIT-FROM` preflight were verified for the CPT start described above. The earlier permission for the abandoned slot was not reused as that evidence. Any later physical-host restart still requires its own numerical control; successful start does not prove a checkpoint restore.
82
 
83
  Checkpoint copies are currently directed to a trusted local workstation, **not Hugging Face**. The retention plan is slot-end weight milestones plus the latest full optimizer checkpoint. That plan does not prove a successful local copy or recovery; publication of documents and code does not imply publication of model weights.
84
 
85
  ## Data and exposure
86
 
87
+ The first fresh CPT reference, **c01**, has **5,347,737,900 input positions**, about 80% English-source FineWeb-Edu and 20% Polish-source GoLLeM web. Operators reproduced it on the training host and checked TRAIN SHA256 `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1` before the observed CPT start. A complete 5,000-update slot would process 5,242,880,000 positions, not the entire input capacity. Neither available capacity nor the early updates establish slot completion. Older s01–s03 references remain historical, not fresh CPT inputs; see [the data card](DATA_CARD.md).
88
 
89
  The PL/EN balance will follow the approved fresh-data inventory. The user explicitly allows a different ratio; source buckets still do not establish the language share of actually consumed windows. Missing ancestral data must be reported as a limitation, not silently counted as verified unique exposure.
90
 
TRAINING_CONFIGURATION.md CHANGED
@@ -2,17 +2,17 @@
2
 
3
  Fabryka AI / SlayerLab
4
 
5
- **Current intent: 1.09 is continued pretraining from 1.08, not a new from-scratch model.** The v5 recipe is selected and reviewed; the first clean reference c01 is byte-approved. Host-side reproduction or transfer, byte checks, fresh validation, numerical control and actual preflight are still required before claiming CPT execution. Later sections preserve completed 1.08 and the abandoned first attempt separately.
6
 
7
  ## Required separation for the revised continuation
8
 
9
  The selected base is [SlayerLab/Slayer1B-1.08 at 5f854a0cc7160594d54d1788d37a44e8ebabe16c](https://huggingface.co/SlayerLab/Slayer1B-1.08/tree/5f854a0cc7160594d54d1788d37a44e8ebabe16c). Use a new output history and Track identity; do not resume the random-initialized attempt or relabel its points as CPT. The launcher pins model SHA256 `1c510d9a59d3b9e62360d1db57e08fa450fb9b09df849df7f2eef132ea6715a4` and config SHA256 `c4c56250100cbb26e1394977ecb613fd41bdb239128592b12a13b30613fa6a94`.
10
 
11
- The approved standalone c01 reference has TRAIN SHA256 `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1` and occupies input range `[0, 5,347,737,900)`. Its initial one-slot queue snapshot has SHA256 `48144a407bcd3b2da8ad9106e0f8d94eba4335bf08dcbb0492f99acbdce91d98`. Separately approved appended queues may extend it while preserving the existing prefix; these initial digests identify the original snapshots, not the whole growing file after an append. **`D/val.bin` must be the fresh CL validation file**, SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, not the legacy Logika validation in the construction output. These are identity requirements, not a claim that host-side verification has already passed.
12
 
13
- The archived v3 invocation reference later in this document is historical evidence only. The reviewed v5 command below is the selected continuation recipe, **not evidence of execution**. Checkpoints are currently destined for the local workstation, not a public weight upload.
14
 
15
- ## Selected v5 CPT command — not yet executed
16
 
17
  | Component | Pinned identity |
18
  |---|---|
@@ -39,13 +39,20 @@ Shared architecture, batch and optimizer peaks below are unchanged: eight B200 r
39
 
40
  Counting footprint: counter SHA256 `abb39b8ce019c1a18a19ccf0ef4c32977c0c5a8f91496d4560125838789150f5`, using EpochSampler from trainer SHA `46f83b51…` with seed 20261109, 2,800,806,882 positions, block 4,096, batch 256 and 1,709 updates. The sampled target intersections with `[0, 1,925,806,163)` contribute 1,232,346,307 positions; the other terms are 13,848 × 1,048,576 = 14,520,680,448 and a conservative 3,110,230,927 Polish positions from 1.07. Review repeated offset 2,959, 683,790 windows/epoch and 437,504 used windows. These terms count source-disjoint positions, not a content-deduplicated historical union.
41
 
 
 
 
 
 
 
 
42
  ## Historical code identities
43
 
44
  | Component | Identity |
45
  |---|---|
46
  | Training implementation for completed 1.08 and abandoned first 1.09 attempt | `train_gpt_ref_r24_2.py`, SHA256 `46f83b51fcec18ffc3273d80fb9eab29e182c52d8667800dab27b80f7ff18afc` |
47
  | First 1.09 launch attempt | `uruchom_109.sh`, SHA256 `dfee11522be49c938f169ca6992cf181f86a3c4e0d26fd5b5fa3646535dc890f` |
48
- | Launcher used for observed 1.09 slot 1 | `uruchom_109.sh` v3, SHA256 `c0a2699d1a2d7ce4cddd873ad92ff5ab5bb0dbcecc6e47d6bce01ee9b848895c` |
49
  | Saved, actually used 1.08 argument file | SHA256 `d283bcb836e9bb89d5bf7dfa60487055086cf04620ad5c5dd36fe3435f2766a9` |
50
 
51
  The older public `train_gpt_ref_r23.py` is **not the identity of this training implementation**. The inference-compatible architecture code packaged with a weight release and the executable used to train it are distinct artifacts; filenames alone do not establish identity.
@@ -97,12 +104,12 @@ The historical call explicitly set `S=5000` because v3 retained a different defa
97
 
98
  A future approved queue must publish new shard bytes before matching manifest ranges and digests. Old queue `6e06f097` and the replay-containing s01–s03 references do not satisfy the revised freshness requirement; new CPT inputs need new approval. Preserve sampler identity only within the same experiment, never by reusing the abandoned output history.
99
 
100
- Recovery with the same output directory is valid only within one approved experiment. The new CPT must not inherit the abandoned attempt's optimizer or random-weight checkpoint. Full optimizer snapshots and weights-only milestones are distinct artifacts; the actual CPT optimizer and recovery policy must be recorded with its reviewed launcher.
101
 
102
  Historical v3 could adjust total steps before decay boundaries. That capability neither selects a length for the new CPT nor turns exposure into unique-source tokens. Recompute the required addition from supported historical/new-data counts and the approved budget before publishing a definitive schedule.
103
 
104
  ## What this does not claim
105
 
106
- The saved 1.08 arguments and first-attempt preflight establish those historical executions only. The old [Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) records from-scratch metrics through update 450, not CPT progress or a verified recovered checkpoint. No actual revised CPT execution or quality result is established.
107
 
108
  The revised continuation also requires its newly approved data bytes, manifest, full-ancestry overlap controls and filtering artifacts. Public code inspection does not supply those data or protected evaluation material. No held-out questions, gold answers, generated evaluation outputs or private exclusion lists belong in this public bundle. See the [model card](README.md) and [data card](DATA_CARD.md) for current scope and limitations.
 
2
 
3
  Fabryka AI / SlayerLab
4
 
5
+ **Current execution: continued pretraining from 1.08 started on 10 November 2026 at approximately 04:50 UTC, on eight B200 GPUs.** Operators verified the fresh c01 input, CL validation identity, fresh numerical control, step-zero `INIT-FROM` with a fresh optimizer and successful preflight. Real training steps are public in the separate CPT run. This is an initial execution receipt, not a completed model or final evaluation. Later sections preserve completed 1.08 and the abandoned first attempt separately.
6
 
7
  ## Required separation for the revised continuation
8
 
9
  The selected base is [SlayerLab/Slayer1B-1.08 at 5f854a0cc7160594d54d1788d37a44e8ebabe16c](https://huggingface.co/SlayerLab/Slayer1B-1.08/tree/5f854a0cc7160594d54d1788d37a44e8ebabe16c). Use a new output history and Track identity; do not resume the random-initialized attempt or relabel its points as CPT. The launcher pins model SHA256 `1c510d9a59d3b9e62360d1db57e08fa450fb9b09df849df7f2eef132ea6715a4` and config SHA256 `c4c56250100cbb26e1394977ecb613fd41bdb239128592b12a13b30613fa6a94`.
10
 
11
+ The approved standalone c01 reference has TRAIN SHA256 `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1` and occupies input range `[0, 5,347,737,900)`. Its initial one-slot queue snapshot has SHA256 `48144a407bcd3b2da8ad9106e0f8d94eba4335bf08dcbb0492f99acbdce91d98`. Separately approved appended queues may extend it while preserving the existing prefix; these initial digests identify the original snapshots, not the whole growing file after an append. **`D/val.bin` is the fresh CL validation file**, SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, not the legacy Logika validation in the construction output. Operators verified these identities for the initial launch; each later append still needs its own approved byte checks.
12
 
13
+ The archived v3 invocation reference later in this document is historical evidence only. The public v5 command below reproduces the selected running recipe; the operator and Track receipts establish execution separately. Working checkpoints are destined for the local workstation, not a public weight upload; a copy or restore requires its own receipt.
14
 
15
+ ## Selected v5 CPT command — execution started
16
 
17
  | Component | Pinned identity |
18
  |---|---|
 
39
 
40
  Counting footprint: counter SHA256 `abb39b8ce019c1a18a19ccf0ef4c32977c0c5a8f91496d4560125838789150f5`, using EpochSampler from trainer SHA `46f83b51…` with seed 20261109, 2,800,806,882 positions, block 4,096, batch 256 and 1,709 updates. The sampled target intersections with `[0, 1,925,806,163)` contribute 1,232,346,307 positions; the other terms are 13,848 × 1,048,576 = 14,520,680,448 and a conservative 3,110,230,927 Polish positions from 1.07. Review repeated offset 2,959, 683,790 windows/epoch and 437,504 used windows. These terms count source-disjoint positions, not a content-deduplicated historical union.
41
 
42
+ ## Current CPT execution receipt
43
+
44
+ Operators confirmed the 04:50 UTC start: fresh 1.07 hardware control within the 0.001 tolerance, pinned 1.08 weights/config converted to the training format, step-zero `INIT-FROM`, fresh optimizer, world size eight, c01/queue/CL validation identities and `PRE-FLIGHT OK`. The inherited 1.08 checkpoint step 1,709 is source metadata, **not the new continuation's initial training counter**.
45
+
46
+ The distinct [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) contains real updates from step 10 onward; its 02:27 UTC registration is not the 04:50 UTC training start. The [model card](README.md) records the first checked step-20 sample. Those early measurements do not establish full-window/billed throughput, completed training, quality acceptance or local checkpoint recovery.
47
+
48
+
49
  ## Historical code identities
50
 
51
  | Component | Identity |
52
  |---|---|
53
  | Training implementation for completed 1.08 and abandoned first 1.09 attempt | `train_gpt_ref_r24_2.py`, SHA256 `46f83b51fcec18ffc3273d80fb9eab29e182c52d8667800dab27b80f7ff18afc` |
54
  | First 1.09 launch attempt | `uruchom_109.sh`, SHA256 `dfee11522be49c938f169ca6992cf181f86a3c4e0d26fd5b5fa3646535dc890f` |
55
+ | Launcher used for the archived from-scratch 1.09 slot 1 | `uruchom_109.sh` v3, SHA256 `c0a2699d1a2d7ce4cddd873ad92ff5ab5bb0dbcecc6e47d6bce01ee9b848895c` |
56
  | Saved, actually used 1.08 argument file | SHA256 `d283bcb836e9bb89d5bf7dfa60487055086cf04620ad5c5dd36fe3435f2766a9` |
57
 
58
  The older public `train_gpt_ref_r23.py` is **not the identity of this training implementation**. The inference-compatible architecture code packaged with a weight release and the executable used to train it are distinct artifacts; filenames alone do not establish identity.
 
104
 
105
  A future approved queue must publish new shard bytes before matching manifest ranges and digests. Old queue `6e06f097` and the replay-containing s01–s03 references do not satisfy the revised freshness requirement; new CPT inputs need new approval. Preserve sampler identity only within the same experiment, never by reusing the abandoned output history.
106
 
107
+ Recovery with the same output directory is valid only within this approved continuation. It must not inherit the abandoned attempt's optimizer or random-weight checkpoint. The selected v5 recipe above records fresh optimizer initialization and checkpoint cadence; full optimizer snapshots and weights-only milestones are distinct artifacts. Successful launch alone is not proof of a local copy or restore.
108
 
109
  Historical v3 could adjust total steps before decay boundaries. That capability neither selects a length for the new CPT nor turns exposure into unique-source tokens. Recompute the required addition from supported historical/new-data counts and the approved budget before publishing a definitive schedule.
110
 
111
  ## What this does not claim
112
 
113
+ The saved 1.08 arguments and first-attempt preflight establish those historical executions only. The old [Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) records from-scratch metrics through update 450, not CPT progress or a verified recovered checkpoint. The current continuation's execution is established by the separate receipts above; no final quality result or completed 80,000-update model is claimed.
114
 
115
  The revised continuation also requires its newly approved data bytes, manifest, full-ancestry overlap controls and filtering artifacts. Public code inspection does not supply those data or protected evaluation material. No held-out questions, gold answers, generated evaluation outputs or private exclusion lists belong in this public bundle. See the [model card](README.md) and [data card](DATA_CARD.md) for current scope and limitations.