Maggio33 commited on
Commit
a4b6366
·
verified ·
1 Parent(s): 517aeda

Slayer1B 1.09: update cards and training files

Browse files
Files changed (3) hide show
  1. DATA_CARD.md +6 -4
  2. README.md +14 -6
  3. TRAINING_CONFIGURATION.md +6 -3
DATA_CARD.md CHANGED
@@ -7,14 +7,14 @@ task_categories:
7
  - text-generation
8
  tags:
9
  - pretraining
10
- - training-in-progress
11
  ---
12
 
13
  # Slayer1B 1.09 — staged training data
14
 
15
  **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
16
 
17
- **CPT data status, 10 November 2026: training from 1.08 started at approximately 04:50 UTC on fresh c01.** Operators reproduced the byte-approved reference on the training host and verified its identity before launch. This card describes that initial input snapshot, not the complete future corpus or achieved over-100B content-unique exposure. The earlier from-scratch attempt is abandoned; its replay-containing s01–s03 approvals do not make those inputs eligible for this continuation.
18
 
19
  ## Target, units and exposure
20
 
@@ -36,7 +36,9 @@ TRAIN SHA256: `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1`
36
 
37
  **Runtime validation is a separate fresh CL split**: expected `val.bin` SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, 2,004,526 positions. The construction manifest's legacy Logika validation (`d27c5764…`) must **not** become the runtime validation set. Validation/held-out binaries and exclusion lists are private and are not included in this repository.
38
 
39
- The separate [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) contains real positive-step training aggregates. At the first checked sample, update 20 corresponds to **20,971,520 new processed positions**. This is neither consumption of the whole c01 file nor a completed 5,000-update slot. The run's registration timestamp is not its actual training-start time; input-source ratios do not establish the language ratio of consumed windows or a globally content-unique history.
 
 
40
 
41
 
42
  The abandoned schedule was 95,000 updates × 1,048,576 positions = **99,614,720,000**, not exactly 100B. It is not a newly approved CPT length or a count of distinct data already seen by the starting model.
@@ -49,7 +51,7 @@ The earlier random-initialization attempt used s01 v2. The references below are
49
  | s02; rebuild/checks required for CPT | 5,302,913,370 | `b4df1406b88f131ac14a431108e33a68450fbd783a748a4978dc5a99a5886bb3` |
50
  | s03; rebuild/checks required for CPT | 5,284,760,292 | `b1091fbc7bcb1a677c829dd07b288f18c353c6b923d99dd0a85338deb5f727be` |
51
 
52
- Their **15,904,047,726 positions** are historical reference capacity, **not ready fresh CPT data**. Queue `6e06f0979f27c60e524852d89660bb2febef59a8b5f21f846e0618af1235715d` belongs to the abandoned attempt. Its [old Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) last received update 450 / 471,859,200 positions; these have not been relabelled as CPT from 1.08. The running continuation uses the distinct Track history linked above.
53
 
54
  The reported ancestry inventory starts 1.00 from random weights, then continues through 1.06 on `slayer-pl-8x3b@d50df04a`, 1.07 on `slayer-1.07-pretrain@6a337bbf`, and 1.08 on R1 TRAIN `b73a69ea` (including Logika and English Technical). The conservative exclusion reference covers the union of these inputs, **not a claim that every reference document was actually trained on**. Known gaps include unavailable shared-history train digests for pack-04–07 and uncertain consumed tails/partial 1.08 coverage. Used-document exclusion, exact token deduplication and normalized-text fingerprints do not prove semantic uniqueness or a lower count of inherited exposure.
55
 
 
7
  - text-generation
8
  tags:
9
  - pretraining
10
+ - training-paused
11
  ---
12
 
13
  # Slayer1B 1.09 — staged training data
14
 
15
  **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
16
 
17
+ **CPT data status, 10 November 2026: execution is paused after spot preemption at approximately 05:15 UTC.** Real training from 1.08 began at approximately 04:50 UTC on fresh c01, following operator checks of its byte identity on the training host. This card describes that initial input snapshot, not the complete future corpus or achieved over-100B content-unique exposure. The earlier from-scratch attempt is abandoned; its replay-containing s01–s03 approvals do not make those inputs eligible for this continuation.
18
 
19
  ## Target, units and exposure
20
 
 
36
 
37
  **Runtime validation is a separate fresh CL split**: expected `val.bin` SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, 2,004,526 positions. The construction manifest's legacy Logika validation (`d27c5764…`) must **not** become the runtime validation set. Validation/held-out binaries and exclusion lists are private and are not included in this repository.
38
 
39
+ The separate [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) preserves real positive-step history. The first checked update 20 represented **20,971,520 new processed positions**.
40
+
41
+ Before this pause, the last received public update was **670**, corresponding to **702,545,920 new processed positions**, ingested at 05:12:59 UTC. This is neither consumption of the whole c01 input nor completion of a 5,000-update slot, and it does not establish the recoverable checkpoint. Operators report checkpoint 600 separately; a local copy or restore is not verified here. The run's registration timestamp is not its training-start time; input-source ratios do not establish the language ratio of consumed windows or a globally content-unique history.
42
 
43
 
44
  The abandoned schedule was 95,000 updates × 1,048,576 positions = **99,614,720,000**, not exactly 100B. It is not a newly approved CPT length or a count of distinct data already seen by the starting model.
 
51
  | s02; rebuild/checks required for CPT | 5,302,913,370 | `b4df1406b88f131ac14a431108e33a68450fbd783a748a4978dc5a99a5886bb3` |
52
  | s03; rebuild/checks required for CPT | 5,284,760,292 | `b1091fbc7bcb1a677c829dd07b288f18c353c6b923d99dd0a85338deb5f727be` |
53
 
54
+ Their **15,904,047,726 positions** are historical reference capacity, **not ready fresh CPT data**. Queue `6e06f0979f27c60e524852d89660bb2febef59a8b5f21f846e0618af1235715d` belongs to the abandoned attempt. Its [old Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) last received update 450 / 471,859,200 positions; these have not been relabelled as CPT from 1.08. The revised continuation has the distinct Track history linked above.
55
 
56
  The reported ancestry inventory starts 1.00 from random weights, then continues through 1.06 on `slayer-pl-8x3b@d50df04a`, 1.07 on `slayer-1.07-pretrain@6a337bbf`, and 1.08 on R1 TRAIN `b73a69ea` (including Logika and English Technical). The conservative exclusion reference covers the union of these inputs, **not a claim that every reference document was actually trained on**. Known gaps include unavailable shared-history train digests for pack-04–07 and uncertain consumed tails/partial 1.08 coverage. Used-document exclusion, exact token deduplication and normalized-text fingerprints do not prove semantic uniqueness or a lower count of inherited exposure.
57
 
README.md CHANGED
@@ -9,14 +9,22 @@ tags:
9
  - causal-lm
10
  - pretraining
11
  - continued-pretraining
12
- - training-in-progress
13
  ---
14
 
15
  # Slayer1B 1.09
16
 
17
  **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
18
 
19
- **Continued pretraining from 1.08 started on 10 November 2026 at approximately 04:50 UTC, on eight B200 GPUs.** This is the intended 1.09 continuation, not the earlier interrupted from-scratch attempt. The 9 November decision at 23:23–23:24 UTC supersedes that earlier interpretation. Training is in progress; no completed 1.09 model, final quality verdict, verified local restore or inference-ready weight release is claimed.
 
 
 
 
 
 
 
 
20
 
21
  ## Objective and starting point
22
 
@@ -28,9 +36,9 @@ The unchanged architecture is specified as **1,073,912,851 parameters**, 20 laye
28
 
29
  Architecture and tokenizer lineage includes [SlayerLab/Slayer1B-1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07/blob/a090b54e658fb57dc5166c1fcc90cc6bf5c80106/README.md). The selected 1.08 base was published as an experiment with an **ARC-Easy retention-guard FAIL**; choosing it for further training does not erase that verdict or establish a successful 1.09.
30
 
31
- The [training configuration](TRAINING_CONFIGURATION.md) records the running CPT recipe and preserves completed 1.08 and the abandoned first attempt as separate historical evidence.
32
 
33
- ## Selected CPT recipe — execution started
34
 
35
  | Setting | Operator-selected recipe |
36
  |---|---|
@@ -44,7 +52,7 @@ The lineage-based planning count is **at least about 18.86B inherited positions
44
 
45
  ## Observed CPT start — not a completed model
46
 
47
- The active [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) has a new identity and real training aggregates. Its 02:27 UTC registration timestamp is **not** the 04:50 UTC training start reported by operators. Fresh hardware control, initialization from 1.08, world size eight and preflight were independently checked by the operators.
48
 
49
  | First checked public sample | Value |
50
  |---|---:|
@@ -70,7 +78,7 @@ This is an early sample, not the latest step; use Track for current progress. In
70
 
71
  The abandoned plan's **95,000 × 1,048,576 = 99,614,720,000 positions** was not exactly 100B and is **not the approved length of the revised continuation**. Its nineteen planned 5,000-update slots are historical configuration, not a measured unique-history count or evidence that those updates completed.
72
 
73
- Slot-1 GPU execution was observed before that historical interruption. **Do not resume the from-scratch state as the requested CPT.** The running continuation uses its own output history and Track identity; historical points have not been relabelled as training from 1.08. Each queue extension still requires approved byte ranges, manifest digests and preserved sampler identity within its experiment.
74
 
75
  ## Archived first-attempt evidence — not CPT progress
76
 
 
9
  - causal-lm
10
  - pretraining
11
  - continued-pretraining
12
+ - training-paused
13
  ---
14
 
15
  # Slayer1B 1.09
16
 
17
  **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
18
 
19
+ **Continued pretraining from 1.08 is paused after spot preemption at approximately 05:15 UTC on 10 November 2026.** Real training began at approximately 04:50 UTC on eight B200 GPUs: about 25 minutes of training within a roughly 67-minute machine allocation. This continuation is not the abandoned from-scratch attempt, and its real history is preserved. No completed 1.09 model, final quality verdict, verified local restore or inference-ready weight release is claimed.
20
+
21
+ | Evidence at this pause | Step |
22
+ |---|---:|
23
+ | Last received public sample, ingested at 05:12:59 UTC | 670 |
24
+ | Last saved checkpoint reported by operators; restore not verified here | 600 |
25
+
26
+ The public sample is **not** a checkpoint receipt or a live GPU reading. Resume is awaiting approved capacity; successful recovery and local checkpoint copies need their own evidence. An automatic-restart plan does not prove that resumed training has already occurred.
27
+
28
 
29
  ## Objective and starting point
30
 
 
36
 
37
  Architecture and tokenizer lineage includes [SlayerLab/Slayer1B-1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07/blob/a090b54e658fb57dc5166c1fcc90cc6bf5c80106/README.md). The selected 1.08 base was published as an experiment with an **ARC-Easy retention-guard FAIL**; choosing it for further training does not erase that verdict or establish a successful 1.09.
38
 
39
+ The [training configuration](TRAINING_CONFIGURATION.md) records the selected CPT recipe and its interrupted execution, preserving completed 1.08 and the abandoned first attempt as separate historical evidence.
40
 
41
+ ## Selected CPT recipe — paused after real execution
42
 
43
  | Setting | Operator-selected recipe |
44
  |---|---|
 
52
 
53
  ## Observed CPT start — not a completed model
54
 
55
+ The separate [CPT Track history](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) preserves real training aggregates. Its 02:27 UTC registration timestamp is **not** the 04:50 UTC training start reported by operators. Fresh hardware control, initialization from 1.08, world size eight and preflight were independently checked for that start.
56
 
57
  | First checked public sample | Value |
58
  |---|---:|
 
78
 
79
  The abandoned plan's **95,000 × 1,048,576 = 99,614,720,000 positions** was not exactly 100B and is **not the approved length of the revised continuation**. Its nineteen planned 5,000-update slots are historical configuration, not a measured unique-history count or evidence that those updates completed.
80
 
81
+ Slot-1 GPU execution was observed before that historical interruption. **Do not resume the from-scratch state as the requested CPT.** The revised continuation has its own output history and Track identity; historical points have not been relabelled as training from 1.08. Each queue extension still requires approved byte ranges, manifest digests and preserved sampler identity within its experiment.
82
 
83
  ## Archived first-attempt evidence — not CPT progress
84
 
TRAINING_CONFIGURATION.md CHANGED
@@ -2,7 +2,10 @@
2
 
3
  Fabryka AI / SlayerLab
4
 
5
- **Current execution: continued pretraining from 1.08 started on 10 November 2026 at approximately 04:50 UTC, on eight B200 GPUs.** Operators verified the fresh c01 input, CL validation identity, fresh numerical control, step-zero `INIT-FROM` with a fresh optimizer and successful preflight. Real training steps are public in the separate CPT run. This is an initial execution receipt, not a completed model or final evaluation. Later sections preserve completed 1.08 and the abandoned first attempt separately.
 
 
 
6
 
7
  ## Required separation for the revised continuation
8
 
@@ -10,9 +13,9 @@ The selected base is [SlayerLab/Slayer1B-1.08 at 5f854a0cc7160594d54d1788d37a44e
10
 
11
  The approved standalone c01 reference has TRAIN SHA256 `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1` and occupies input range `[0, 5,347,737,900)`. Its initial one-slot queue snapshot has SHA256 `48144a407bcd3b2da8ad9106e0f8d94eba4335bf08dcbb0492f99acbdce91d98`. Separately approved appended queues may extend it while preserving the existing prefix; these initial digests identify the original snapshots, not the whole growing file after an append. **`D/val.bin` is the fresh CL validation file**, SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, not the legacy Logika validation in the construction output. Operators verified these identities for the initial launch; each later append still needs its own approved byte checks.
12
 
13
- The archived v3 invocation reference later in this document is historical evidence only. The public v5 command below reproduces the selected running recipe; the operator and Track receipts establish execution separately. Working checkpoints are destined for the local workstation, not a public weight upload; a copy or restore requires its own receipt.
14
 
15
- ## Selected v5 CPT command — execution started
16
 
17
  | Component | Pinned identity |
18
  |---|---|
 
2
 
3
  Fabryka AI / SlayerLab
4
 
5
+ **Current execution is paused after spot preemption at approximately 05:15 UTC on 10 November 2026.** Real CPT from 1.08 began at approximately 04:50 UTC on eight B200 GPUs, after verified c01/CL validation identities, fresh numerical control, step-zero `INIT-FROM` with a fresh optimizer and successful preflight. This was about 25 minutes of training, not the roughly 67-minute machine allocation. Real steps remain in the separate CPT history; neither a completed model nor a verified restore is claimed. Later sections preserve completed 1.08 and the abandoned first attempt separately.
6
+
7
+ The [model card](README.md) distinguishes the last received public sample from the operator-reported checkpoint. Recovery must use the confirmed state of this continuation, preserve its sampler identity and pass fresh controls on any new physical host. Waiting for capacity or an automatic-start configuration is not evidence of a completed `RESUME`.
8
+
9
 
10
  ## Required separation for the revised continuation
11
 
 
13
 
14
  The approved standalone c01 reference has TRAIN SHA256 `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1` and occupies input range `[0, 5,347,737,900)`. Its initial one-slot queue snapshot has SHA256 `48144a407bcd3b2da8ad9106e0f8d94eba4335bf08dcbb0492f99acbdce91d98`. Separately approved appended queues may extend it while preserving the existing prefix; these initial digests identify the original snapshots, not the whole growing file after an append. **`D/val.bin` is the fresh CL validation file**, SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, not the legacy Logika validation in the construction output. Operators verified these identities for the initial launch; each later append still needs its own approved byte checks.
15
 
16
+ The archived v3 invocation reference later in this document is historical evidence only. The public v5 command below reproduces the selected CPT recipe; separate receipts establish its start and current interruption. Working checkpoints are destined for the local workstation, not a public weight upload; a copy or restore requires its own receipt.
17
 
18
+ ## Selected v5 CPT command — paused after real execution
19
 
20
  | Component | Pinned identity |
21
  |---|---|