Slayer1B 1.09: update cards and training files
Browse files- DATA_CARD.md +6 -4
- README.md +14 -6
- TRAINING_CONFIGURATION.md +6 -3
DATA_CARD.md
CHANGED
|
@@ -7,14 +7,14 @@ task_categories:
|
|
| 7 |
- text-generation
|
| 8 |
tags:
|
| 9 |
- pretraining
|
| 10 |
-
- training-
|
| 11 |
---
|
| 12 |
|
| 13 |
# Slayer1B 1.09 — staged training data
|
| 14 |
|
| 15 |
**Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
|
| 16 |
|
| 17 |
-
**CPT data status, 10 November 2026: training from 1.08
|
| 18 |
|
| 19 |
## Target, units and exposure
|
| 20 |
|
|
@@ -36,7 +36,9 @@ TRAIN SHA256: `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1`
|
|
| 36 |
|
| 37 |
**Runtime validation is a separate fresh CL split**: expected `val.bin` SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, 2,004,526 positions. The construction manifest's legacy Logika validation (`d27c5764…`) must **not** become the runtime validation set. Validation/held-out binaries and exclusion lists are private and are not included in this repository.
|
| 38 |
|
| 39 |
-
The separate [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a)
|
|
|
|
|
|
|
| 40 |
|
| 41 |
|
| 42 |
The abandoned schedule was 95,000 updates × 1,048,576 positions = **99,614,720,000**, not exactly 100B. It is not a newly approved CPT length or a count of distinct data already seen by the starting model.
|
|
@@ -49,7 +51,7 @@ The earlier random-initialization attempt used s01 v2. The references below are
|
|
| 49 |
| s02; rebuild/checks required for CPT | 5,302,913,370 | `b4df1406b88f131ac14a431108e33a68450fbd783a748a4978dc5a99a5886bb3` |
|
| 50 |
| s03; rebuild/checks required for CPT | 5,284,760,292 | `b1091fbc7bcb1a677c829dd07b288f18c353c6b923d99dd0a85338deb5f727be` |
|
| 51 |
|
| 52 |
-
Their **15,904,047,726 positions** are historical reference capacity, **not ready fresh CPT data**. Queue `6e06f0979f27c60e524852d89660bb2febef59a8b5f21f846e0618af1235715d` belongs to the abandoned attempt. Its [old Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) last received update 450 / 471,859,200 positions; these have not been relabelled as CPT from 1.08. The
|
| 53 |
|
| 54 |
The reported ancestry inventory starts 1.00 from random weights, then continues through 1.06 on `slayer-pl-8x3b@d50df04a`, 1.07 on `slayer-1.07-pretrain@6a337bbf`, and 1.08 on R1 TRAIN `b73a69ea` (including Logika and English Technical). The conservative exclusion reference covers the union of these inputs, **not a claim that every reference document was actually trained on**. Known gaps include unavailable shared-history train digests for pack-04–07 and uncertain consumed tails/partial 1.08 coverage. Used-document exclusion, exact token deduplication and normalized-text fingerprints do not prove semantic uniqueness or a lower count of inherited exposure.
|
| 55 |
|
|
|
|
| 7 |
- text-generation
|
| 8 |
tags:
|
| 9 |
- pretraining
|
| 10 |
+
- training-paused
|
| 11 |
---
|
| 12 |
|
| 13 |
# Slayer1B 1.09 — staged training data
|
| 14 |
|
| 15 |
**Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
|
| 16 |
|
| 17 |
+
**CPT data status, 10 November 2026: execution is paused after spot preemption at approximately 05:15 UTC.** Real training from 1.08 began at approximately 04:50 UTC on fresh c01, following operator checks of its byte identity on the training host. This card describes that initial input snapshot, not the complete future corpus or achieved over-100B content-unique exposure. The earlier from-scratch attempt is abandoned; its replay-containing s01–s03 approvals do not make those inputs eligible for this continuation.
|
| 18 |
|
| 19 |
## Target, units and exposure
|
| 20 |
|
|
|
|
| 36 |
|
| 37 |
**Runtime validation is a separate fresh CL split**: expected `val.bin` SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, 2,004,526 positions. The construction manifest's legacy Logika validation (`d27c5764…`) must **not** become the runtime validation set. Validation/held-out binaries and exclusion lists are private and are not included in this repository.
|
| 38 |
|
| 39 |
+
The separate [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) preserves real positive-step history. The first checked update 20 represented **20,971,520 new processed positions**.
|
| 40 |
+
|
| 41 |
+
Before this pause, the last received public update was **670**, corresponding to **702,545,920 new processed positions**, ingested at 05:12:59 UTC. This is neither consumption of the whole c01 input nor completion of a 5,000-update slot, and it does not establish the recoverable checkpoint. Operators report checkpoint 600 separately; a local copy or restore is not verified here. The run's registration timestamp is not its training-start time; input-source ratios do not establish the language ratio of consumed windows or a globally content-unique history.
|
| 42 |
|
| 43 |
|
| 44 |
The abandoned schedule was 95,000 updates × 1,048,576 positions = **99,614,720,000**, not exactly 100B. It is not a newly approved CPT length or a count of distinct data already seen by the starting model.
|
|
|
|
| 51 |
| s02; rebuild/checks required for CPT | 5,302,913,370 | `b4df1406b88f131ac14a431108e33a68450fbd783a748a4978dc5a99a5886bb3` |
|
| 52 |
| s03; rebuild/checks required for CPT | 5,284,760,292 | `b1091fbc7bcb1a677c829dd07b288f18c353c6b923d99dd0a85338deb5f727be` |
|
| 53 |
|
| 54 |
+
Their **15,904,047,726 positions** are historical reference capacity, **not ready fresh CPT data**. Queue `6e06f0979f27c60e524852d89660bb2febef59a8b5f21f846e0618af1235715d` belongs to the abandoned attempt. Its [old Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) last received update 450 / 471,859,200 positions; these have not been relabelled as CPT from 1.08. The revised continuation has the distinct Track history linked above.
|
| 55 |
|
| 56 |
The reported ancestry inventory starts 1.00 from random weights, then continues through 1.06 on `slayer-pl-8x3b@d50df04a`, 1.07 on `slayer-1.07-pretrain@6a337bbf`, and 1.08 on R1 TRAIN `b73a69ea` (including Logika and English Technical). The conservative exclusion reference covers the union of these inputs, **not a claim that every reference document was actually trained on**. Known gaps include unavailable shared-history train digests for pack-04–07 and uncertain consumed tails/partial 1.08 coverage. Used-document exclusion, exact token deduplication and normalized-text fingerprints do not prove semantic uniqueness or a lower count of inherited exposure.
|
| 57 |
|
README.md
CHANGED
|
@@ -9,14 +9,22 @@ tags:
|
|
| 9 |
- causal-lm
|
| 10 |
- pretraining
|
| 11 |
- continued-pretraining
|
| 12 |
-
- training-
|
| 13 |
---
|
| 14 |
|
| 15 |
# Slayer1B 1.09
|
| 16 |
|
| 17 |
**Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
|
| 18 |
|
| 19 |
-
**Continued pretraining from 1.08
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
## Objective and starting point
|
| 22 |
|
|
@@ -28,9 +36,9 @@ The unchanged architecture is specified as **1,073,912,851 parameters**, 20 laye
|
|
| 28 |
|
| 29 |
Architecture and tokenizer lineage includes [SlayerLab/Slayer1B-1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07/blob/a090b54e658fb57dc5166c1fcc90cc6bf5c80106/README.md). The selected 1.08 base was published as an experiment with an **ARC-Easy retention-guard FAIL**; choosing it for further training does not erase that verdict or establish a successful 1.09.
|
| 30 |
|
| 31 |
-
The [training configuration](TRAINING_CONFIGURATION.md) records the
|
| 32 |
|
| 33 |
-
## Selected CPT recipe —
|
| 34 |
|
| 35 |
| Setting | Operator-selected recipe |
|
| 36 |
|---|---|
|
|
@@ -44,7 +52,7 @@ The lineage-based planning count is **at least about 18.86B inherited positions
|
|
| 44 |
|
| 45 |
## Observed CPT start — not a completed model
|
| 46 |
|
| 47 |
-
The
|
| 48 |
|
| 49 |
| First checked public sample | Value |
|
| 50 |
|---|---:|
|
|
@@ -70,7 +78,7 @@ This is an early sample, not the latest step; use Track for current progress. In
|
|
| 70 |
|
| 71 |
The abandoned plan's **95,000 × 1,048,576 = 99,614,720,000 positions** was not exactly 100B and is **not the approved length of the revised continuation**. Its nineteen planned 5,000-update slots are historical configuration, not a measured unique-history count or evidence that those updates completed.
|
| 72 |
|
| 73 |
-
Slot-1 GPU execution was observed before that historical interruption. **Do not resume the from-scratch state as the requested CPT.** The
|
| 74 |
|
| 75 |
## Archived first-attempt evidence — not CPT progress
|
| 76 |
|
|
|
|
| 9 |
- causal-lm
|
| 10 |
- pretraining
|
| 11 |
- continued-pretraining
|
| 12 |
+
- training-paused
|
| 13 |
---
|
| 14 |
|
| 15 |
# Slayer1B 1.09
|
| 16 |
|
| 17 |
**Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
|
| 18 |
|
| 19 |
+
**Continued pretraining from 1.08 is paused after spot preemption at approximately 05:15 UTC on 10 November 2026.** Real training began at approximately 04:50 UTC on eight B200 GPUs: about 25 minutes of training within a roughly 67-minute machine allocation. This continuation is not the abandoned from-scratch attempt, and its real history is preserved. No completed 1.09 model, final quality verdict, verified local restore or inference-ready weight release is claimed.
|
| 20 |
+
|
| 21 |
+
| Evidence at this pause | Step |
|
| 22 |
+
|---|---:|
|
| 23 |
+
| Last received public sample, ingested at 05:12:59 UTC | 670 |
|
| 24 |
+
| Last saved checkpoint reported by operators; restore not verified here | 600 |
|
| 25 |
+
|
| 26 |
+
The public sample is **not** a checkpoint receipt or a live GPU reading. Resume is awaiting approved capacity; successful recovery and local checkpoint copies need their own evidence. An automatic-restart plan does not prove that resumed training has already occurred.
|
| 27 |
+
|
| 28 |
|
| 29 |
## Objective and starting point
|
| 30 |
|
|
|
|
| 36 |
|
| 37 |
Architecture and tokenizer lineage includes [SlayerLab/Slayer1B-1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07/blob/a090b54e658fb57dc5166c1fcc90cc6bf5c80106/README.md). The selected 1.08 base was published as an experiment with an **ARC-Easy retention-guard FAIL**; choosing it for further training does not erase that verdict or establish a successful 1.09.
|
| 38 |
|
| 39 |
+
The [training configuration](TRAINING_CONFIGURATION.md) records the selected CPT recipe and its interrupted execution, preserving completed 1.08 and the abandoned first attempt as separate historical evidence.
|
| 40 |
|
| 41 |
+
## Selected CPT recipe — paused after real execution
|
| 42 |
|
| 43 |
| Setting | Operator-selected recipe |
|
| 44 |
|---|---|
|
|
|
|
| 52 |
|
| 53 |
## Observed CPT start — not a completed model
|
| 54 |
|
| 55 |
+
The separate [CPT Track history](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) preserves real training aggregates. Its 02:27 UTC registration timestamp is **not** the 04:50 UTC training start reported by operators. Fresh hardware control, initialization from 1.08, world size eight and preflight were independently checked for that start.
|
| 56 |
|
| 57 |
| First checked public sample | Value |
|
| 58 |
|---|---:|
|
|
|
|
| 78 |
|
| 79 |
The abandoned plan's **95,000 × 1,048,576 = 99,614,720,000 positions** was not exactly 100B and is **not the approved length of the revised continuation**. Its nineteen planned 5,000-update slots are historical configuration, not a measured unique-history count or evidence that those updates completed.
|
| 80 |
|
| 81 |
+
Slot-1 GPU execution was observed before that historical interruption. **Do not resume the from-scratch state as the requested CPT.** The revised continuation has its own output history and Track identity; historical points have not been relabelled as training from 1.08. Each queue extension still requires approved byte ranges, manifest digests and preserved sampler identity within its experiment.
|
| 82 |
|
| 83 |
## Archived first-attempt evidence — not CPT progress
|
| 84 |
|
TRAINING_CONFIGURATION.md
CHANGED
|
@@ -2,7 +2,10 @@
|
|
| 2 |
|
| 3 |
Fabryka AI / SlayerLab
|
| 4 |
|
| 5 |
-
**Current execution
|
|
|
|
|
|
|
|
|
|
| 6 |
|
| 7 |
## Required separation for the revised continuation
|
| 8 |
|
|
@@ -10,9 +13,9 @@ The selected base is [SlayerLab/Slayer1B-1.08 at 5f854a0cc7160594d54d1788d37a44e
|
|
| 10 |
|
| 11 |
The approved standalone c01 reference has TRAIN SHA256 `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1` and occupies input range `[0, 5,347,737,900)`. Its initial one-slot queue snapshot has SHA256 `48144a407bcd3b2da8ad9106e0f8d94eba4335bf08dcbb0492f99acbdce91d98`. Separately approved appended queues may extend it while preserving the existing prefix; these initial digests identify the original snapshots, not the whole growing file after an append. **`D/val.bin` is the fresh CL validation file**, SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, not the legacy Logika validation in the construction output. Operators verified these identities for the initial launch; each later append still needs its own approved byte checks.
|
| 12 |
|
| 13 |
-
The archived v3 invocation reference later in this document is historical evidence only. The public v5 command below reproduces the selected
|
| 14 |
|
| 15 |
-
## Selected v5 CPT command —
|
| 16 |
|
| 17 |
| Component | Pinned identity |
|
| 18 |
|---|---|
|
|
|
|
| 2 |
|
| 3 |
Fabryka AI / SlayerLab
|
| 4 |
|
| 5 |
+
**Current execution is paused after spot preemption at approximately 05:15 UTC on 10 November 2026.** Real CPT from 1.08 began at approximately 04:50 UTC on eight B200 GPUs, after verified c01/CL validation identities, fresh numerical control, step-zero `INIT-FROM` with a fresh optimizer and successful preflight. This was about 25 minutes of training, not the roughly 67-minute machine allocation. Real steps remain in the separate CPT history; neither a completed model nor a verified restore is claimed. Later sections preserve completed 1.08 and the abandoned first attempt separately.
|
| 6 |
+
|
| 7 |
+
The [model card](README.md) distinguishes the last received public sample from the operator-reported checkpoint. Recovery must use the confirmed state of this continuation, preserve its sampler identity and pass fresh controls on any new physical host. Waiting for capacity or an automatic-start configuration is not evidence of a completed `RESUME`.
|
| 8 |
+
|
| 9 |
|
| 10 |
## Required separation for the revised continuation
|
| 11 |
|
|
|
|
| 13 |
|
| 14 |
The approved standalone c01 reference has TRAIN SHA256 `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1` and occupies input range `[0, 5,347,737,900)`. Its initial one-slot queue snapshot has SHA256 `48144a407bcd3b2da8ad9106e0f8d94eba4335bf08dcbb0492f99acbdce91d98`. Separately approved appended queues may extend it while preserving the existing prefix; these initial digests identify the original snapshots, not the whole growing file after an append. **`D/val.bin` is the fresh CL validation file**, SHA256 `4c50400b983b41411ebf2f68c09b1ab554517a99af270b731a84c80f4a2636a1`, not the legacy Logika validation in the construction output. Operators verified these identities for the initial launch; each later append still needs its own approved byte checks.
|
| 15 |
|
| 16 |
+
The archived v3 invocation reference later in this document is historical evidence only. The public v5 command below reproduces the selected CPT recipe; separate receipts establish its start and current interruption. Working checkpoints are destined for the local workstation, not a public weight upload; a copy or restore requires its own receipt.
|
| 17 |
|
| 18 |
+
## Selected v5 CPT command — paused after real execution
|
| 19 |
|
| 20 |
| Component | Pinned identity |
|
| 21 |
|---|---|
|