Slayer1B 1.09: update cards and documentation

DATA_CARD.md

@@ -7,21 +7,28 @@ task_categories:
 - text-generation
 tags:
 - pretraining
-- training-paused
+- staged-training-data
 ---
 
 # Slayer1B 1.09 — staged training data
 
 **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
 
-**CPT data snapshot, 10 November 2026, 19:23 UTC:** the continuation from Slayer1B 1.08 is paused by the cost guard after the on-node update-11,600 checkpoint. Weights-only and full recovery checkpoints at update 10,000 have been verified on the trusted local workstation; this does not assert a local update-11,600 copy. The c01–c06 queue is byte-approved and reproduced on the training host, providing data through update 30,000. Input capacity is distinct from consumed training exposure. This card does not describe a completed future corpus or achieved over-100B content-unique history.
+**Historical CPT data snapshot, 10 November 2026, 19:23 UTC:** the continuation from Slayer1B 1.08 was paused by the cost guard after the on-node update-11,600 checkpoint. At that snapshot, weights-only and full recovery checkpoints at update 10,000 had been verified on the trusted local workstation; this does not assert a local update-11,600 copy. The c01–c06 queue was byte-approved and reproduced on the training host, providing data through update 30,000. Input capacity is distinct from consumed training exposure. This dated snapshot does not describe current hardware availability, a completed future corpus or achieved over-100B content-unique history; later execution status belongs to the linked CPT Track history and operator reports.
 
 ## Target, units and exposure
 
-The current objective is the largest useful training exposure within a **USD 425 campaign ceiling**. It replaces the earlier commitment to complete 80,000 updates and exceed 100B positions. Final length is reviewed from measured cost before WSD decay and rounded down to an affordable 5,000-update boundary; roughly 35,000–40,000 updates is a price-dependent projection, not a selected or completed endpoint. The operational trainer still retains its 80,000-update scheduling argument during the constant-rate phase. Source allocation targets 20% PL / 80% EN; this is not the cumulative language ratio of all consumed model history.
+**Planning update — 11 November 2026, 12:07 UTC:** all of the existing **USD 425 campaign budget** is allocated to the main base on the currently selected data, retaining the existing cost guards and operational reserve. No funds are reserved for a future gold-data continuation. The objective is the largest useful affordable exposure at constant learning rates after warmup, with **no in-run terminal decay** and a full optimizer-state checkpoint at the budget or allocation stop. This supersedes the earlier requirement to select an endpoint that completes WSD decay; the earlier 35,000–40,000 projections are not selected or completed endpoints. The trainer retains `KROKI=80000` as a mechanical scheduling argument, not a funded endpoint. Its configured decay begins at update 64,000, beyond the current budget; operators must review the stopping point if prices change to keep the main base at constant learning rates, rather than shortening the argument and bringing decay forward. Source allocation targets 20% PL / 80% EN; this is not the cumulative language ratio of all consumed model history.
 
 At a hypothetical completed endpoint of 40,000 updates, new exposure would be **41,943,040,000 positions**. The reviewed inherited count is **18,863,257,682 source/file-disjoint positions**. A conservative c01 duplicate correction deducts **114,627,128 positions** (32,536,363 between sources plus 82,090,765 within sources; sum of lists, not a measured union). The resulting conditional source-position total would be **60,691,670,554**, not a globally content-deduplicated count or an achieved outcome. Retained input capacity, processed positions, ancestral reference size and content uniqueness must be reported separately.
 
+## Separate future gold-data option — planning, not an assembled corpus
+
+The 11 November, 12:07 decision also calls for parallel curation of approximately **10B tokens of quality-selected data for pretraining or finishing at context 4,096**, by the dataset team with licensed books selected with Arkadiusz Słota. This prospective “gold” dataset is deliberately isolated from the main-base inputs and evaluation material. It requires its own manifest, provenance and licensing/admission gate before use, plus a separate run and budget decision; a later finishing continuation with decay is an alternative, not a requirement for the main base.
+
+No gold dataset has been assembled or released, no budget for training it has been approved, and no current USD 425 allocation is earmarked for it. “Gold” describes the intended quality-based selection, not a measured performance gain or an approved addition to the main-base queue. Existing source candidates and published upstream capacities below do not establish its composition or retained size.
+
+
 ## Byte-approved queue snapshot c01–c06
 
 | Segment | Input positions | Selected Polish source | TRAIN SHA256 |
@@ -35,7 +42,7 @@ At a hypothetical completed endpoint of 40,000 updates, new exposure would be **
 
 English-source input is FineWeb-Edu. Total approved input capacity is **32,086,749,670 positions**, including EOS. Queue SHA256 is `5b1c2755aa79db7fb6d5dcb40b17bb65465dae36fcddfe3db3f9d8e44d51cc9a`, with 5,000 updates per segment and contiguous logical ranges. Six completed stages would process **31,457,280,000 positions**, less than their input capacity; six-stage completion is not claimed. Future segments require their own approvals.
 
-The source selection changed at the c02–c03 boundary. Training-loss values across this boundary do not measure the same input distribution. Full local recovery state at update 10,000 has a verified transfer receipt; neither that receipt nor a lower training loss establishes final model quality.
+The source selection changed at the c02–c03 boundary. Training-loss values across this boundary do not measure the same input distribution. The historical 10 November snapshot records a verified transfer receipt for full local recovery state at update 10,000; neither that receipt nor a lower training loss establishes final model quality.
 
 ## Byte-approved c01 reference — input capacity, not completed exposure
 
@@ -53,7 +60,7 @@ TRAIN SHA256: `3341080241aeabd1e03c77530a6d735cb9501ecd788fba60ea065a944cd09ee1`
 
 The separate [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) preserves real positive-step history. The first checked update 20 represented **20,971,520 new processed positions**.
 
-The separate CPT Track history preserves current processed-position samples. Verified passage through update 10,000 corresponds to **10,485,760,000 processed positions** at the unchanged batch size. Weights-only checkpoints at 5,000 and 10,000 and the latest full local recovery checkpoint at 10,000 have separate transfer checksums. This completed exposure is not the capacity of all six approved segments and is not a globally content-unique count.
+The separate CPT Track history preserves later processed-position samples. At the historical 10 November, 19:23 UTC snapshot, verified passage through update 10,000 corresponded to **10,485,760,000 processed positions** at the unchanged batch size. Weights-only checkpoints at 5,000 and 10,000 and the then-latest full local recovery checkpoint at 10,000 had separate transfer checksums. This completed exposure is not the capacity of all six approved segments and is not a globally content-unique count.
 
 
 The abandoned schedule was 95,000 updates × 1,048,576 positions = **99,614,720,000**, not exactly 100B. It is not a newly approved CPT length or a count of distinct data already seen by the starting model.

README.md

@@ -16,22 +16,24 @@ tags:
 
 **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
 
-**Execution snapshot, 11 November 2026, 10:45 UTC: continued pretraining from Slayer1B 1.08 is running on eight B200 spot GPUs after an H200 segment and checkpoint recoveries. The campaign ceiling remains USD 425.** Operators reported update 27,140 at this snapshot; this is progress telemetry, not the endpoint of the run. The public tracking plan remains 40,000 updates, while the trainer retains its 80,000-update scheduling argument during the constant-rate phase. The next budget-dependent length review is at update 30,000; final length must allow completion of learning-rate decay. Neither configured value is a promise of completed exposure. This is not a completed model or inference-ready weight release.
+**Historical execution snapshot, 11 November 2026, 10:45 UTC:** continued pretraining from Slayer1B 1.08 was running on eight B200 spot GPUs after an H200 segment and checkpoint recoveries. Operators reported update 27,140 at that snapshot; this is dated progress telemetry, not a current availability claim or the endpoint of the run. The public tracking plan then showed 40,000 updates, while the trainer retained its 80,000-update scheduling argument. Neither value establishes completed exposure. The later planning decision below supersedes the requirement to complete terminal learning-rate decay. This is not a completed model or inference-ready weight release.
 
 | Verified execution evidence | Scope |
 |---|---|
 | CPT start | Approximately 04:50 UTC, 10 November 2026, on eight B200 GPUs |
 | Historical H200 recovery | Operators verified fresh numerical control and checkpoint restores during the earlier H200 segment |
-| Current compute at the dated snapshot | Eight B200 spot GPUs; training continued beyond update 27,000 |
+| Compute at the historical 10:45 UTC snapshot | Eight B200 spot GPUs; training had continued beyond update 27,000; later availability requires operator confirmation |
 | Checksum-verified weights-only copies on the workstation | Updates 5,000, 10,000, 15,000, 20,000 and 25,000; not public weight releases |
 | Latest verified full-state workstation copy | Update 25,000; includes optimizer state, retained separately from weights-only copies; not a claim about the newest recovery checkpoint on the cloud disk |
 
-The [CPT Track history](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) provides the changing public telemetry. This card is an execution and budget snapshot, not a completed 1.09 model, final quality verdict or inference-ready weight release.
+The [CPT Track history](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) provides changing public telemetry; consult it and the operators for later execution status and hardware availability. The planning update is a decision, not evidence that its future steps have executed. This card is not a completed 1.09 model, final quality verdict or inference-ready weight release.
 
 
 ## Objective and starting point
 
-The current objective is **the largest useful training exposure affordable within USD 425**, using eligible, diverse data filtered against the available ancestral references. The public tracking plan is **40,000 updates**, not a selected final trainer length: lower spot prices can permit more updates, while interruptions and other costs can reduce the affordable total. Final length must be rounded down to an affordable 5,000-update boundary before WSD decay, including the cost of completing that decay. There is no promise of 80,000 updates, more than 100B globally unique tokens or the earlier completion date. Model quality is evaluated separately; positions per dollar measure operating efficiency, not a quality score.
+**Planning update — 11 November 2026, 12:07 UTC:** all of the existing **USD 425 campaign budget** is allocated to the main base on the currently selected data, with no funds earmarked for a future gold-data continuation. The objective is the largest useful affordable exposure, retaining constant learning rates after warmup and **no in-run terminal decay**. Existing cost guards and the operational reserve remain unchanged. The main-base run ends at the budget or allocation boundary with a full optimizer-state checkpoint, not by completing WSD decay. The historical 40,000-update tracking plan is not a funded endpoint; no later endpoint is asserted here. There is no promise of 80,000 updates, more than 100B globally unique tokens or a completion date. Model quality is evaluated separately; positions per dollar measure operating efficiency, not a quality score.
+
+A separate future option is approximately **10B tokens of quality-selected “gold” data at context 4,096**, curated in parallel by the dataset team with licensed books selected with Arkadiusz Słota. It is deliberately isolated from the main base and evaluation material, requires its own manifest and admission/licensing gate, and would have its own run and budget decision. A finishing continuation with decay is an alternative for that separate run, not part of the current main-base objective. No such dataset has been assembled or released, no future training budget has been approved, and selection for quality is not evidence of a measured quality gain.
 
 The starting weights are **[SlayerLab/Slayer1B-1.08](https://huggingface.co/SlayerLab/Slayer1B-1.08/tree/5f854a0cc7160594d54d1788d37a44e8ebabe16c)**, not random initialization or the abandoned attempt's checkpoint. Operators verified step-zero `INIT-FROM`, fresh optimizer state, approved c01 bytes, fresh numerical control and successful preflight before the observed updates. The selected public v5 launcher uses a separate output history for this continuation.
 
@@ -45,12 +47,14 @@ The [training configuration](TRAINING_CONFIGURATION.md) distinguishes the still-
 
 | Setting | Operator-selected recipe |
 |---|---|
-| Configured scheduling argument | Currently 80,000 updates; this keeps the WSD constant-rate phase flexible and is not a commitment to complete 83.89B new positions |
+| Configured scheduling argument | `KROKI=80000` remains unchanged; a mechanical scheduling argument, not a funded endpoint; configured decay begins at update 64,000, beyond the current budget |
 | New-source allocation | **20% PL / 80% EN target**; not a measured cumulative model-language ratio |
-| Optimizer / schedule | Muon peak 0.02 plus AdamW peak 0.0006; 2,000-update warmup; final 20% WSD decay to 0.00006 |
+| Optimizer / schedule | Muon peak 0.02 plus AdamW peak 0.0006; 2,000-update warmup, then constant learning rates under the main-base plan; the configured final-20% WSD decay to 0.00006 is not to be entered in this run |
 | Checkpoints | Full state every 400 updates after the approved cadence change; weights-only checkpoints at each 5,000-update boundary; workstation copies verified through update 25,000 |
 | Code | Launcher v5 SHA256 `796c71016c1e167bfd3ca98b78518f91c3502a96eac233dd7186ef6639fcc656`; exact command and converter pins in the configuration document |
 
+Operators must review the affordable stopping point if prices or availability change, keeping the main-base run out of the configured decay phase. Do not shorten `KROKI` to the budget estimate: that would move decay earlier. This planning update changes no trainer or launcher code.
+
 The reviewed inherited count is **18,863,257,682 source/file-disjoint positions**, not a globally content-deduplicated history. The measured c01 duplicate correction is conservatively **114,627,128 positions**, the sum of two exclusion-list counts rather than a measured union. If a 40,000-update endpoint is selected and completed, then 41,943,040,000 new positions would give a conditional source-position total of **60,691,670,554** after that correction. Historical content-level near-duplicates remain unresolved. This projection is not evidence of completed exposure or a globally unique corpus.
 
 ## Initial CPT execution evidence
@@ -144,6 +148,6 @@ Scores are percentages with reported pointwise 95% bootstrap intervals (10,000 d
 
 ## Cost, licensing and limitations
 
-The total campaign ceiling remains **USD 425**, including GPU allocations, preparation, interruptions, storage and evaluation tracked by the resource meter. At the operator's 11 November, 10:39 UTC snapshot, the meter reported **approximately USD 239 spent**, leaving approximately **USD 186**; this is counter attribution, not a settled invoice or a permanently current balance. Hardware selection is based on effective cost per processed position and verified availability. The next budget-dependent length review is at update 30,000, before the selected endpoint's WSD decay boundary. No higher spending limit or guaranteed completion date is implied.
+The total campaign ceiling remains **USD 425**, including GPU allocations, preparation, interruptions, storage and evaluation tracked by the resource meter. At the operator's 11 November, 10:39 UTC snapshot, the meter reported **approximately USD 239 spent**, leaving approximately **USD 186**; this is counter attribution, not a settled invoice or a permanently current balance. Hardware selection is based on effective cost per processed position and verified availability. Under the 12:07 planning decision, budget reviews preserve the existing cost guards and operational reserve, with no earmark for future gold-data training and no requirement to fund terminal decay in the main base. No higher spending limit or guaranteed completion date is implied.
 
 The selected 1.08 base declares Apache-2.0; source-text rights and future publication still require separate approval. Privacy filters are heuristic; generated text may be inaccurate, biased, offensive or personal. No reasoning quality, instruction-following ability or professional reliability has been demonstrated for the revised 1.09.

TRAINING_CONFIGURATION.md

@@ -2,9 +2,9 @@
 
 Fabryka AI / SlayerLab
 
-**Execution snapshot, 10 November 2026, 19:23 UTC:** continued pretraining is paused by the cost guard after the on-node update-11,600 checkpoint. The operator stopped GPU allocation at approximately 19:21 UTC after four price readings exceeded the cost-per-token ceiling, retaining the VM and disk for recovery. Before this pause, operators confirmed continued pretraining on eight H200 GPUs after this continuation's own `RESUME @ 9600`. Real CPT from 1.08 began at approximately 04:50 UTC on eight B200 GPUs. Transfer checksums confirm weights-only updates 5,000 and 10,000 and the latest full local update-10,000 checkpoint; they do not assert a local update-11,600 copy.
+**Historical execution snapshot, 10 November 2026, 19:23 UTC:** continued pretraining was paused by the cost guard after the on-node update-11,600 checkpoint. The operator stopped GPU allocation at approximately 19:21 UTC after four price readings exceeded the cost-per-token ceiling, retaining the VM and disk for recovery. Before this pause, operators confirmed continued pretraining on eight H200 GPUs after this continuation's own `RESUME @ 9600`. Real CPT from 1.08 began at approximately 04:50 UTC on eight B200 GPUs. Transfer checksums at that snapshot confirmed weights-only updates 5,000 and 10,000 and the latest full local update-10,000 checkpoint; they do not assert a local update-11,600 copy. This is historical evidence, not current hardware availability; consult the [CPT Track history](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac47-0ea7a23f573a) and operators for later status.
 
-The current objective is maximum training exposure within **USD 425 total campaign cost**. The trainer still has a configured 80,000-update scheduling argument in its constant-rate phase; this is not an obligation to complete 80,000 updates or exceed 100B positions. Final length is selected from the measured remaining budget before the permitted WSD decay boundary.
+**Planning update — 11 November 2026, 12:07 UTC:** all of the existing **USD 425 total campaign budget** is allocated to the main base on the currently selected data, retaining existing cost guards and the operational reserve. No funds are earmarked for future gold-data training. The main-base objective is maximum useful affordable exposure with constant learning rates after warmup and **no in-run terminal decay**. End at the budget or allocation boundary with a full optimizer-state checkpoint. This supersedes the earlier requirement to select a final length that completes WSD decay; it does not change the trainer, authorize extra spending or guarantee an endpoint.
 
 
 ## Required separation for the revised continuation
@@ -34,14 +34,23 @@ PY=/workspace/venv/bin/python TORCHRUN=/workspace/venv/bin/torchrun \
 bash /workspace/kod/uruchom_109.sh
 ```
 
-Set `START`, `GPU`, `S`, `KROKI` and `CKPT_EVERY` explicitly; historical launcher defaults are not the operating plan. The command currently retains `KROKI=80000` because WSD length does not affect the constant-rate phase. At each 5,000-update boundary, the operator reviews remaining budget and the full cost of completing decay. Select the largest affordable total rounded down to a multiple of 5,000, changing it before `min(0.8 * previous_total, 0.8 * new_total)` is reached. Current price-dependent scenarios project 35,000–40,000 updates, not a selected runtime value. A 35,000-update endpoint starts decay at 28,000, so its latest permitted decision boundary is 25,000, not 30,000. A 40,000-update endpoint starts decay at 32,000 and can still be selected at 30,000, provided its complete cost fits the budget.
+Set `START`, `GPU`, `S`, `KROKI` and `CKPT_EVERY` explicitly; historical launcher defaults are not the operating plan. Retain **`KROKI=80000` unchanged**: it is a mechanical scheduling argument, not a funded endpoint. With the configured 20% WSD decay, decay starts at **update 64,000**, beyond the currently affordable main-base run. Review remaining budget, prices and allocation availability at the 5,000-update reviews and when conditions change, preserving cost guards and the operational reserve. Stop with full optimizer state while learning rates remain constant. Do not shorten `KROKI` to an affordable-update estimate: that would move decay earlier, contrary to the decision. If changing prices make the configured decay boundary reachable, operators must review the plan before it is reached rather than silently entering decay. Earlier 35,000–40,000 projections and the historical 40,000-update tracking plan are not selected endpoints; no new funded endpoint is asserted here.
 
-Current control values are microbatch 8 × accumulation 4 × eight ranks, context 4,096, Muon peak 0.02 / AdamW peak 0.0006, 2,000-update warmup and a final 20% decay to 0.00006. Full checkpoints now use an interval of 400 updates; each 5,000-update boundary also retains a weights-only checkpoint. The initial phase used 200. Target source allocation is 20% PL / 80% EN, not the cumulative language ratio of the full model history.
+Current control values are microbatch 8 × accumulation 4 × eight ranks, context 4,096, Muon peak 0.02 / AdamW peak 0.0006 and 2,000-update warmup. The configured final 20% decay to 0.00006 remains in the scheduler but is not to be entered under the main-base plan. Full checkpoints used an interval of 400 updates; from 11 November 2026, 13:00 UTC, each subsequent start sets 200 because of short spot allocation lifetimes. The allocation started at 12:48 UTC retains its previous 400-update interval until restart. Each 5,000-update boundary also retains a weights-only checkpoint. The initial phase used 200. Target source allocation is 20% PL / 80% EN, not the cumulative language ratio of the full model history.
 
 `--init-from` imports weights/config from 1.08 into this new experiment with a fresh optimizer. `--resume` restores this experiment's own state; it must not resume the abandoned random-initialized attempt. A verified new run and a verified recovery are separate receipts.
 
 At a projected 40,000 updates, processed new positions would be **41,943,040,000**. The reviewed inherited source/file-disjoint count is **18,863,257,682**. Deducting the conservative c01 duplicate correction of **114,627,128** gives a conditional total of **60,691,670,554**, not more than 100B. These calculations do not certify global content uniqueness, final exposure or the cumulative PL/EN split.
 
+## Separate future gold-data run — not funded or executed
+
+The 11 November, 12:07 planning decision separates a possible later pretraining/finishing run on approximately **10B tokens of quality-selected “gold” data at context 4,096**. The dataset team is to curate it in parallel, with licensed books selected with Arkadiusz Słota. It must remain isolated from main-base inputs and evaluation material, with its own manifest and licensing/admission gate, separate run identity and separate budget decision. A finishing continuation with decay is an alternative for that future run, not an in-run requirement of the main base.
+
+No such dataset has been assembled or released and no training budget has been approved for it. Quality-based selection is an intention, not a measured gain. The existing USD 425 campaign allocation has no reserve earmarked for this future option; its operational reserve and cost guards remain unchanged.
+
+## Inherited-count evidence
+
+
 Inherited-count footprint: counter SHA256 `abb39b8ce019c1a18a19ccf0ef4c32977c0c5a8f91496d4560125838789150f5`, using EpochSampler from trainer SHA `46f83b51…`, seed 20261109, 2,800,806,882 input positions, block 4,096, batch 256 and 1,709 updates. The sampled intersections with `[0, 1,925,806,163)` contribute 1,232,346,307 positions; other terms are 13,848 × 1,048,576 = 14,520,680,448 and a conservative 3,110,230,927 Polish positions from 1.07. Review repeated offset 2,959, 683,790 windows/epoch and 437,504 used windows. This is a source-position count, not a content-deduplicated historical union.
 
 ## Current CPT execution receipt