Slayer1B 1.09: status training in progress

DATA_CARD.md

@@ -1,5 +1,5 @@
 ---
-pretty_name: Slayer1B 1.09 proposed training-data composition
+pretty_name: Slayer1B 1.09 staged training-data composition
 language:
 - pl
 - en
@@ -7,14 +7,14 @@ task_categories:
 - text-generation
 tags:
 - pretraining
-- planning-stage
+- training-in-progress
 ---
 
-# Slayer1B 1.09 — data plan
+# Slayer1B 1.09 — staged training data
 
 **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
 
-**Planning-stage documentation, 9 November 2026; decisions through 19:11.** The first rebalanced reference shard is prepared and byte-approved; the full 100B corpus is not prepared and no 1.09 training is claimed. Goal: a 1B model, 100B TRAIN positions, approximately 20–30% Polish, 72 hours and USD 400; these are not achieved results.
+**Training in progress, 9 November 2026; execution snapshot through 22:24 UTC.** Slot 1 started from scratch at 22:21:55 UTC on the byte-approved s01 v2 input. Three reference shards are approved, but the full 100B corpus is not prepared. The 1B model, 100B positions, approximately 20–30% Polish, 72 hours and USD 400 remain goals, not achieved results.
 
 ## Target, units and exposure
 
@@ -22,7 +22,15 @@ The full-corpus target is **100,000,000,000 native Slayer v6 TRAIN positions**,
 
 Provisional schedule: 95,000 updates × 1,048,576 positions = **99,614,720,000**, not 100B. Nineteen 5,000-update slots process **5,242,880,000 positions each** from input shards targeted at 5.3–5.4B. An owner-approved schedule must resolve the 100B mismatch.
 
-The selected plan is random-weight initialization with the existing tokenizer, not continuation from trained 1.07 weights; no launch is claimed. Rebalanced s01 v2 has a prepared reference of **5,316,374,064 positions**: **4,180,222,380 English-source** and **1,136,151,684 Polish-source** positions, approximately 21.37% PL. These describe input capacity, not the 5,242,880,000 positions processed by its planned slot. Exact consumed language shares are unavailable.
+The observed run uses random-weight initialization with the existing tokenizer, not trained 1.07 weights. Rebalanced s01 v2 was rebuilt on the training host with its approved digest. Its **5,316,374,064 positions** comprise **4,180,222,380 English-source** and **1,136,151,684 Polish-source** positions, approximately 21.37% PL. These are input capacity, not completed exposure: its 5,000-update slot targets **5,242,880,000 processed positions**. Exact consumed language shares are unavailable.
+
+| Byte-approved reference | Input positions | TRAIN SHA256 |
+|---|---:|---|
+| s01 v2; active slot-1 input | 5,316,374,064 | `2ab70797649947e0c14b8067161ed8a2541585428ca52c5ac8c9a744669bbb06` |
+| s02; host preparation pending | 5,302,913,370 | `b4df1406b88f131ac14a431108e33a68450fbd783a748a4978dc5a99a5886bb3` |
+| s03; host preparation pending | 5,284,760,292 | `b1091fbc7bcb1a677c829dd07b288f18c353c6b923d99dd0a85338deb5f727be` |
+
+Their combined **15,904,047,726 positions** are prepared reference capacity, not consumed tokens or a full 100B manifest. The active one-slot queue has SHA256 `6e06f0979f27c60e524852d89660bb2febef59a8b5f21f846e0618af1235715d`. Follow actual exposure on the [separate 1.09 Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b).
 
 Earlier models' material is reused as data, not weights. Used documents are recorded for exclusion from later shards, with exact token-sequence deduplication between inputs. This is not a guarantee of semantic uniqueness or absence of near-duplicates.
 
@@ -34,7 +42,7 @@ Linked public cards describe upstream material, not approval of every document:
 
 | Candidate | Published unit / selection condition |
 |---|---|
-| [GoLLeM PL](https://huggingface.co/datasets/SlayerLab/gollem-corpus-16b-pl), `17f9d37a` | Principal PL candidate: 16.58B published V32k tokens, including 14.62B HPLT web; not measured v6. Per-record `license`/`is_share_alike`; curated rights and web TDM checked separately |
+| [GoLLeM PL](https://huggingface.co/datasets/SlayerLab/gollem-corpus-16b-pl), `17f9d37a` | Principal PL candidate: 16.58B published V32k tokens, including 14.62B HPLT web; not retained v6 counts. Web `CC0-1.0` labels HPLT packaging, not text rights: web texts use separately reviewed TDM. Curated records require their own `license`/`is_share_alike` checks |
 | [FinePDFs-Edu](https://huggingface.co/datasets/HuggingFaceFW/finepdfs-edu), `9cfabe21`, PL/EN | PL pool approximately 8.3B v6 is an estimate, not retained size; ODC-By/Common Crawl terms; deduplicate against Logika |
 | [PL Logika](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b), `e24c8b71` | Published 1,911,918,958 native positions including EOS; earlier filtered reference approximately 1.63B, not a new 1.09 measurement; mixed terms/provenance |
 | [English Technical](https://huggingface.co/datasets/SlayerLab/english-technical-1b), `3b833d36`; [1.07 replay](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain), `6a337bbf` | Published technical TRAIN 899,001,323 native tokens before EOS; replay 8,891,005,707 EOS-inclusive positions; not s01 retained totals |

README.md

@@ -8,20 +8,20 @@ tags:
 - polish
 - causal-lm
 - pretraining
-- planning
+- training-in-progress
 ---
 
 # Slayer1B 1.09
 
 **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)**
 
-**Planning-stage card, 9 November 2026, reflecting decisions through 19:11 UTC.** No trained 1.09 checkpoint, actual initialization, measured throughput, evaluation scores or public run URL is established. This is not a released model or an inference-ready artifact.
+**Training in progress, 9 November 2026; execution snapshot through 22:24 UTC.** From-scratch training started at 22:21:55 UTC on eight B200 GPUs; preflight and the first 30 updates were observed. This is not a completed 100B run, an accepted successor or an inference-ready weight release. No 1.09 quality scores or checkpoint publication are claimed here.
 
 ## Objective and starting point
 
 The objective is a roughly 1B-parameter causal language model trained on **100 billion native v6 token positions, including document EOS 0, within 72 hours and USD 400 total**. The requested balance is approximate: **20–30% Polish**, with the remainder mainly English. The current selected recipe remains near **21.37% / 78.63%**; a later adjustment requires an explicit shard decision. These are targets, not completed exposure. Base-model training takes priority over instruction tuning.
 
-The selected plan is **random-weight, from-scratch initialization** with the existing v6 tokenizer, not continuation from trained 1.07 weights. Initialization has not yet been observed. Reusing a tokenizer does not mean inheriting trained model weights.
+The run uses **random-weight, from-scratch initialization** with the existing v6 tokenizer, not continuation from trained 1.07 weights. The operator's step-zero preflight and early training establish this starting state. Reusing a tokenizer does not mean inheriting trained model weights.
 
 The unchanged architecture is specified as **1,073,912,851 parameters**, 20 layers, width 2,048, 16 heads, SwiGLU, RMSNorm, RoPE base 500,000, QK normalization and value residuals. The reused v6 tokenizer has vocabulary 32,768 and pin `0640d3bd3674d7a6e59c540d945a2ad275a12e1bcaeb6007c37a90751f9815dd`.
 
@@ -29,9 +29,9 @@ Architecture and tokenizer lineage: [SlayerLab/Slayer1B-1.07](https://huggingfac
 
 The [training configuration](TRAINING_CONFIGURATION.md) identifies exact code digests and separates the saved, actually used 1.08 flags from this selected 1.09 recipe.
 
-## Proposed recipe, not launch evidence
+## Active recipe, not completed training
 
-| Setting | Proposal |
+| Setting | Selected run configuration |
 |---|---|
 | Trainer / optimizer | Existing r24.2; Muon with AdamW; fresh optimizer |
 | Batch / context | 256 × 4,096 = 1,048,576 positions per update |
@@ -42,7 +42,13 @@ The [training configuration](TRAINING_CONFIGURATION.md) identifies exact code di
 
 **95,000 × 1,048,576 = 99,614,720,000 positions (99.61472B), not 100B.** Each 5,000-update slot processes 5.24288B positions; planned input shards are larger, about 5.3–5.4B. The 385,280,000-position shortfall needs an approved step/batch policy; no correction is selected here.
 
-Queue mode uses append-only shards, manifest ranges and SHA-256 identities. Resume must preserve existing prefixes and earlier steps; new byte ranges must match tranche approval before use. Reported CPU smoke matched slotwise and continuous training bit-for-bit, including two-process DDP. The launcher received GREEN; this is not a production GPU run or approval of unverified new shards.
+Queue mode uses append-only shards, manifest ranges and SHA-256 identities. Resume must preserve existing prefixes and earlier steps; new byte ranges must match tranche approval before use. Reported CPU smoke matched slotwise and continuous training bit-for-bit, including two-process DDP. The reviewed launcher is now running slot 1 on GPUs; this does not approve unverified later shards or establish completion.
+
+## Early execution evidence
+
+The [public Track run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) is separate from 1.08. At update 30 it records training loss **10.3959**, interval throughput **576,593 positions/s** and **31,457,280 processed positions** (30 × 1,048,576). This is an early interval, not end-to-end billed throughput or a quality evaluation. The first slot targets 5,000 updates; its completion is not claimed.
+
+The resumed host's numerical equivalence check against 1.07 is still pending. Review allowed slot 1 to start on the matching hardware/software, but requires that check before slot 2; a difference above 0.001 bits/byte requires a stop and investigation. Do not describe that pending check as passed.
 
 ## Data and exposure
 
@@ -62,6 +68,6 @@ Comparison is descriptive: no replay-only control is budgeted, and initializatio
 
 ## Cost, licensing and limitations
 
-USD 400 authorizes the **total session**, including preparation, restarts, evaluation and interruptions, not just steady-state training. Funding is reported ready by the account owner; actual expense and 1.09 speed remain unmeasured. Meeting the deadline is unverified.
+USD 400 remains the **authorized total-session budget**, including preparation, restarts, evaluation and interruptions, not just steady-state training. Early interval speed is measured above; sustained end-to-end speed and final billed expense are not established. Budget and deadline compliance remain unverified; this card does not approve a larger budget or a shortened run as completed 100B training.
 
 1.07 declares Apache-2.0; this does not assign a license to future 1.09 weights or source texts. Release and publication rights require separate approval. Privacy filters are heuristic; generated text may be inaccurate, biased, offensive or personal. No reasoning quality, instruction-following ability or professional reliability has been demonstrated for 1.09.

TRAINING_CONFIGURATION.md

@@ -2,7 +2,7 @@
 
 Fabryka AI / SlayerLab
 
-**1.08 is a completed continued-pretraining experiment; 1.09 is the separate from-scratch 100B-target run.** A launch attempt is not proof of completed training. This record distinguishes saved 1.08 arguments from the selected 1.09 recipe.
+**1.08 is a completed continued-pretraining experiment; 1.09 is the separate from-scratch 100B-target run, now training.** The operator reports a 22:21:55 UTC start on 9 November 2026, step-zero preflight and the first 30 updates. The [public 1.09 run](https://track.fabryka.ai/run/08cf77cf-e995-5ee1-b843-05cbb96e4f4b) confirms early metrics, not completed training. This record distinguishes saved 1.08 arguments from the active 1.09 recipe.
 
 ## Code identities
 
@@ -10,7 +10,7 @@ Fabryka AI / SlayerLab
 |---|---|
 | Actual training implementation, both recipes | `train_gpt_ref_r24_2.py`, SHA256 `46f83b51fcec18ffc3273d80fb9eab29e182c52d8667800dab27b80f7ff18afc` |
 | First 1.09 launch attempt | `uruchom_109.sh`, SHA256 `dfee11522be49c938f169ca6992cf181f86a3c4e0d26fd5b5fa3646535dc890f` |
-| Selected restart launcher v3 | `uruchom_109.sh`, SHA256 `c0a2699d1a2d7ce4cddd873ad92ff5ab5bb0dbcecc6e47d6bce01ee9b848895c` |
+| Launcher used for observed 1.09 slot 1 | `uruchom_109.sh` v3, SHA256 `c0a2699d1a2d7ce4cddd873ad92ff5ab5bb0dbcecc6e47d6bce01ee9b848895c` |
 | Saved, actually used 1.08 argument file | SHA256 `d283bcb836e9bb89d5bf7dfa60487055086cf04620ad5c5dd36fe3435f2766a9` |
 
 The older public `train_gpt_ref_r23.py` is **not the identity of this training implementation**. The inference-compatible architecture code packaged with a weight release and the executable used to train it are distinct artifacts; filenames alone do not establish identity.
@@ -53,7 +53,7 @@ Both argument sets contain `--resume` for automatic recovery. That flag does **n
 
 Output directories, run IDs, data directories, the queue manifest and `--events-jsonl` are operational arguments, not omitted learning choices. For 1.08 they identify `slayer1b_108_r1`; 1.09 uses a separate output history. Do not share output directories between experiments.
 
-## Selected 1.09 operator invocation
+## Observed 1.09 slot-1 invocation
 
 On the allocated, verified node, with the pinned trainer and launcher already installed and an approved data directory:
 
@@ -74,6 +74,6 @@ Launcher v3 permits a total-step adjustment only before both the old and new dec
 
 ## What this does not claim
 
-The 1.08 argument file is direct execution evidence. The 1.09 launcher is a selected, reviewed recipe; runtime logs, checkpoints and final counters establish what actually happened. Steady interval throughput is not end-to-end billed throughput, especially when checkpoint and validation schedules differ.
+The 1.08 argument file is direct execution evidence. The 1.09 slot-1 launch, preflight and early public metrics establish partial execution of the reviewed v3 recipe; checkpoints and final counters must establish later progress and completion. Steady interval throughput is not end-to-end billed throughput, especially when checkpoint and validation schedules differ.
 
 Exact execution also requires the approved data bytes, manifest and filtering artifacts. The full 100B corpus is being assembled; public code inspection alone does not supply those data or protected evaluation material. No held-out questions, gold answers, generated evaluation outputs or private row-exclusion lists belong in this public bundle. See the [model card](README.md) and [data card](DATA_CARD.md) for status, source rights and limitations.