|
Download DATA_CARD.md from SlayerLab/Slayer1B-1.08: direct link, hf CLI and curl.
- Browser
- Download file 8.28 kB
-
https://huggingface.co/SlayerLab/Slayer1B-1.08/resolve/main/DATA_CARD.md
- Command line
-
hf download hf://SlayerLab/Slayer1B-1.08/DATA_CARD.md
-
curl -L -o DATA_CARD.md https://huggingface.co/SlayerLab/Slayer1B-1.08/resolve/main/DATA_CARD.md
8.28 kB
| pretty_name: Slayer1B 1.08 training-data composition, reference v2 | |
| language: | |
| - pl | |
| - en | |
| task_categories: | |
| - text-generation | |
| tags: | |
| - continued-pretraining | |
| - prepared-reference | |
| # Slayer1B 1.08 — training-data composition | |
| **Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)** | |
| **Prepared reference v2, 9 November 2026.** Target reconstruction must match the approved digests before training. This documents the new 1.08 composition, not a separately hosted mixed-corpus release; the earlier 1.08b experiment is distinct. | |
| ## Prepared training reference | |
| The composition combines analytical/reasoning-oriented text, English technical material and general-language replay. Retained counts come from the reference assembly manifest; they describe available input positions, not tokens consumed by a training run. Component names identify sources, not an audited per-document language distribution. | |
| | Retained component | Native token positions including EOS | | |
| |---|---:| | |
| | PL Logika | 1,631,862,622 | | |
| | English technical | 293,943,541 | | |
| | Replay `pl_web` | 435,000,183 | | |
| | Replay `en_fineweb_edu` | 220,000,324 | | |
| | Replay `en_mix` | 220,000,212 | | |
| | **TRAIN total** | **2,800,806,882** | | |
| Reference manifest SHA256: `c0f1e7771104f60b92ec38d3576fcfe34ed125a072da529d8b56fe2da0f87e7b`. TRAIN SHA256: `b73a69eaf399bb594300e7d1914a967a38b80b5bc35505fbc6459a6a27be496f`. | |
| | Separate partition | Documents | Token positions including EOS | | |
| |---|---:|---:| | |
| | Polish-logic held-out | 3,158 | 18,294,311 | | |
| | Trainer validation, a subset of that held-out | 360 | 2,002,010 | | |
| | Eligible English VALID held-out | 5,218 | 17,988,388 | | |
| These are partition totals, not domain-evaluation sample sizes. | |
| ## Published inputs before filtering | |
| The following source-published counts use their stated split and EOS units. | |
| | Source / revision | Split | Documents | Native Slayer tokens | | |
| |---|---|---:|---:| | |
| | [PL Logika](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b/tree/e24c8b71526ca1dc56a280100f88c2f09a79ecdd), `e24c8b71526ca1dc56a280100f88c2f09a79ecdd` | published train | 327,219 | 1,911,918,958, including EOS | | |
| | [English Technical](https://huggingface.co/datasets/SlayerLab/english-technical-1b/tree/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3), `3b833d36bd3ba9aeecf46f39605c9ac849bb96a3` | train | 188,904 | 899,001,323, before pack-added EOS | | |
| | Same English release | valid | 10,370 | 49,886,904, before pack-added EOS | | |
| | Same English release | test | 10,665 | 51,226,632, before pack-added EOS | | |
| The English all-split total, 1,000,114,859 tokens, is not training size. Its pack manifest adds one EOS per document: source TRAIN becomes **899,190,227 token positions**. Proxy cl100k counts are a different unit. Source counts, retained reference positions and observed training exposure are distinct quantities. | |
| Replay comes from [Slayer 1.07 pretraining data](https://huggingface.co/datasets/SlayerLab/slayer-1.07-pretrain/tree/6a337bbfba518d88f14eb54f7d0a9b9f297df0d1), revision `6a337bbfba518d88f14eb54f7d0a9b9f297df0d1`: `pl_web`, `en_fineweb_edu` and `en_mix`. These texts were seen by 1.07; retained quotas are shown above. | |
| ## Composition and rights | |
| Licenses differ by source; compilation terms do not grant rights to every underlying text. Reference selection excludes GPL/GFDL sources and mixed GPL/LGPL English sources. Retained components still carry their individual terms, not a blanket permissive-license grant. | |
| ### Polish corpus | |
| Required conditions remain source-specific: | |
| - **FinePDFs:** retain the notice “Contains information from FinePDFs, which is made available under the ODC Attribution License,” applicable Common Crawl terms and annotation attribution. PDF-content rights remain unresolved; upstream relies on a TDM rationale and did not recheck domain-level opt-outs. | |
| - **Dolci:** preserve original Dolci, translation and component notices, including NVIDIA Nemotron CC BY 4.0. The translation license does not replace underlying obligations. Attribution identifies Gemma translation and mixed upstream reasoning teachers; embedded conversation-history provenance remains uncertain. | |
| - **DOAB:** preserve each book's creator/editor, title, publisher, license, URL and modification notices. Selected material includes English books, so the corpus label is not a per-document language guarantee. | |
| - **Tax material:** preserve Romuald Członkowski and Eureka MF/KIS attribution. Upstream did not locate the portal terms; absence of known restrictions is not confirmed permission. | |
| The pinned [attribution inventory](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b/blob/e24c8b71526ca1dc56a280100f88c2f09a79ecdd/ATRYBUCJA.md) carries the full notices; this summary does not replace them. | |
| ### English technical corpus and replay | |
| English sources span systems, machine learning, compilers, CNC/CAM, simulation and graphics. Row licenses and matching notices govern selection: excluded GPL sources include Linux, Blender, OpenFOAM and some CNC material; retained entries use MIT, Apache-2.0, Apache-2.0 WITH LLVM-exception, BSD-3-Clause, dual Apache/MIT or the FreeCAD repository's CC0 terms. The [source-license inventory](https://huggingface.co/datasets/SlayerLab/english-technical-1b/blob/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3/licenses/SOURCES.md) records the notices. The retained technical count above reflects all filters, not just license exclusion. | |
| FreeCAD documentation: **FreeCAD wiki contributors**, via `FreeCAD/FreeCAD-documentation` at `0499378a`, repository license **CC0-1.0**. This identifies the conversion repository's license; the underlying wiki attribution is retained. | |
| Replay includes ODC-By FineWeb-Edu/FineMath/open-web-math, CC BY DCLM and HPLT. HPLT's compilation is CC0, not its underlying web texts; the replay publisher relies on Directive (EU) 2019/790 Article 4. Redistribution requires separate rights review. Required notices follow the pinned replay card. | |
| ## Preparation and split isolation | |
| Preparation uses the same Slayer v6 tokenizer and little-endian uint16 token format, with EOS 0 after each document. | |
| Reference TRAIN excludes whole documents flagged by the contact/identifier filter or containing literal special-token IDs in their text. Assembly metadata records removal counts and overlapping filters. | |
| English valid/test remain outside TRAIN; eligible VALID uses the TRAIN license exclusions and removes documents also present in TRAIN. Polish held-out sections are grouped by source-book identity rather than splitting chapters across train and evaluation. | |
| Target reconstruction uses the pinned preparation recipe, exclusion lists and the same input revisions. TRAIN, validation and held-out digests must match the approved reference artifacts before use. | |
| ## Quality, privacy and limitations | |
| The reference uses recorded per-row identifier/overlap exclusions for new sources and inherits the replay release's previous checks. Legacy replay did not detect near-duplicates, omitted EWoK from its scan and retained some short overlaps. These limitations and exposure inherited through the base model remain relevant to evaluation. | |
| Whole-document removal reduces FinePDFs coverage, including academic text with author contact details. Residual names, addresses, translation artifacts, PDF-layout damage and tax-text fragmentation can remain. Polish metadata reports no toxicity classifier and masking exceptions in programming/math contexts. Technical text can contain unsafe instructions; reasoning traces can be wrong. Domain labels and source counts establish neither model capability nor safe professional advice. | |
| ## Source metadata | |
| Further pinned evidence: Polish [aggregate manifest](https://huggingface.co/datasets/SlayerLab/slayer-pl-logika-2b/blob/e24c8b71526ca1dc56a280100f88c2f09a79ecdd/manifest.json); English [aggregate manifest](https://huggingface.co/datasets/SlayerLab/english-technical-1b/blob/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3/manifest.json) and [EOS-inclusive pack manifest](https://huggingface.co/datasets/SlayerLab/english-technical-1b/blob/3b833d36bd3ba9aeecf46f39605c9ac849bb96a3/slayer_v6/manifest.json). This documentation was prepared from public cards and aggregate/license metadata, not raw records or token binaries. | |