Card coherence: past tense for the completed public-data SFT attempt; no weights released
Browse files
README.md
CHANGED
|
@@ -11,7 +11,6 @@ tags:
|
|
| 11 |
- instruct
|
| 12 |
- sft
|
| 13 |
- experimental
|
| 14 |
-
- in-preparation
|
| 15 |
---
|
| 16 |
|
| 17 |
# Slayer1B 1.09 Instruct — experimental
|
|
@@ -22,39 +21,39 @@ tags:
|
|
| 22 |
|
| 23 |
**Status update: 14 November 2026.** A first supervised fine-tuning attempt (public data only) was trained and did not pass our blind evaluation against the base model; no instruct weights are released.
|
| 24 |
|
| 25 |
-
The intended task
|
| 26 |
|
| 27 |
## Starting checkpoint
|
| 28 |
|
| 29 |
-
|
| 30 |
|
| 31 |
The [base model card](https://huggingface.co/SlayerLab/Slayer1B-1.09/blob/6484a05ff42bc8a00a5147aaabb745198680feaa/README.md) describes the inherited architecture and continued pretraining. The archive contains native training checkpoints; it must not be mistaken for a Transformers-compatible chat release.
|
| 32 |
|
| 33 |
## Public-data-only SFT plan
|
| 34 |
|
| 35 |
-
The
|
| 36 |
|
| 37 |
-
- **Aya Dataset**, by the Aya Open Science contributors / Cohere Labs. [Pinned source revision](https://huggingface.co/datasets/CohereLabs/aya_dataset/tree/f9ea04583f02a8f86404ff6c58bf75fe637df8a2). Upstream license: **Apache-2.0**. The
|
| 38 |
-
- **OpenAssistant Conversations Release 2 (OASST2)**, by the OpenAssistant community. [Pinned source revision](https://huggingface.co/datasets/OpenAssistant/oasst2/tree/179dd21fc55192153d94adb0e0ce8f69e222bf75). Upstream license: **Apache-2.0**. The
|
| 39 |
|
| 40 |
-
Listing a source does not mean that the whole dataset was used. The
|
| 41 |
|
| 42 |
-
The original sources remain available at the links above. The locally prepared training and validation selections are **not redistributed in this repository**.
|
| 43 |
|
| 44 |
## Training and evaluation boundaries
|
| 45 |
|
| 46 |
-
The
|
| 47 |
|
| 48 |
-
|
| 49 |
|
| 50 |
-
The evaluation
|
| 51 |
|
| 52 |
-
**
|
| 53 |
|
| 54 |
## Availability and limitations
|
| 55 |
|
| 56 |
-
|
| 57 |
|
| 58 |
-
The base checkpoint declares Apache-2.0, and the two
|
| 59 |
|
| 60 |
-
|
|
|
|
| 11 |
- instruct
|
| 12 |
- sft
|
| 13 |
- experimental
|
|
|
|
| 14 |
---
|
| 15 |
|
| 16 |
# Slayer1B 1.09 Instruct — experimental
|
|
|
|
| 21 |
|
| 22 |
**Status update: 14 November 2026.** A first supervised fine-tuning attempt (public data only) was trained and did not pass our blind evaluation against the base model; no instruct weights are released.
|
| 23 |
|
| 24 |
+
The intended task remains assistant conversation and instruction following in Polish and English, for research. This repository documents preparation and the completed public-data SFT attempt; it is not an instruction-tuned inference release and does not establish a quality improvement over the base or suitability for production.
|
| 25 |
|
| 26 |
## Starting checkpoint
|
| 27 |
|
| 28 |
+
That attempt started from **Slayer1B 1.09, update 35,000**, from the public [checkpoint archive at its pinned revision](https://huggingface.co/SlayerLab/Slayer1B-1.09-checkpoints/tree/c9609f273a69a317c62ad3ad033c1d88a0f97912), file `wagi_35000.pt`. This starting point is a base checkpoint, not an existing instruct release.
|
| 29 |
|
| 30 |
The [base model card](https://huggingface.co/SlayerLab/Slayer1B-1.09/blob/6484a05ff42bc8a00a5147aaabb745198680feaa/README.md) describes the inherited architecture and continued pretraining. The archive contains native training checkpoints; it must not be mistaken for a Transformers-compatible chat release.
|
| 31 |
|
| 32 |
## Public-data-only SFT plan
|
| 33 |
|
| 34 |
+
The documented supervised fine-tuning preparation plan selected Polish and English material from these public sources. The list records the preparation plan, not the exact set of records used:
|
| 35 |
|
| 36 |
+
- **Aya Dataset**, by the Aya Open Science contributors / Cohere Labs. [Pinned source revision](https://huggingface.co/datasets/CohereLabs/aya_dataset/tree/f9ea04583f02a8f86404ff6c58bf75fe637df8a2). Upstream license: **Apache-2.0**. The preparation plan selected Polish instruction–answer pairs.
|
| 37 |
+
- **OpenAssistant Conversations Release 2 (OASST2)**, by the OpenAssistant community. [Pinned source revision](https://huggingface.co/datasets/OpenAssistant/oasst2/tree/179dd21fc55192153d94adb0e0ce8f69e222bf75). Upstream license: **Apache-2.0**. The preparation plan selected Polish and English conversations. OASST2 contains message trees, so a message count is not a conversation count.
|
| 38 |
|
| 39 |
+
Listing a source does not mean that the whole dataset was used. The preparation plan required provenance, privacy and evaluation-overlap checks before training. Candidate counts before those checks are not presented as final training counts or token exposure.
|
| 40 |
|
| 41 |
+
The original sources remain available at the links above. The locally prepared training and validation selections are **not redistributed in this repository**. The documented preparation plan did not introduce newly generated assistant responses.
|
| 42 |
|
| 43 |
## Training and evaluation boundaries
|
| 44 |
|
| 45 |
+
The completed public-data attempt used **supervised fine-tuning (SFT)**. No DPO or weight-averaging result is claimed. A future preference-training experiment would first require an accepted SFT checkpoint, as well as its own data decision, objective and evaluation.
|
| 46 |
|
| 47 |
+
Any future chat artifact must use the same conversation format in training and chat. Its training loss mask and the end of an assistant's turn must be checked before release.
|
| 48 |
|
| 49 |
+
The evaluation protocol separated checkpoint selection from the final held-out instruction test. Its primary comparison was blind pairwise assessment against the **same untuned starting checkpoint with a fixed conversation example**. Standard base-model benchmark scores remain separate descriptive measurements, not results for this instruction-tuning experiment.
|
| 50 |
|
| 51 |
+
**The release decision is summarised in Status; no numerical instruction-following scores, per-item responses or detailed pairwise results are published in this repository.** A lower training loss alone would not demonstrate better instruction following.
|
| 52 |
|
| 53 |
## Availability and limitations
|
| 54 |
|
| 55 |
+
No post-trained inference artifact is released. This documentation therefore does not provide a runnable chat example; loading the base archive does not make it an instruct model.
|
| 56 |
|
| 57 |
+
The base checkpoint declares Apache-2.0, and the two listed SFT sources above declare Apache-2.0. Those upstream declarations do not replace review of the selected records, attribution requirements or any future post-trained release. This card does not redistribute source conversations or grant additional rights in them.
|
| 58 |
|
| 59 |
+
Instruction-tuned models may produce incorrect, biased, unsafe or personal text. No professional reliability, privacy guarantee, or improvement in reasoning or multi-turn conversation is claimed.
|