Maggio33 commited on
Commit
71f230a
·
verified ·
1 Parent(s): 2c74499

Card coherence: past tense for the completed public-data SFT attempt; no weights released

Browse files
Files changed (1) hide show
  1. README.md +14 -15
README.md CHANGED
@@ -11,7 +11,6 @@ tags:
11
  - instruct
12
  - sft
13
  - experimental
14
- - in-preparation
15
  ---
16
 
17
  # Slayer1B 1.09 Instruct — experimental
@@ -22,39 +21,39 @@ tags:
22
 
23
  **Status update: 14 November 2026.** A first supervised fine-tuning attempt (public data only) was trained and did not pass our blind evaluation against the base model; no instruct weights are released.
24
 
25
- The intended task is assistant conversation and instruction following in Polish and English. The intended use is research; this snapshot does not establish conversational quality, a quality improvement over the base, or suitability for production.
26
 
27
  ## Starting checkpoint
28
 
29
- The selected starting point is **Slayer1B 1.09, update 35,000**, from the public [checkpoint archive at its pinned revision](https://huggingface.co/SlayerLab/Slayer1B-1.09-checkpoints/tree/c9609f273a69a317c62ad3ad033c1d88a0f97912), file `wagi_35000.pt`. It is a base checkpoint, not an existing instruct release.
30
 
31
  The [base model card](https://huggingface.co/SlayerLab/Slayer1B-1.09/blob/6484a05ff42bc8a00a5147aaabb745198680feaa/README.md) describes the inherited architecture and continued pretraining. The archive contains native training checkpoints; it must not be mistaken for a Transformers-compatible chat release.
32
 
33
  ## Public-data-only SFT plan
34
 
35
- The current supervised fine-tuning plan draws from the Polish and English portions of these public sources:
36
 
37
- - **Aya Dataset**, by the Aya Open Science contributors / Cohere Labs. [Pinned source revision](https://huggingface.co/datasets/CohereLabs/aya_dataset/tree/f9ea04583f02a8f86404ff6c58bf75fe637df8a2). Upstream license: **Apache-2.0**. The current plan selects Polish instruction–answer pairs.
38
- - **OpenAssistant Conversations Release 2 (OASST2)**, by the OpenAssistant community. [Pinned source revision](https://huggingface.co/datasets/OpenAssistant/oasst2/tree/179dd21fc55192153d94adb0e0ce8f69e222bf75). Upstream license: **Apache-2.0**. The current plan selects Polish and English conversations. OASST2 contains message trees, so a message count is not a conversation count.
39
 
40
- Listing a source does not mean that the whole dataset was used. The prepared selection is subject to provenance, privacy and evaluation-overlap checks before training. Candidate counts before those checks are not presented as final training counts or token exposure.
41
 
42
- The original sources remain available at the links above. The locally prepared training and validation selections are **not redistributed in this repository**. This plan does not add newly generated assistant responses.
43
 
44
  ## Training and evaluation boundaries
45
 
46
- The planned first stage is **supervised fine-tuning (SFT)**. There is no claimed DPO or weight-averaging result in this snapshot. A later preference-training experiment would need its own data decision, objective and evaluation.
47
 
48
- The planned training and chat interfaces must use the same conversation format. The training loss mask and the end of an assistant's turn must be checked before a chat artifact is released.
49
 
50
- The evaluation plan separates checkpoint selection from the final held-out instruction test. The primary comparison is blind pairwise assessment against the **same untuned starting checkpoint with a fixed conversation example**. Standard base-model benchmark scores are separate descriptive measurements, not results for this instruction-tuning experiment.
51
 
52
- **No instruction-following scores or generated-response comparisons are published for this experiment in this snapshot.** A lower training loss alone would not demonstrate better instruction following.
53
 
54
  ## Availability and limitations
55
 
56
- There is no released post-trained inference artifact in this preparation snapshot, so this card does not provide a runnable chat example or claim that loading the base archive yields an instruct model.
57
 
58
- The base checkpoint declares Apache-2.0, and the two candidate SFT sources above declare Apache-2.0. Those upstream declarations do not replace review of the selected records, attribution requirements or the eventual post-trained release. This card does not redistribute source conversations or grant additional rights in them.
59
 
60
- The planned model may produce incorrect, biased, unsafe or personal text. No professional reliability, privacy guarantee, or improvement in reasoning or multi-turn conversation is claimed before evaluation.
 
11
  - instruct
12
  - sft
13
  - experimental
 
14
  ---
15
 
16
  # Slayer1B 1.09 Instruct — experimental
 
21
 
22
  **Status update: 14 November 2026.** A first supervised fine-tuning attempt (public data only) was trained and did not pass our blind evaluation against the base model; no instruct weights are released.
23
 
24
+ The intended task remains assistant conversation and instruction following in Polish and English, for research. This repository documents preparation and the completed public-data SFT attempt; it is not an instruction-tuned inference release and does not establish a quality improvement over the base or suitability for production.
25
 
26
  ## Starting checkpoint
27
 
28
+ That attempt started from **Slayer1B 1.09, update 35,000**, from the public [checkpoint archive at its pinned revision](https://huggingface.co/SlayerLab/Slayer1B-1.09-checkpoints/tree/c9609f273a69a317c62ad3ad033c1d88a0f97912), file `wagi_35000.pt`. This starting point is a base checkpoint, not an existing instruct release.
29
 
30
  The [base model card](https://huggingface.co/SlayerLab/Slayer1B-1.09/blob/6484a05ff42bc8a00a5147aaabb745198680feaa/README.md) describes the inherited architecture and continued pretraining. The archive contains native training checkpoints; it must not be mistaken for a Transformers-compatible chat release.
31
 
32
  ## Public-data-only SFT plan
33
 
34
+ The documented supervised fine-tuning preparation plan selected Polish and English material from these public sources. The list records the preparation plan, not the exact set of records used:
35
 
36
+ - **Aya Dataset**, by the Aya Open Science contributors / Cohere Labs. [Pinned source revision](https://huggingface.co/datasets/CohereLabs/aya_dataset/tree/f9ea04583f02a8f86404ff6c58bf75fe637df8a2). Upstream license: **Apache-2.0**. The preparation plan selected Polish instruction–answer pairs.
37
+ - **OpenAssistant Conversations Release 2 (OASST2)**, by the OpenAssistant community. [Pinned source revision](https://huggingface.co/datasets/OpenAssistant/oasst2/tree/179dd21fc55192153d94adb0e0ce8f69e222bf75). Upstream license: **Apache-2.0**. The preparation plan selected Polish and English conversations. OASST2 contains message trees, so a message count is not a conversation count.
38
 
39
+ Listing a source does not mean that the whole dataset was used. The preparation plan required provenance, privacy and evaluation-overlap checks before training. Candidate counts before those checks are not presented as final training counts or token exposure.
40
 
41
+ The original sources remain available at the links above. The locally prepared training and validation selections are **not redistributed in this repository**. The documented preparation plan did not introduce newly generated assistant responses.
42
 
43
  ## Training and evaluation boundaries
44
 
45
+ The completed public-data attempt used **supervised fine-tuning (SFT)**. No DPO or weight-averaging result is claimed. A future preference-training experiment would first require an accepted SFT checkpoint, as well as its own data decision, objective and evaluation.
46
 
47
+ Any future chat artifact must use the same conversation format in training and chat. Its training loss mask and the end of an assistant's turn must be checked before release.
48
 
49
+ The evaluation protocol separated checkpoint selection from the final held-out instruction test. Its primary comparison was blind pairwise assessment against the **same untuned starting checkpoint with a fixed conversation example**. Standard base-model benchmark scores remain separate descriptive measurements, not results for this instruction-tuning experiment.
50
 
51
+ **The release decision is summarised in Status; no numerical instruction-following scores, per-item responses or detailed pairwise results are published in this repository.** A lower training loss alone would not demonstrate better instruction following.
52
 
53
  ## Availability and limitations
54
 
55
+ No post-trained inference artifact is released. This documentation therefore does not provide a runnable chat example; loading the base archive does not make it an instruct model.
56
 
57
+ The base checkpoint declares Apache-2.0, and the two listed SFT sources above declare Apache-2.0. Those upstream declarations do not replace review of the selected records, attribution requirements or any future post-trained release. This card does not redistribute source conversations or grant additional rights in them.
58
 
59
+ Instruction-tuned models may produce incorrect, biased, unsafe or personal text. No professional reliability, privacy guarantee, or improvement in reasoning or multi-turn conversation is claimed.