Text Generation
PEFT
Safetensors
lora
character-training
voice
style
synthetic-data
while-ai
conversational
Instructions to use while-ai/retail-concise-4b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use while-ai/retail-concise-4b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-4B-Instruct-2507") model = PeftModel.from_pretrained(base_model, "while-ai/retail-concise-4b") - Notebooks
- Google Colab
- Kaggle
Zero Proof AI commited on
State self-distillation and the absent random control
Browse files
README.md
CHANGED
|
@@ -123,6 +123,21 @@ that bias would move.
|
|
| 123 |
**RLHF Book ch. 4** for the SFT setup: prompt and system tokens carry no
|
| 124 |
loss, and the learning rate sits in the stated range.
|
| 125 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
## Training
|
| 127 |
|
| 128 |
LoRA rank 16, alpha 32, two epochs, learning rate 1e-4, bf16, 630 rows. Final
|
|
|
|
| 123 |
**RLHF Book ch. 4** for the SFT setup: prompt and system tokens carry no
|
| 124 |
loss, and the learning rate sits in the stated range.
|
| 125 |
|
| 126 |
+
**Self-distilled.** The teacher that wrote these replies is the same base
|
| 127 |
+
model, Qwen3-4B-Instruct-2507, with the constitution in its prompt. It is not
|
| 128 |
+
a stronger model. The constitution is doing the work, which is the point: the
|
| 129 |
+
register is available to the base when it is described, and training moves it
|
| 130 |
+
into the weights so it holds with nothing in the prompt. Stated because
|
| 131 |
+
`voice_rows` accepts a `model=` argument it never uses, so a reader would
|
| 132 |
+
otherwise reasonably assume a separate teacher (RLHF Book ch. 12: training on
|
| 133 |
+
a model's own output teaches it its own habits, which is acceptable here and
|
| 134 |
+
should not be assumed).
|
| 135 |
+
|
| 136 |
+
**No random-selection control** (RLHF Book ch. 9). Rows were kept when they
|
| 137 |
+
were complete, not when a judge scored them, so there is no reward selection
|
| 138 |
+
for a random arm to control against. Recorded so the absence is not read as
|
| 139 |
+
an oversight of the kind that applies to reward-selected sets.
|
| 140 |
+
|
| 141 |
## Training
|
| 142 |
|
| 143 |
LoRA rank 16, alpha 32, two epochs, learning rate 1e-4, bf16, 630 rows. Final
|