Zero Proof AI commited on
Commit
3281e5d
·
verified ·
1 Parent(s): 8dbed04

State self-distillation and the absent random control

Browse files
Files changed (1) hide show
  1. README.md +15 -0
README.md CHANGED
@@ -123,6 +123,21 @@ that bias would move.
123
  **RLHF Book ch. 4** for the SFT setup: prompt and system tokens carry no
124
  loss, and the learning rate sits in the stated range.
125
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
126
  ## Training
127
 
128
  LoRA rank 16, alpha 32, two epochs, learning rate 1e-4, bf16, 630 rows. Final
 
123
  **RLHF Book ch. 4** for the SFT setup: prompt and system tokens carry no
124
  loss, and the learning rate sits in the stated range.
125
 
126
+ **Self-distilled.** The teacher that wrote these replies is the same base
127
+ model, Qwen3-4B-Instruct-2507, with the constitution in its prompt. It is not
128
+ a stronger model. The constitution is doing the work, which is the point: the
129
+ register is available to the base when it is described, and training moves it
130
+ into the weights so it holds with nothing in the prompt. Stated because
131
+ `voice_rows` accepts a `model=` argument it never uses, so a reader would
132
+ otherwise reasonably assume a separate teacher (RLHF Book ch. 12: training on
133
+ a model's own output teaches it its own habits, which is acceptable here and
134
+ should not be assumed).
135
+
136
+ **No random-selection control** (RLHF Book ch. 9). Rows were kept when they
137
+ were complete, not when a judge scored them, so there is no reward selection
138
+ for a random arm to control against. Recorded so the absence is not read as
139
+ an oversight of the kind that applies to reward-selected sets.
140
+
141
  ## Training
142
 
143
  LoRA rank 16, alpha 32, two epochs, learning rate 1e-4, bf16, 630 rows. Final