Zero Proof AI commited on
Re-measured at a non-binding cap with finish_reason recorded
Browse files
README.md
CHANGED
|
@@ -15,7 +15,7 @@ tags:
|
|
| 15 |
# zeroproof-retail-concise-4b
|
| 16 |
|
| 17 |
**A Qwen3-4B retail agent that answers first and stops, with nothing in the
|
| 18 |
-
prompt telling it to.**
|
| 19 |
11.8% for the base, with no increase in omitted information.
|
| 20 |
|
| 21 |
The companion to
|
|
@@ -30,17 +30,30 @@ process serving base weights and adapter. Every row graded in both arms.
|
|
| 30 |
|
| 31 |
| | Base | This adapter |
|
| 32 |
|---|---|---|
|
| 33 |
-
| In the register | 0.118 | 0.
|
| 34 |
| Omitted required information | 0.015 | 0.015 |
|
| 35 |
| Median reply length | 442 chars | 201 chars |
|
| 36 |
|
| 37 |
-
Delta +0.
|
| 38 |
-
|
| 39 |
-
p =
|
| 40 |
|
| 41 |
Register judged by Phi-4, a different model family from the policy. Omission
|
| 42 |
decided in code against the identifiers each question requires.
|
| 43 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
### Answer-production rate
|
| 45 |
|
| 46 |
The register rate is conditional on the model having produced an answer at
|
|
|
|
| 15 |
# zeroproof-retail-concise-4b
|
| 16 |
|
| 17 |
**A Qwen3-4B retail agent that answers first and stops, with nothing in the
|
| 18 |
+
prompt telling it to.** 83.8% of held-out replies are in the register against
|
| 19 |
11.8% for the base, with no increase in omitted information.
|
| 20 |
|
| 21 |
The companion to
|
|
|
|
| 30 |
|
| 31 |
| | Base | This adapter |
|
| 32 |
|---|---|---|
|
| 33 |
+
| In the register | 0.118 | 0.838 |
|
| 34 |
| Omitted required information | 0.015 | 0.015 |
|
| 35 |
| Median reply length | 442 chars | 201 chars |
|
| 36 |
|
| 37 |
+
Delta +0.721, paired bootstrap over prompts, 95% CI **[+0.647, +0.794]**.
|
| 38 |
+
98 prompts improved, 0 regressed, 36 unchanged. Sign test one-sided
|
| 39 |
+
p = 3.2e-30. At 136 prompts the eval resolves +0.074 or larger.
|
| 40 |
|
| 41 |
Register judged by Phi-4, a different model family from the policy. Omission
|
| 42 |
decided in code against the identifiers each question requires.
|
| 43 |
|
| 44 |
+
**Truncation, measured not inferred.** `finish_reason` is recorded per row.
|
| 45 |
+
On this evaluation every reply in both arms finished on its own: 0.0% not-EOS
|
| 46 |
+
base, 0.0% not-EOS trained, an inter-arm gap of 0.0 points, with the token cap
|
| 47 |
+
at 2048 where neither arm reaches it. Base replies run to a median of 437
|
| 48 |
+
characters and a 95th percentile of 2083, so the cap is not shaping the
|
| 49 |
+
quantity being judged.
|
| 50 |
+
|
| 51 |
+
The cap is not a neutral parameter when the trait is concision: too low it
|
| 52 |
+
clips the base toward brevity and understates the gain, too high it lets the
|
| 53 |
+
base ramble and inflates it. Re-measuring at 2048 against an earlier 700 moved
|
| 54 |
+
the airline delta not at all (+0.899 both) and retail by -0.014, so the cap was
|
| 55 |
+
doing no work in either direction.
|
| 56 |
+
|
| 57 |
### Answer-production rate
|
| 58 |
|
| 59 |
The register rate is conditional on the model having produced an answer at
|