faxenoff
/

code-daemon-denoise-v1

@@ -157,20 +157,35 @@ ids — do not remap them yourself.
 ## 6. Evaluation
-On a frozen held-out bilingual word set, at `strip_threshold = 0.95`:
 | metric | value |
 |---|--:|
-| SAFE (keep) F1 | **0.79** |
-| BALLAST (drop) F1 | **0.84** |
-| Strip precision | **0.88** |
-The INT8 vocab-pruned build scores the same as the full-vocabulary FP build (F1 0.79 vs 0.79) at
-38% of the size — the pruning removes rows the two target languages never reach, so there is nothing
-to lose by it.
 Strip precision is the number to watch if you tune the threshold: it says how often a word the model
-drops really was ballast.
 ---

 ## 6. Evaluation
+Re-measured 2026-09-07 on the shipped INT8 engine against the frozen held-out set
+(224 words, 95 keep / 129 drop), at `strip_threshold = 0.95`:
 | metric | value |
 |---|--:|
+| SAFE (keep) F1 | **0.70** |
+| BALLAST (drop) F1 | **0.82** |
+| Strip precision | **0.89** |
+| Strip recall | **0.77** |
+| Technical words wrongly dropped | **12 / 95** |
 Strip precision is the number to watch if you tune the threshold: it says how often a word the model
+drops really was ballast. The count of technical words wrongly dropped is the one to watch if you
+care about the expensive direction.
+**Head calibration (2026-09-07).** The head shipped before this date was fitted against an earlier
+build of the encoder, and its scores saturated: 174 of the 224 held-out words landed at P < 0.01 or
+P > 0.99, so `strip_threshold` decided only twelve of them and every useful operating point sat above
+t = 0.99 — out of reach of the documented knob. The head has been **temperature-scaled** (w and b
+divided by T = 2.13). That is a monotone transform, so the model's ranking, its ROC and all SAFE
+metrics are unchanged by construction; only the probability scale moves, which puts the threshold
+back in charge. At the default 0.95 this halves the technical words wrongly dropped (24 → 12) for
+15 fewer ballast words removed (114 → 99).
+**On the numbers previously published here (SAFE F1 0.79 / strip precision 0.88).** They are not
+reproducible from the artifacts in this repository, and re-measuring is what surfaced the saturation
+above. Two independent implementations — the daemon's own harness and a standalone script — agree on
+the table above to the row. The earlier figures appear to belong to an encoder build that predates
+the vocabulary prune and is no longer published; they are corrected rather than defended.
 ---