Publish encoder-v4

README.md

@@ -49,7 +49,7 @@ Measured with `rukh eval encoder` on 2026-09-21, over
 
 ### Acceptance bars
 
-The project set these two bars for the encoder in `GOAL.md` before any of this was trained. This
+The project set these two bars for the encoder in `docs/acceptance.md` before any of this was trained. This
 is where the release stands against them, and it is the same table whether the answer is
 flattering or not.
 
@@ -99,10 +99,10 @@ How to read these numbers:
 
 - the blunder F1 of the encoder and of the material baseline are measured on the same rows of the held-out 'val' split, which is drawn by game_id, never by position, and those rows are cut in two by game_id again: the 'tune' half chooses the threshold and the 'score' half is what gets reported
 - the checkpoint carries no label-count curve: run `rukh train heads --curve` and evaluate one of its checkpoints to fill that table
-- the encoder is +9.7 F1 points from the baseline; GOAL.md asks for at least +5
-- the value head is correlated against `tanh(cp / 400)`, the bounded score it is trained on, and not against raw `cp`, where a forced mate is worth ±9 99x and a few rows would decide Pearson for the whole set; the GOAL.md bar of 0.80 is read on Spearman
+- the encoder is +9.7 F1 points from the baseline; docs/acceptance.md asks for at least +5
+- the value head is correlated against `tanh(cp / 400)`, the bounded score it is trained on, and not against raw `cp`, where a forced mate is worth ±9 99x and a few rows would decide Pearson for the whole set; the docs/acceptance.md bar of 0.80 is read on Spearman
 - the baseline counts material (1/3/3/5/9) and mobility and looks one ply ahead at captures: it cannot see a positional sacrifice, and calls Fischer's 17...Be6 (Byrne-Fischer, 1956) a nine-point blunder
-- the two blunder detectors do not see the same thing: the baseline is given the predecessor position and the move that was played, while the encoder is given only the resulting position and has to infer that something was thrown away. That is the comparison GOAL.md asks for, but it is not a level playing field
+- the two blunder detectors do not see the same thing: the baseline is given the predecessor position and the move that was played, while the encoder is given only the resulting position and has to infer that something was thrown away. That is the comparison docs/acceptance.md asks for, but it is not a level playing field
 - the blunder threshold 0.09617 was chosen on the 'tune' half (3,793 rows, 2,455 games) by maximising F1 there, and the reported numbers are measured on the 'score' half (3,660 rows, 2,368 games), which no threshold ever saw; the same rows at the fixed threshold 0.5 give an F1 of 0.0000 against 0.1855 tuned
 - the material baseline is a hard yes/no rule: it has no threshold, so nothing was tuned on its side and it got no half to tune on. The margin therefore compares a model at its best operating point against a rule at its only one
 - a blunder is rare (3.7 % of the labelled rows), so **accuracy is meaningless** here: a model that always answers "no blunder" scores 96.3 % without knowing anything about chess. F1 at an arbitrary threshold is nearly as bad, because an uncalibrated sigmoid can rank the positions well and still put every probability below 0.5: that number measures the operating point, not the representation. This is why the threshold is chosen on a `tune` half and the F1 is reported on a `score` half, and why ROC AUC and average precision, which no threshold can flatter, are reported next to it
@@ -140,26 +140,31 @@ similar positions (`rukh encoder embed --positions <parquet> --out <npy>`).
 - `onnx/parity.json`
 
 The ONNX graph returns **two** outputs, `value` and `blunder`, because that is all the demo's
-evaluation bar and blunder alert need; `blunder` comes out as a probability, so the page
-compares it against 0.5. `model-fp16.onnx` is for WebGPU and `model-int8.onnx` for the WASM
-fallback. The metadata carries `rukh_kind=encoder` and `rukh_heads`.
+evaluation bar and blunder alert need; `blunder` comes out as a probability. That probability is
+**not calibrated**: a blunder is a few per cent of the labelled rows, so the head answers low and
+never reaches 0.5. Its operating point was tuned on a held-out half and travels in the metadata as
+`rukh_blunder_threshold`, which is what a reader should compare against -- 0.5 is a threshold this
+head cannot cross, and a page using it shows an alert that never fires. `model-fp16.onnx` is for
+WebGPU and `model-int8.onnx` for the WASM fallback. The metadata also carries `rukh_kind=encoder`
+and `rukh_heads`.
 
 ### How faithful the ONNX files are
 
 Every exported file was run against the PyTorch checkpoint on 1000
-held-out labelled positions, comparing the blunder decision at p >= 0.5. `onnx/parity.json` in this
+held-out labelled positions, comparing the blunder decision at p >= 0.0961714. `onnx/parity.json` in this
 repository is that measurement, as the exporter wrote it.
 
 | File | Same blunder decision as PyTorch | Worst `value` drift |
 |---|---|---|
 | `model.onnx` (fp32) | 100.0 % | 4.44e-06 |
 | `model-fp16.onnx` (fp16) | 100.0 % | 0.00164 |
-| `model-int8.onnx` (int8) | 100.0 % | 0.117 |
+| `model-int8.onnx` (int8) | 98.7 % | 0.117 |
 
 The bar the project set itself is 99.9 %.
-Every precision, int8 included, makes the same call on every position checked: quantizing this
-model costs nothing that the demo's blunder alert can see. That is worth stating next to the
-decoder's number, where it is not true.
+
+`model-int8.onnx` does not reach it: it flips the blunder decision on 1.3 % of
+positions, roughly one in 77. That is the file the WASM fallback loads, so a phone on the
+int8 build gets a measurably different alert from the one in the results table above.
 
 ## Training recipe
 
@@ -276,7 +281,7 @@ The pretraining checkpoint the heads started from: `checkpoints/encoder-mmm-v4-2
 
 ## In the course
 
-- Built in [M3 · El encoder: los labs](https://lab.rukh.borjaglez.com/curso/m3/03-labs-del-encoder/), the lesson that runs every command behind this repository.
+- Built in [M3 · El encoder: los labs](https://lab.rukh.borjaglez.com/curso/m3/11-labs-del-encoder/), the lesson that runs every command behind this repository.
 - Measured in the [single results table](https://lab.rukh.borjaglez.com/proyecto/) as the stage `encoder-v4`: every stage of the course, the same suite, the same day.
 - Bring it to the paths the configs read: `uv run rukh pull encoder-v4`.
 

onnx/model-fp16.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c418d19953f58d9c47806723f137f86a914ba3e853e5c6819f3b0845212ce26e
-size 78745459
+oid sha256:a0297a40964208925840a32dda80ef4ba32210ce4ae08a16f2c58cebd0abcad2
+size 78745496

onnx/model-int8.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d19e8df829d5c6ed94f81d02ba27e526444791004999a81210312820040d10bd
-size 43393427
+oid sha256:a620e0bee38c8f5ea269a46fb419e0b77f7abc4e25851a08c238bf54a17bce83
+size 43393464

onnx/model.onnx

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5b83eb089a88a2ccee2eb276d6e0b2b05d42a0bbebb12438ac3bf6b2c0635621
-size 156660338
+oid sha256:65d9b4cd2b6061d6d3b8036810f371dbb96783a22134e0c22f2d742a0ef3a019
+size 156660375

onnx/parity.json

@@ -1,7 +1,7 @@
 {
   "version": 1,
   "kind": "encoder",
-  "measures": "the blunder decision at p >= 0.5",
+  "measures": "the blunder decision at p >= 0.0961714",
   "positions": 1000,
   "source": "validation-labels",
   "exporter": "dynamo",
@@ -22,7 +22,7 @@
     },
     "int8": {
       "file": "model-int8.onnx",
-      "agreement": 1.0,
+      "agreement": 0.987,
       "max_abs_value_delta": 0.11746431887149811,
       "max_abs_blunder_delta": 0.022291138768196106
     }