Upload folder using huggingface_hub
Browse files
README.md
CHANGED
|
@@ -28,7 +28,26 @@ pipeline_tag: text-classification
|
|
| 28 |
> [`munod/tachyone-multi`](https://huggingface.co/munod/tachyone-multi)); measured numbers below come
|
| 29 |
> from `benchmarks/report.md`.
|
| 30 |
>
|
| 31 |
-
> **This revision is B-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
> acceptance closes.** Diagnosis settled B-15's recorded follow-up: no single temperature
|
| 33 |
> serves both regimes of the legal P3 basis (training slice acc 0.803 wants T β 1.5β2; the
|
| 34 |
> hard remainder slice at acc 0.372 wants T β₯ 10) *and* the acceptance read (acc 0.9956
|
|
@@ -177,21 +196,21 @@ interleaved touch-up (`training/interleave_continue.py`).
|
|
| 177 |
|
| 178 |
| Checkpoint | Split | Accuracy | ECE (calibrated) | p50 (ms) |
|
| 179 |
| --- | --- | --- | --- | --- |
|
| 180 |
-
| English (ModernBERT-large + five-domain LoRA r=16 + choice-head bank) | support | **1.0000** | 0.017 | 58.
|
| 181 |
-
| English, five-domain split | 5 domains | **0.9983** | 0.016 | 58.
|
| 182 |
-
| English, **holdout phrasing** (never in training) | 5 domains | **0.9825** | 0.032 |
|
| 183 |
-
| Multilingual (mmBERT-base + LoRA r=64 + fitted bank) β routed runtime | support | **0.
|
| 184 |
-
| Multilingual, five-domain split | 5 domains | **0.
|
| 185 |
-
| Multilingual, **holdout phrasing** (never in training) | 5 domains | **0.
|
| 186 |
|
| 187 |
The English ECE column reads the **B-18 fitted confidence** (below): a `peakedness Γ
|
| 188 |
strength` cell map for `choice` (fitted slice-balanced over the in-template and holdout
|
| 189 |
slices) plus the P3 `noul` map, with every temperature byte-unchanged β the P3 in-domain
|
| 190 |
`choice` miss (0.5243 at T=10.0) is history (0.0010, acceptance met). The multilingual rows
|
| 191 |
-
are the **routed system**:
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
|
| 196 |
Per primitive (English, five-domain): `choice` **0.9956**, `noul` **0.9996**, `score`
|
| 197 |
**0.9996**; (multilingual, five-domain): `choice` **0.9564**, `noul` **0.9136**, `score`
|
|
@@ -253,9 +272,10 @@ worst new domain **0.993**, per-domain ECE **0.0005β0.0038** (zero exceptions)
|
|
| 253 |
benchmark: multilingual support 0.9760, worst new domain 0.9747, per-domain ECE 0.0019β0.0202
|
| 254 |
(zero exceptions), strict 1.000, every language β₯ baseline; overall 0.9885 / ECE 0.0069 when
|
| 255 |
the multilingual checkpoint answers everything** β the routed rows above show the pair as
|
| 256 |
-
served. On the support-only *routed* split (13β15% of
|
| 257 |
-
rows
|
| 258 |
-
**L-013**
|
|
|
|
| 259 |
0.05 ECE target on that split, and on the **holdout** the B-1 per-language ECE criterion was
|
| 260 |
**NOT met** at B-15 (7 cells, worst `score/it` 0.1495) β the **B-16** cycle (serve-key
|
| 261 |
detector + fitted confidence, 2026-10-09) moved it to **11/12 with `score/it` 0.0521
|
|
|
|
| 28 |
> [`munod/tachyone-multi`](https://huggingface.co/munod/tachyone-multi)); measured numbers below come
|
| 29 |
> from `benchmarks/report.md`.
|
| 30 |
>
|
| 31 |
+
> **This revision is B-19: routing reads the same detector as calibration, and the routed
|
| 32 |
+
> pair finally matches the arms.** The docs' *"~13β15% of multilingual rows fall through to
|
| 33 |
+
> the English checkpoint"* was real β measured **14.4%** (five-domain) / **12.1%** (support):
|
| 34 |
+
> 405 empty states whose language lives only in the localized question plus 677 Latin states
|
| 35 |
+
> without signal (`nl` losing 35% of its rows). Routing now keys on
|
| 36 |
+
> `detect_calibration_language` over the state **and** the request's question texts (the B-16
|
| 37 |
+
> detector, diagonal 99.80/99.95% β the same function that keys temperatures and cells, so
|
| 38 |
+
> fit and serve cannot drift), through the additive `Router.route(state, question_texts=β¦)`;
|
| 39 |
+
> non-Latin scripts and the no-keyword path are byte-identical to history, and `und`/accented
|
| 40 |
+
> Latin now routes multilingual (accents are a non-English signal). **Misroutes 14.4% β
|
| 41 |
+
> 0.05%** (4/7,500; support 0; English 0, clean and noisy alike). Routed multilingual
|
| 42 |
+
> accuracy **0.9300 β 0.9993 / 0.9221 β 0.9957 / 0.9264 β 0.9828** (ECE β 0.006/0.003/
|
| 43 |
+
> 0.006); the six checkpoint-alone reports are byte-identical and English accuracy-identical
|
| 44 |
+
> (quality is gated checkpoint-alone per **L-013**). On the 1,082 rescued rows the English
|
| 45 |
+
> arm scored 0.4871 against the multilingual arm's **1.0000**. Disclosed as measured:
|
| 46 |
+
> MASSIVE 0.058 β **0.051** (rows re-routed between arms), typed-decisions 0.306 and XNLI
|
| 47 |
+
> 0.333 byte-unchanged; four Dutch states with no signal anywhere still reach English (the
|
| 48 |
+
> recorded L-013 default). Record: `BACKLOG.md` **B-19**.
|
| 49 |
+
>
|
| 50 |
+
> **The v0.12.0 revision was B-18 (history):** the English confidence chain was rebuilt and P3's in-domain
|
| 51 |
> acceptance closes.** Diagnosis settled B-15's recorded follow-up: no single temperature
|
| 52 |
> serves both regimes of the legal P3 basis (training slice acc 0.803 wants T β 1.5β2; the
|
| 53 |
> hard remainder slice at acc 0.372 wants T β₯ 10) *and* the acceptance read (acc 0.9956
|
|
|
|
| 196 |
|
| 197 |
| Checkpoint | Split | Accuracy | ECE (calibrated) | p50 (ms) |
|
| 198 |
| --- | --- | --- | --- | --- |
|
| 199 |
+
| English (ModernBERT-large + five-domain LoRA r=16 + choice-head bank) | support | **1.0000** | 0.017 | 58.2 |
|
| 200 |
+
| English, five-domain split | 5 domains | **0.9983** | 0.016 | 58.8 |
|
| 201 |
+
| English, **holdout phrasing** (never in training) | 5 domains | **0.9825** | 0.032 | 59.2 |
|
| 202 |
+
| Multilingual (mmBERT-base + LoRA r=64 + fitted bank) β routed runtime | support | **0.9993** | 0.006 | 54.5 |
|
| 203 |
+
| Multilingual, five-domain split | 5 domains | **0.9957** | 0.003 | 53.2 |
|
| 204 |
+
| Multilingual, **holdout phrasing** (never in training) | 5 domains | **0.9828** | 0.006 | 53.8 |
|
| 205 |
|
| 206 |
The English ECE column reads the **B-18 fitted confidence** (below): a `peakedness Γ
|
| 207 |
strength` cell map for `choice` (fitted slice-balanced over the in-template and holdout
|
| 208 |
slices) plus the P3 `noul` map, with every temperature byte-unchanged β the P3 in-domain
|
| 209 |
`choice` miss (0.5243 at T=10.0) is history (0.0010, acceptance met). The multilingual rows
|
| 210 |
+
are the **routed system**: since **B-19** the router keys on the calibration detector (state
|
| 211 |
+
+ localized question), so only **0.05%** of rows still reach the English checkpoint by
|
| 212 |
+
design (**L-013** β empty *and* unsignaled), and the routed rows sit where the arms sit
|
| 213 |
+
(five-domain routed **0.9957** against the multilingual checkpoint's own **0.9961**).
|
| 214 |
|
| 215 |
Per primitive (English, five-domain): `choice` **0.9956**, `noul` **0.9996**, `score`
|
| 216 |
**0.9996**; (multilingual, five-domain): `choice` **0.9564**, `noul` **0.9136**, `score`
|
|
|
|
| 272 |
benchmark: multilingual support 0.9760, worst new domain 0.9747, per-domain ECE 0.0019β0.0202
|
| 273 |
(zero exceptions), strict 1.000, every language β₯ baseline; overall 0.9885 / ECE 0.0069 when
|
| 274 |
the multilingual checkpoint answers everything** β the routed rows above show the pair as
|
| 275 |
+
served. On the support-only *routed* split (before B-19, 13β15% of
|
| 276 |
+
rows fell through to the English checkpoint by language routing β `.specs/project/BACKLOG.md`
|
| 277 |
+
**L-013**; since B-19 that is 0.05%, one detector deciding language everywhere) the
|
| 278 |
+
per-language story is unchanged in kind: three of six languages sit above the
|
| 279 |
0.05 ECE target on that split, and on the **holdout** the B-1 per-language ECE criterion was
|
| 280 |
**NOT met** at B-15 (7 cells, worst `score/it` 0.1495) β the **B-16** cycle (serve-key
|
| 281 |
detector + fitted confidence, 2026-10-09) moved it to **11/12 with `score/it` 0.0521
|