Upload README.md with huggingface_hub

README.md

@@ -104,32 +104,32 @@ printf 'Қазақша\nHello\n' | spellman detect --lines               # kaz /
 
 | eval | accuracy |
 |---|---|
-| held-out mix (225,559 rows, pristine test) | 98.62% |
-| Tatoeba (37,051, out-of-domain) | 98.57% |
-| Tatoeba granularity ladder (word / pair / triple) | 66.0 / 86.4 / 93.5 |
-| wild Russian tweets (2,606, label-audited; 61% @mentions, 20% URLs) | 91.17% |
-| COSMUS gold-labeled wild Russian (2,808, unseen) | 94.66% |
+| held-out mix (239,042 rows, pristine test) | 98.30% |
+| Tatoeba (37,051, out-of-domain) | 98.42% |
+| Tatoeba granularity ladder (word / pair / triple) | 66.7 / 86.2 / 93.3 |
+| wild Russian tweets (2,606, label-audited; 61% @mentions, 20% URLs) | 92.56% |
+| COSMUS gold-labeled wild Russian (2,808, unseen) | 96.79% |
+| short-utterance referee (orthography-certified, ≤19 chars) | 89.90% |
 | throughput | ~3.6 µs/sample (fp16 JIT plan, CPU) |
 
-On identical eval rows it beats fastText lid.176 (Tatoeba 94.90% →
-98.57%) and lingua 1.8 high-accuracy on every shared language after the
-wild-data lane, at ~30× its speed; whichlang and lingua cover only 10
-and 17 of these 30 classes respectively. Against open-LID SOTA GlotLID
-v3 it wins the wild held-out workload (98.62 vs 94.76) and the
-single-word rung (66.0 vs 43.9) while running ~100× faster; GlotLID's
-much larger training set still wins clean out-of-domain sentences
-(99.25 vs 98.57).
+On identical eval rows it beats fastText lid.176 (Tatoeba 94.90% vs
+98.42%) and lingua 1.8 high-accuracy on the shared languages at ~30x
+its speed; whichlang and lingua cover only 10 and 17 of these 30
+classes. Against open-LID SOTA GlotLID v3 it wins the wild held-out
+workload (98.30 vs 93.58), the single-word rung (66.7 vs 43.9), the
+wild-Russian referee (92.56 vs 82.73) and the short-utterance referee
+(89.90 vs 71.25) while running ~100x faster; GlotLID's larger training
+set still leads clean out-of-domain sentences (99.25 vs 98.42).
 
 ## Training data
 
-FineWeb-2 per-language configs sampled as line-windows (ODC-BY) +
-FineWeb (English), ~104k Tatoeba sentences (CC BY 2.0 FR), per-language
-community/parallel corpora for the weak classes, and a wild-register
-lane of real social media: 1.85M Ukrainian tweets (ukr-twi-corpus),
-Telegram Uzbek (tahrirchi/uz-crawl), Chuvash web corpora (CC0), Kazakh
-reviews (KazSAnDRA), Leipzig community crawls (ky/tg/uz), cardiffnlp
-multilingual tweets, sentiment140, German-politics 2025 tweets, and
-Portuguese/Brazilian tweets — every source LID-hygiene-filtered with
-twin-language protection. The full recipe is recorded in the repository
-(data_mix2/manifest.json). Training methodology and the full recipe are
-documented in the spellman repository (design doc + training guide).
+FineWeb-2 per-language configs, ~104k Tatoeba sentences (CC BY 2.0 FR),
+per-language community/parallel corpora, a wild social-media lane (1.85M
+Ukrainian tweets, Telegram Uzbek, Chuvash CC0 web, Kazakh reviews,
+Leipzig community crawls, cardiffnlp multilingual tweets, sentiment140,
+2025 German-politics tweets), a verified short-utterance lane (3-19 char
+wild rows under 2-of-3 judge consensus and orthographic twin rules), and
+lexically-diverse rus/ukr sets selected by greedy lemma coverage
+(pymorphy3; 93% of the top-15k lemmas in 14k rows). Every source
+LID-hygiene-filtered with twin-language protection; the exact recipe is
+recorded in the repository (data_mix4/manifest.json).