countrycode-small-v1.2.1

README.md

@@ -7,7 +7,7 @@ tags: [country-codes, iso-3166, entity-normalization, selective-prediction, cali
 base_model: jhu-clsp/mmBERT-small
 ---
 
-# CountryCode-Decide (small, v1.2)
+# CountryCode-Decide (small, v1.2.1)
 
 Maps a country or US state written as free text — any language or script, misspellings,
 abbreviations, historical names, register junk — to ISO 3166-1 alpha-2 / ISO 3166-2:US codes with
@@ -40,7 +40,7 @@ uvx --from huggingface_hub hf download aildan/country-code-decide-small --exclud
 from countrycode.infer.pipeline import CountryCoder
 
 coder = CountryCoder.load("models/country-code-decide-small")
-d = coder.code("Untted Klngdom", field="country", domain="register", alpha=0.02)
+d = coder.code("Untted Klngdom", field="country", domain="register", alpha=0.05)
 d.target, d.candidates, d.abstain     # ('GB', ('GB',), False); d.confidence ≈ 0.95
 ```
 
@@ -48,9 +48,10 @@ Input: `text`, optional `field` (`country`, `jurisdiction`, `nationality`, `unkn
 `register_country` (ISO alpha-2 of the register, or `US-XX` for a US state register). Output per
 string: `target` (alpha-2, `US-XX`, `NONE` or `None` = abstain), `country`, `us_state`,
 `confidence`, `candidates` (codes with calibrated probability ≥ 0.05, at most three), `abstain`,
-for an error-rate target `alpha` (certified targets — curated: 5 %; register: 2 %, 5 %; `None` always answers). Pass
+for an error-rate target `alpha` (certified targets — curated: 5 %; register: 5 %; `None` always answers). Pass
 `domain="register"` for register and form fields: the certified register calibration applies only
-then.
+then; without a domain the stricter of the two policies applies, which is conservative but not
+itself certified.
 
 ## Files
 
@@ -85,10 +86,11 @@ generated (synthetic)**; no generated string is used for calibration or evaluati
   - `junk`, `regjunk` — strings that are not places: placeholders, dates, numbers, generated
     company names, legal terms, street lines (→ `NONE`).
 
-  Every release passed five quality gates before training: **G1** label correctness (each string
+  Every release the model uses passed the quality gates before training: **G1** label correctness (each string
   checked against every name of every code; strings closer to another code are dropped) plus a
   human audit of 300 random items per release (every item when a release is smaller) with zero
-  label errors required; **G2** realism (edit statistics compared with real typos); **G3**
+  label errors required; **G2** realism (edit statistics compared with real typos — reported, no
+  pass/fail threshold); **G3**
   usefulness (kept only if it improved held-out real strings; the OCR, LLM-paraphrase and
   region generators failed this and are not used); **G4** no overlap with any evaluation string; **G5** reproducibility
   (code, config, seed and source item recorded per string).
@@ -115,39 +117,59 @@ generated (synthetic)**; no generated string is used for calibration or evaluati
 
 Fresh registers never used for training, calibration or model selection (New York, Oregon): every
 distinct string of their jurisdiction and country fields, labelled by exact lexicon match or by a
-human reviewer, read once after a frozen pre-registration.
+human reviewer, read once after a frozen pre-registration (release 1.2). Release 1.2.1 keeps the
+weights and corrects the certificate after an external review, so its numbers below are post-hoc.
 
-| Fresh registers, 361 distinct strings, read once | CountryCode-Decide small v1.2 |
+| Fresh registers, 361 distinct strings | CountryCode-Decide small v1.2.1 |
 |---|---|
 | Accuracy, every string answered | 88.6 % (95 % CI 85.3–91.7) |
 | Right country (a US state counts as US) | 91.2 % |
-| At the certified 2 % error target | answers 86 % automatically, realised error 1.0 % |
-| Best non-LLM baseline (multilingual-e5 nearest name) | 82.5 % |
+| At the certified 5 % error target | answers 88 % automatically, realised error 2.5 % |
+| Best non-LLM baseline (exact lexicon + context rules) | 89.5 % |
 | Zero-shot LLM (qwen3.5_35b-a3b-q4_K_M) | 85.0 %, ~29× slower (LLM on a GPU, this model on a CPU) |
 | Model file · CPU latency | 75 MB (int8 ONNX) · p50 4.1 ms per string |
 
+| Predictor | automatic labels (exact lexicon match, n = 261) | hand-labelled (n = 100) | hand-labelled answered |
+|---|---|---|---|
+| CountryCode-Decide (always answers) | 99.2 % | 61.0 % | 100 |
+| CountryCode-Decide at the certified 5 % target | 98.1 % | 56.0 % | 61 |
+| Zero-shot LLM (qwen3.5_35b-a3b-q4_K_M) | 94.3 % | 61.0 % | 97 |
+| Lexicon + context rules | 99.6 % | 63.0 % | 62 |
+| multilingual-e5-small nearest name | 100.0 % | 37.0 % | 100 |
+| exact lexicon match | 100.0 % | 24.0 % | 23 |
+
+Automatic labels come from an exact lexicon match, so a lexicon is right on them by construction; on
+the hand-labelled strings the model, the LLM and an exact lexicon with the training context rules are
+about level. On held-out strings of the training registers (656 strings, many misspelled or abbreviated; other strings of the same registers were in training) the model is right on 82.6 %, the best other predictor (qwen3.5_35b-a3b-q4_K_M zero-shot) on 73.8 %, the lexicon with context rules on 35.8 %; on the 472 of them with no exact lexicon match 81.1 % against 73.1 %. On GBIF's curated list the LLM is better: 69.6 % against 58.8 %.
+
 - Country-level accuracy (right country; a US state counts as US): 0.912 (highest: v1_model 0.921).
 - On the 33 strings not in the lexicon (no exact name match) the model's accuracy is 0.212.
-- At the certified 2 % error target the model answers 86% of the strings automatically with a realised error of 1.0 %; non-places get a code 0.0 % of the time (n = 4). The 2 % certificate (fixed-sequence test on the register calibration strings) is not contradicted on the fresh registers. The 1 % target could not be certified.
+- At the certified 5 % error target the model answers 88% of the strings automatically with a realised error of 2.5 %; non-places get a code 0.0 % of the time (n = 4). The 5 % target is certified over 318 register units (one per register and normalised key); 1 % and 2 % are not.
 - The zero-shot LLM (qwen3.5_35b-a3b-q4_K_M) is less accurate when every string must be answered (0.850 vs the model's 0.886); its own confidence covers 71% of the strings at 5 % empirical risk, without a certificate. It is ~29× slower (the LLM on a GPU, this model on a CPU).
 
+The guarantee is an average over strings like the calibration strings, and the errors concentrate on hard ones: at the 5 % target 13.1 % of the 61 answered hand-labelled strings are wrong, 0.0 % of the 256 answered strings that an exact lexicon match labelled.
+
+**The certificate, corrected.** The pre-registered certificate treated the 592 register calibration strings as independent draws and certified 2 % and 5 %. The same string often appears in several fields of a register, so they are not independent: they form 318 register-and-key units. Counted per unit — a correction made after the test read, prompted by an external review — 2 % is not certified, and 5 % is, at λ = 0.762. This release (small-v1.2.1) ships that calibration; the weights are unchanged.
+
 | predictor | n | acc | acc (freq-weighted) | country acc | coverage @1 % risk | coverage @5 % risk | non-place false accept | ECE | ms / item |
 |---|---|---|---|---|---|---|---|---|---|
+| lexicon_context | 361 | 0.895 | 1.000 | 0.901 | 0.000 | 0.895 | 0.250 | 0.015 | 0.0 |
 | model | 361 | 0.886 | 1.000 | 0.912 | 0.856 | 0.911 | 0.750 | 0.058 | 4.2 |
-| v1_model | 361 | 0.884 | 1.000 | 0.921 | 0.197 | 0.864 | 1.000 | 0.253 | 3.7 |
-| model@0.05 | 361 | 0.873 | 1.000 | 0.881 | 0.856 | 0.903 | 0.000 | 0.041 | 4.1 |
-| model@0.02 | 361 | 0.856 | 1.000 | 0.864 | 0.856 | 0.856 | 0.000 | 0.032 | 4.1 |
-| llm_qwen3.5_35b-a3b-q4_K_M | 361 | 0.850 | 0.997 | 0.858 | 0.000 | 0.709 | 0.000 | 0.108 | 116.9 |
+| v1_model | 361 | 0.884 | 1.000 | 0.921 | 0.197 | 0.864 | 1.000 | 0.253 | 3.9 |
+| model@0.05 | 361 | 0.864 | 1.000 | 0.873 | 0.856 | 0.878 | 0.000 | 0.033 | 4.2 |
+| llm_qwen3.5_35b-a3b-q4_K_M | 361 | 0.850 | 0.997 | 0.858 | 0.000 | 0.709 | 0.000 | 0.108 | cached |
 | e5_knn | 361 | 0.825 | 0.995 | 0.847 | 0.091 | 0.825 | 0.750 | 0.108 | 0.6 |
-| lexicon_fuzzy | 361 | 0.795 | 0.994 | 0.802 | 0.000 | 0.820 | 0.500 | 0.054 | 0.9 |
+| lexicon_fuzzy | 361 | 0.795 | 0.994 | 0.802 | 0.000 | 0.820 | 0.500 | 0.054 | 1.0 |
 | lexicon_exact | 361 | 0.789 | 0.994 | 0.793 | 0.000 | 0.787 | 0.250 | 0.014 | 0.0 |
 | ons_style | 361 | 0.787 | 0.994 | 0.793 | 0.000 | 0.792 | 0.500 | 0.044 | 0.3 |
-| countrynames | 361 | 0.634 | 0.761 | 0.745 | 0.000 | 0.000 | 0.000 | 0.226 | 2.5 |
-| pycountry | 361 | 0.634 | 0.761 | 0.776 | 0.000 | 0.000 | 0.250 | 0.184 | 1.4 |
+| countrynames | 361 | 0.634 | 0.761 | 0.745 | 0.000 | 0.000 | 0.000 | 0.226 | 2.7 |
+| pycountry | 361 | 0.634 | 0.761 | 0.776 | 0.000 | 0.000 | 0.250 | 0.184 | 1.5 |
 | coco | 361 | 0.596 | 0.761 | 0.601 | 0.000 | 0.000 | 0.250 | 0.120 | 0.5 |
 | model@0.01 | 361 | 0.011 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | – | 4.1 |
+| model@0.02 | 361 | 0.011 | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | – | 4.1 |
 
-Pre-registered hypotheses:
+Pre-registered hypotheses of release 1.2, as read (P1 and P2 rest on the item-level certificate
+corrected above):
 
 | hypothesis | result | evidence |
 |---|---|---|
@@ -157,23 +179,29 @@ Pre-registered hypotheses:
 | P4 | met | model_accuracy=0.886, best_non_llm=e5_knn, best_non_llm_accuracy=0.825 |
 | A1 | reported (not judged) | certified=False, answered=0.000, risk_answered=nan |
 
-Gold labels: The register calibration strings were labelled a second time, blind (no proposal and no first label shown): agreement 99.2 % on 592 strings (5 disagreements); one-sided 95 % upper bound on disagreement 1.8 %.
+Gold labels: The register calibration strings were labelled a second time, blind (no proposal and no first label shown): agreement 99.2 % on 592 strings (5 disagreements); one-sided 95 % upper bound on disagreement 1.8 %. On the 218 hand-labelled strings the passes disagree on 5 (2.3 %); the 374 automatic or preset labels agree by construction. Test labels were labelled once.
 
 Latency and size: p50 4.1 ms, p95 5.1 ms per string on CPU (4 threads, ONNX Runtime, tokenisation included); model file 75 MB.
 
-Browser: onnxruntime-web computes the 8-bit matrix products in fp32 where the CPU server uses int8, so probabilities differ slightly: on 3,570 held-out strings the browser makes the same decision as the server for 99.9 % at the 2 % target (same top code 98.9 %; top probability off by 0.005 on average, at most 0.08). The certificate applies to the server and Python path.
-
 ## Limitations and risks
 
+- For clean fields an exact lexicon with context rules is as accurate as this model; use the model
+  for what a lexicon cannot read. Its advantage was measured on held-out strings of registers whose
+  other strings were in training; on fresh registers, strings with no exact lexicon match are hard.
+- The certificate holds for strings drawn like the calibration strings (register fields), counted
+  per register and normalised key, each target on its own (with probability ≥ 90 % over the
+  calibration draw); errors concentrate on hard strings. Recalibrate before relying on it for a
+  different source. The curated GBIF domain is much harder than registers for this model, and there
+  a 35B LLM is more accurate.
 - One human reviewer labelled the register strings; the calibration strings were labelled twice by
-  that reviewer, the second time blind (consistency, not inter-annotator agreement).
-- Certified error rates hold for the population they were fitted on; recalibrate before relying on
-  them for a different source. Each target is certified on its own (with probability ≥ 90 % over the
-  calibration draw); the 2 % and 5 % guarantees are not simultaneous. The curated GBIF domain is much harder than registers for this
-  model, and there a 35B LLM is more accurate.
+  that reviewer, the second time blind (consistency, not inter-annotator agreement); test labels
+  were labelled once.
+- Of the 4,997 evaluation strings, 5 are not in Latin script, so accuracy on other scripts is not measured.
+- One training seed (13); the usefulness gate compared runs whose spread is about its margin.
 - Ambiguous names are returned as candidate lists; downstream systems must not silently pick one.
-- Known gaps: Canadian province codes in US jurisdiction fields (`ON`, `BC`) are not recognised;
-  `Congo` without context is answered as `CG`; strings with no exact lexicon match are much harder.
+- Known gaps: bare Canadian province codes in US jurisdiction fields (`ON`, `BC`) are not recognised
+  (every real such string is an evaluation string, so the no-overlap gate removed them from the
+  generated province variants); `Congo` without context is answered as `CG`.
 - Not legal advice; keep a person in the loop.
 
 ## Citation

calibration.json

@@ -3,14 +3,14 @@
   "curated": {
    "temps": [
     2.2867150467655692,
-    0.20570054404174923
+    1.0
    ],
    "thresholds": {
     "0.01": null,
     "0.02": null,
     "0.05": 0.7326620612561575
    },
-   "conformal_q": 0.9881178825692849,
+   "conformal_q": 0.9879384474540245,
    "delta": 0.1,
    "n_calib": 2915,
    "notes": {
@@ -21,132 +21,134 @@
     "orders": {
      "0.01": [
       0.8546291565996004,
-      0.8390992644282689,
-      0.815636762285532,
-      0.794996899525596,
-      0.7748608612350834,
+      0.8390176441806108,
+      0.8154103968963973,
+      0.794829109940437,
+      0.7746235878318407,
       0.7534030835576375,
       0.7326620612561575,
       0.7033246652891548,
       0.6761439611185102,
-      0.6475467692562777,
-      0.6213245643341573,
-      0.5946804043226823,
-      0.5653143285582992,
-      0.540053686766102,
-      0.5066697045408962,
-      0.48271092703635426,
-      0.45380416543196983,
-      0.4246718500834122,
-      0.3976970451765659,
-      0.37467128570170205,
-      0.3506514219211769,
-      0.32197280693188274,
-      0.2990604112196519,
-      0.27503146879371937,
-      0.252418403528896,
-      0.23091191183238988,
-      0.2141808864372958,
-      0.19620488113933368,
-      0.18272890196951042,
-      0.16927406464865355,
-      0.15402624306273502,
-      0.14341572573042205,
-      0.13224606865027388,
-      0.12309528268471495,
-      0.11278604736699106,
-      0.10315858712876423,
-      0.0946956273398957,
-      0.08657892920282398,
-      0.07406414692224761,
-      0.05809758489467727
+      0.6470571390741137,
+      0.621272642015397,
+      0.5945858554596489,
+      0.5644175562128235,
+      0.5383910801544289,
+      0.5053288200390005,
+      0.4811340553290624,
+      0.452356317766877,
+      0.42197435647087733,
+      0.3958372615386735,
+      0.37271575694400216,
+      0.3454190259897346,
+      0.32019942511666205,
+      0.2978205681901842,
+      0.27252864145905314,
+      0.25023107658087435,
+      0.2280356861181734,
+      0.2107918478555132,
+      0.1922878241655564,
+      0.1805002272443855,
+      0.1657752910487553,
+      0.15325223241615193,
+      0.14276862651155658,
+      0.13142562118108092,
+      0.12133406984392998,
+      0.11152977248316712,
+      0.10209767159471866,
+      0.09431107359946388,
+      0.08508807647979337,
+      0.07348928030213907,
+      0.05685468801548344
      ],
      "0.02": [
       0.8546291565996004,
-      0.8390992644282689,
-      0.815636762285532,
-      0.794996899525596,
-      0.7748608612350834,
+      0.8390176441806108,
+      0.8154103968963973,
+      0.794829109940437,
+      0.7746235878318407,
       0.7534030835576375,
       0.7326620612561575,
       0.7033246652891548,
       0.6761439611185102,
-      0.6475467692562777,
-      0.6213245643341573,
-      0.5946804043226823,
-      0.5653143285582992,
-      0.540053686766102,
-      0.5066697045408962,
-      0.48271092703635426,
-      0.45380416543196983,
-      0.4246718500834122,
-      0.3976970451765659,
-      0.37467128570170205,
-      0.3506514219211769,
-      0.32197280693188274,
-      0.2990604112196519,
-      0.27503146879371937,
-      0.252418403528896,
-      0.23091191183238988,
-      0.2141808864372958,
-      0.19620488113933368,
-      0.18272890196951042,
-      0.16927406464865355,
-      0.15402624306273502,
-      0.14341572573042205,
-      0.13224606865027388,
-      0.12309528268471495,
-      0.11278604736699106,
-      0.10315858712876423,
-      0.0946956273398957,
-      0.08657892920282398,
-      0.07406414692224761,
-      0.05809758489467727
+      0.6470571390741137,
+      0.621272642015397,
+      0.5945858554596489,
+      0.5644175562128235,
+      0.5383910801544289,
+      0.5053288200390005,
+      0.4811340553290624,
+      0.452356317766877,
+      0.42197435647087733,
+      0.3958372615386735,
+      0.37271575694400216,
+      0.3454190259897346,
+      0.32019942511666205,
+      0.2978205681901842,
+      0.27252864145905314,
+      0.25023107658087435,
+      0.2280356861181734,
+      0.2107918478555132,
+      0.1922878241655564,
+      0.1805002272443855,
+      0.1657752910487553,
+      0.15325223241615193,
+      0.14276862651155658,
+      0.13142562118108092,
+      0.12133406984392998,
+      0.11152977248316712,
+      0.10209767159471866,
+      0.09431107359946388,
+      0.08508807647979337,
+      0.07348928030213907,
+      0.05685468801548344
      ],
      "0.05": [
       0.8546291565996004,
-      0.8390992644282689,
-      0.815636762285532,
-      0.794996899525596,
-      0.7748608612350834,
+      0.8390176441806108,
+      0.8154103968963973,
+      0.794829109940437,
+      0.7746235878318407,
       0.7534030835576375,
       0.7326620612561575,
       0.7033246652891548,
       0.6761439611185102,
-      0.6475467692562777,
-      0.6213245643341573,
-      0.5946804043226823,
-      0.5653143285582992,
-      0.540053686766102,
-      0.5066697045408962,
-      0.48271092703635426,
-      0.45380416543196983,
-      0.4246718500834122,
-      0.3976970451765659,
-      0.37467128570170205,
-      0.3506514219211769,
-      0.32197280693188274,
-      0.2990604112196519,
-      0.27503146879371937,
-      0.252418403528896,
-      0.23091191183238988,
-      0.2141808864372958,
-      0.19620488113933368,
-      0.18272890196951042,
-      0.16927406464865355,
-      0.15402624306273502,
-      0.14341572573042205,
-      0.13224606865027388,
-      0.12309528268471495,
-      0.11278604736699106,
-      0.10315858712876423,
-      0.0946956273398957,
-      0.08657892920282398,
-      0.07406414692224761,
-      0.05809758489467727
+      0.6470571390741137,
+      0.621272642015397,
+      0.5945858554596489,
+      0.5644175562128235,
+      0.5383910801544289,
+      0.5053288200390005,
+      0.4811340553290624,
+      0.452356317766877,
+      0.42197435647087733,
+      0.3958372615386735,
+      0.37271575694400216,
+      0.3454190259897346,
+      0.32019942511666205,
+      0.2978205681901842,
+      0.27252864145905314,
+      0.25023107658087435,
+      0.2280356861181734,
+      0.2107918478555132,
+      0.1922878241655564,
+      0.1805002272443855,
+      0.1657752910487553,
+      0.15325223241615193,
+      0.14276862651155658,
+      0.13142562118108092,
+      0.12133406984392998,
+      0.11152977248316712,
+      0.10209767159471866,
+      0.09431107359946388,
+      0.08508807647979337,
+      0.07348928030213907,
+      0.05685468801548344
      ]
     },
-    "cert_accuracy_answer_all": 0.5879931389365352
+    "cert_accuracy_answer_all": 0.5883361921097771,
+    "cert_unit": "source+normalised key",
+    "n_cert_units": 2796
    },
    "candidate_rule": "min_p",
    "candidate_min_p": 0.05,
@@ -159,8 +161,8 @@
    ],
    "thresholds": {
     "0.01": null,
-    "0.02": 0.8273787041356887,
-    "0.05": 0.5755507073107454
+    "0.02": null,
+    "0.05": 0.7621416162421972
    },
    "conformal_q": 0.9997323706032406,
    "delta": 0.1,
@@ -172,22 +174,6 @@
     "n_state_temp": 287,
     "orders": {
      "0.01": [
-      0.9925414973102457,
-      0.9915781353254195,
-      0.9907942719266384,
-      0.9894260363762005,
-      0.987933664766585,
-      0.9869348188789901,
-      0.984970552423103,
-      0.981732858682758,
-      0.9800487587095484,
-      0.9776581347170207,
-      0.9760000197322725,
-      0.9737263416639429,
-      0.9686611034920956,
-      0.9659087754032977,
-      0.9642625980022396,
-      0.9582639057490853,
       0.9560924898447123,
       0.9519729487735396,
       0.9433091050310406,
@@ -204,14 +190,6 @@
       0.19320342648908526
      ],
      "0.02": [
-      0.9977847208372861,
-      0.9973991627848905,
-      0.9970712258744602,
-      0.9966005819956624,
-      0.9959485052934133,
-      0.9954949082688425,
-      0.9950891637253053,
-      0.9946737876579338,
       0.9942224845234497,
       0.993663439221272,
       0.9925414973102457,
@@ -288,7 +266,9 @@
       0.19320342648908526
      ]
     },
-    "cert_accuracy_answer_all": 0.9222972972972973
+    "cert_accuracy_answer_all": 0.9222972972972973,
+    "cert_unit": "source+normalised key",
+    "n_cert_units": 318
    },
    "candidate_rule": "min_p",
    "candidate_min_p": 0.05,

meta.json

@@ -1,6 +1,6 @@
 {
  "max_len": 48,
- "version": "v1.2",
+ "version": "v1.2.1",
  "run": "runs/r-trim-C",
  "base": "jhu-clsp/mmBERT-small",
  "pending_g1": false,
@@ -325,7 +325,7 @@
   "AT": "Austria",
   "AU": "Australia",
   "AW": "Aruba",
-  "AX": "\u00c5land Islands",
+  "AX": "Åland Islands",
   "AZ": "Azerbaijan",
   "BA": "Bosnia & Herzegovina",
   "BB": "Barbados",
@@ -336,7 +336,7 @@
   "BH": "Bahrain",
   "BI": "Burundi",
   "BJ": "Benin",
-  "BL": "St. Barth\u00e9lemy",
+  "BL": "St. Barthélemy",
   "BM": "Bermuda",
   "BN": "Brunei",
   "BO": "Bolivia",
@@ -354,7 +354,7 @@
   "CF": "Central African Republic",
   "CG": "Congo - Brazzaville",
   "CH": "Switzerland",
-  "CI": "C\u00f4te d\u2019Ivoire",
+  "CI": "Côte d’Ivoire",
   "CK": "Cook Islands",
   "CL": "Chile",
   "CM": "Cameroon",
@@ -363,7 +363,7 @@
   "CR": "Costa Rica",
   "CU": "Cuba",
   "CV": "Cape Verde",
-  "CW": "Cura\u00e7ao",
+  "CW": "Curaçao",
   "CX": "Christmas Island",
   "CY": "Cyprus",
   "CZ": "Czechia",
@@ -498,7 +498,7 @@
   "PW": "Palau",
   "PY": "Paraguay",
   "QA": "Qatar",
-  "RE": "R\u00e9union",
+  "RE": "Réunion",
   "RO": "Romania",
   "RS": "Serbia",
   "RU": "Russia",
@@ -519,7 +519,7 @@
   "SO": "Somalia",
   "SR": "Suriname",
   "SS": "South Sudan",
-  "ST": "S\u00e3o Tom\u00e9 & Pr\u00edncipe",
+  "ST": "São Tomé & Príncipe",
   "SV": "El Salvador",
   "SX": "Sint Maarten",
   "SY": "Syria",
@@ -535,7 +535,7 @@
   "TM": "Turkmenistan",
   "TN": "Tunisia",
   "TO": "Tonga",
-  "TR": "T\u00fcrkiye",
+  "TR": "Türkiye",
   "TT": "Trinidad & Tobago",
   "TV": "Tuvalu",
   "TW": "Taiwan",
@@ -981,5 +981,6 @@
   },
   "all_pass": true
  },
- "train_run": "small-v12f"
+ "train_run": "small-v12f",
+ "calibration_note": "v1.2 weights; certificate over (source, normalised key) units"
 }