geosearch-ner

Multilingual toponym NER, fine-tuned from urchade/gliner_multi-v2.1. Labels: CITY, REGION, STATE, COUNTRY.

Span F1 0.763 (P 0.759 / R 0.767) on 47 held-out queries at threshold 0.4.

This is the first stage of a multilingual toponym search pipeline:

text
  -> GLiNER (this model)
  -> char-n-gram BM25 retrieval
  -> CatBoost reranker
  -> ranked GeoNames places

Serving it: the word-splitter contract

This checkpoint requires a per-ideograph word splitter, and serving it without one loses every Chinese span with no error of any kind.

GLiNER classifies token spans, and its default whitespace splitter treats a run of Han characters as one token — so 莫斯科 inside 莫斯科新闻 is not a span the model can even express, let alone predict. This model was trained with one token per Han ideograph (src/ner/tokenizer.py::CjkAwareSplitter), which gliner_config.json cannot record: words_splitter_type names only GLiNER's built-in kinds.

So the requirement is recorded in ner_meta.json, shipped beside the weights in this repo:

{"words_splitter": "src.ner.tokenizer.CjkAwareSplitter"}

The serving code reads that file and refuses to start when the configured splitter disagrees with it. If you load this model yourself, re-attach an equivalent splitter — measured effect of getting it wrong: 56/56 Chinese spans found versus 0.

Decision threshold

Served at 0.3, below GLiNER's own 0.5, and that is a property of serving rather than of the model: retrieval is recall-hungry, because a city span never extracted can never be retrieved, while a spurious span only adds a candidate the reranker can demote. The value is derived from the sweep in make ner-eval --sweep, not asserted.

Results

Overall

bucket	precision	recall	F1	gold	predicted
overall	0.759	0.767	0.763	86	87

By label

bucket	precision	recall	F1	gold	predicted
CITY	0.855	0.887	0.870	53	55
REGION	0.800	0.267	0.400	15	5
STATE	0.333	0.833	0.476	6	15
COUNTRY	0.833	0.833	0.833	12	12

By language

bucket	precision	recall	F1	gold	predicted
ru	0.818	0.818	0.818	22	22
en	0.809	0.809	0.809	21	21
tr	0.708	0.708	0.708	24	24
zh	0.700	0.737	0.718	19	20

Versus the zero-shot baseline

Zero-shot urchade/gliner_multi-v2.1, measured with the same word splitter (otherwise the Chinese delta would be an artefact of segmentation rather than of fine-tuning):

bucket	precision	recall	F1	gold	predicted
overall	0.726	0.616	0.667	86	73

Metrics are micro-averaged over spans, not macro over queries: a query naming three cities feeds three names into retrieval, so each one is a unit of work the pipeline either gets right or does not.

Training

Selected by validation span F1, not eval_loss — the checkpoint saved is whichever epoch scored best when called the way the engine calls it (predict_entities on raw text).

best epoch	12
best val span F1	0.763
epochs	15
batch size	8
learning rate	5e-06
others LR	1e-05
weight decay	0.01
warmup ratio	0.1
seed	42
trained at	2026-08-07T08:49:14+00:00

Trained on synthetic queries generated for this project — see mki0809/geosearch-queries.

Data licence and attribution

Derived from GeoNames, licensed CC BY 4.0.

Modifications made to the source data:

• filtered to populated places only (feature_class = 'P')
• restricted to RU, US, TR, CN
• restricted to name variants in ru, en, tr, zh
• dropped feature codes PPLH, PPLQ, PPLW, PPLX
• name variants grouped per place and joined into single document strings

The fine-tune inherits any licence conditions of its base model; check urchade/gliner_multi-v2.1 before commercial use.