mki0809/geosearch-ner

geosearch-ner

Multilingual toponym NER, fine-tuned from `urchade/gliner_multi-v2.1`. Labels: CITY, REGION, STATE, COUNTRY.

Span F1 0.763 (P 0.759 / R 0.767) on 47 held-out queries at threshold 0.4.

This is the first stage of a multilingual toponym search pipeline:

text
  -> GLiNER (this model)
  -> char-n-gram BM25 retrieval
  -> CatBoost reranker
  -> ranked GeoNames places

Serving it: the word-splitter contract

This checkpoint requires a per-ideograph word splitter, and serving it without one loses every Chinese span with no error of any kind.

GLiNER classifies token spans, and its default whitespace splitter treats a run of Han characters as one token — so 莫斯科 inside 莫斯科新闻 is not a span the model can even express, let alone predict. This model was trained with one token per Han ideograph (src/ner/tokenizer.py::CjkAwareSplitter), which gliner_config.json cannot record: words_splitter_type names only GLiNER's built-in kinds.

So the requirement is recorded in `ner_meta.json`, shipped beside the weights in this repo:

{"words_splitter": "src.ner.tokenizer.CjkAwareSplitter"}

The serving code reads that file and refuses to start when the configured splitter disagrees with it. If you load this model yourself, re-attach an equivalent splitter — measured effect of getting it wrong: 56/56 Chinese spans found versus 0.

Decision threshold

Served at 0.3, below GLiNER's own 0.5, and that is a property of serving rather than of the model: retrieval is recall-hungry, because a city span never extracted can never be retrieved, while a spurious span only adds a candidate the reranker can demote. The value is derived from the sweep in make ner-eval --sweep, not asserted.

Results

Overall

By label

By language

Versus the zero-shot baseline

Zero-shot urchade/gliner_multi-v2.1, measured with the same word splitter (otherwise the Chinese delta would be an artefact of segmentation rather than of fine-tuning):

Metrics are micro-averaged over spans, not macro over queries: a query naming three cities feeds three names into retrieval, so each one is a unit of work the pipeline either gets right or does not.

Training

Selected by validation span F1, not eval_loss — the checkpoint saved is whichever epoch scored best when called the way the engine calls it (predict_entities on raw text).

Trained on synthetic queries generated for this project — see `mki0809/geosearch-queries`.

Data licence and attribution

Derived from GeoNames, licensed CC BY 4.0.

Modifications made to the source data:

• —filtered to populated places only (feature_class = 'P')
• —restricted to RU, US, TR, CN
• —restricted to name variants in ru, en, tr, zh
• —dropped feature codes PPLH, PPLQ, PPLW, PPLX
• —name variants grouped per place and joined into single document strings

The fine-tune inherits any licence conditions of its base model; check `urchade/gliner_multi-v2.1` before commercial use.