Modelpublic

kamranibiyev/azNER-xlmr

sourceHugging Facemitupdated 1mo agoView on Hugging Face

0likes327downloads

Model Card

azwikiner-xlmr

Named entity recognition for Azerbaijani: XLM-RoBERTa base fine-tuned on azWikiNER, a manually corrected corpus of 16,740 Azerbaijani Wikipedia segments with 16 entity categories. Trained in April 2021; construction and evaluation are described in Ibiyev & Novák, TSD 2021 (doi:10.1007/978-3-030-83527-9_26) and in the author's PhD dissertation (PPKE ITK, 2026).

Entity categories

CARDINAL, EVENT, FAC, GPE, LOC, MONEY, NORP, ORDINAL, ORG, PER, PERCENT, PROD, QUANTITY, TIME, WORKOFART, and TITLE (professions, roles and military ranks attached to person names, e.g. aktyor 'actor', şair 'poet'). TIME covers both times and dates; PROD corresponds to OntoNotes PRODUCT. The classifier head has two vestigial classes that do not occur in practice: an interim OTHER label (one training span) and an empty label inherited from a trailing newline in the label file.

Results

Entity-level scores on the azWikiNER test set (1,618 segments), computed with the standard conlleval script:

Tagset	Precision	Recall	F1
Full (16 types)	80.21	83.38	81.77
Common (15 types shared with zero-shot OntoNotes models)	81.30	84.37	82.81

Per-type F1 on the test set (full tagset):

Type	Gold spans	F1
GPE	1354	92.05
PER	519	86.45
LOC	254	66.93
ORG	248	68.95
TITLE	210	67.44
CARDINAL	146	77.74
TIME	131	77.61
WORKOFART	55	43.64
FAC	52	57.41
NORP	46	55.77
EVENT	37	81.16
ORDINAL	34	81.69
QUANTITY	29	81.36
PERCENT	10	100.00
MONEY	8	40.00
PROD	8	16.67

Training

Fine-tuned with the token-classification example of transformers 4.5.0 on 4 GPUs: AdamW, learning rate 1e-4 with linear decay, no warmup, no weight decay, per-device batch size 8 (effective 32), mixed precision (fp16), seed 42, 6 epochs. Splits: 13,483 train / 1,639 dev / 1,618 test segments. The full training configuration is included in this repository as training_args.bin.

Usage

python

from transformers import pipeline
 
ner = pipeline("ner", model="kamranibiyev/azwikiner-xlmr", aggregation_strategy="simple")
ner("Elnur Hüseynov Bakıda anadan olub.")

Limitations

The training text comes from Wikipedia; expect lower accuracy on other genres. The numeric and date-like categories entered the corpus through a zero-shot OntoNotes pass with rule-based merging rather than span-by-span human verification, so scores on those types should be read with care. Long inputs are truncated at the tokenizer limit. There is no DATE tag; dates fall under TIME.

Citation

bibtex

@inproceedings{ibiyev2021azwikiner,
  author    = {Kamran Ibiyev and Attila Nov{\'a}k},
  title     = {Using Zero-Shot Transfer to Initialize azWikiNER, a Gold
               Standard Named Entity Corpus for the Azerbaijani Language},
  booktitle = {Text, Speech, and Dialogue (TSD 2021)},
  pages     = {305--317},
  publisher = {Springer},
  year      = {2021},
  doi       = {10.1007/978-3-030-83527-9_26},
}

Dataset: https://doi.org/10.5281/zenodo.22019336