phenningsson/sdhk-ner-old-swedish-v2

Named Entity Recognition for Old Swedish Charters (SDHK)

This model performs Named Entity Recognition (NER) on Old Swedish charter digital editions from the Svenskt Diplomatariums Huvudkartotek (SDHK), the main catalogue of the Swedish Diplomatarium maintained by the Swedish National Archives (Riksarkivet). The model identifies Person and Location entities, with Organisations collapsed into the Location category.

Model Description

• —Model type: Token classification (NER)
• —Base model: phenningsson/sdhk-mlm-pretrained-full (XLM-RoBERTa-Large domain-adapted to Old Swedish)
• —Language: Old Swedish (medieval Swedish charter language; training corpus drawn from charters dated 1380–1382)
• —Entity types: Person, Location (BIO tagging scheme)
• —F1 Score: 0.976

This model is fine-tuned for NER through token classification on a manually verified corpus of Old Swedish charters from 1380-1382 produced by a three-signal entity projection pipeline. The base language model is an XLM-RoBERTa-Large model which was first domain-adapted to Old Swedish through continued Masked Language Modelling (MLM) on the full SDHK Old Swedish corpus, and then fine-tuned for token classification on the manually verified training dataset of Old Swedish charters from 1380-1382.

Intended Uses

• —NER of Old Swedish charter digital editions
• —Digital humanities research on medieval Swedish documents
• —Semi-automatic annotation of historical Swedish medieval diplomatic sources
• —Information extraction from Old Swedish charters, deeds, and letters of the late medieval period

How to Use

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("phenningsson/sdhk-ner-old-swedish-v2", add_prefix_space=True)
model = AutoModelForTokenClassification.from_pretrained("phenningsson/sdhk-ner-old-swedish-v2")

# Use aggregation_strategy="first" to properly combine subword tokens
ner_pipeline = pipeline("ner", model=model, tokenizer=tokenizer, aggregation_strategy="first")

text = "Alla thøm thetta breff høra eller sea helsar husfru Jngeburg Thordsdotter af Liliostadhum æwerdhelika medh Gudhi ."
results = ner_pipeline(text)

for entity in results:
    # entity['word'] can corrupt multi-byte UTF-8 chars; use start/end offsets instead
    word = text[entity['start']:entity['end']]
    print(f"{word}: {entity['entity_group']} ({entity['score']:.3f})")

Expected output:

Jngeburg Thordsdotter: Person (1.000)
Liliostadhum: Location (1.000)

Training Data

A voting function combines the three signals into the final entity projection. The output is then manually verified by creator of this model, and an initial NER model is trained. The initial NER model is then applied to a set of unannotated charters from 1380-1382, and the model predictions are then manually corrected, and then used as training data. This process results in the data/1380_1382_dataset/. More information is available at the companion repository.

Training set statistics (80/10/10 split, seed = 42):

Evaluation sets:

The expert test set was produced independently by expert annotators working on the original SDHK charter digital editions, with adjudication of the 10 charters shared between the annotators and used to calculate the inter-annotator agreement (IAA).

Evaluation Results

Reported metrics are entity-level (exact span + type) via seqeval, computed on the 75-charter expert test set:

Per-class results:

Inter-annotator agreement (10 shared charters across domain expert annotators, computed via NLTK AnnotationTask):

Labels

The model uses BIO tagging with the following labels:

Limitations

• —Time period: The training corpus is drawn exclusively from digital editions of SDHK charters dated 1380–1382. Performance may degrade on Old Swedish texts from earlier or substantially later periods, texts in different genres (e.g. literary or religious texts), or indeed digital editions that use a different editorial convention than the modern one of the SDHK.
• —Domain: Optimised for charter language, i.e. relatively formulaic prose with frequent dating clauses, witness lists, and Latin formulaic vocabulary. Performance on other Old Swedish text types is not represented.
• —Entity types: Only Person and Location entities are supported, with Organisation entities being tagged as Location entities. Other entity types (dates, monetary values, etc.) are not recognised.

Domain-Adaptation

The backbone of this NER model is `phenningsson/sdhk-mlm-pretrained-full`, which in turn is a model based on XLM-RoBERTa-Large that has been domain adapted using the full SDHK Old Swedish corpus (all SDHK charters whose edition text is in Swedish). Domain adaptation was carried out prior to NER fine-tuning in order to familiarise the model with Old Swedish orthography, morphology, and charter vocabulary that is not represented in the multilingual XLM-RoBERTa pre-training data.

MLM hyperparameters:

• —Base model: xlm-roberta-large
• —Epochs: 8
• —Effective batch size: 32 (batch 2 × gradient accumulation 16)
• —Learning rate: 3e-5
• —Max sequence length: 256
• —Warm-up ratio: 6%
• —LR schedule: cosine
• —MLM masking probability: 15%

NER Training

NER is framed as a token classification task, with a classification head added on top of the domain-adapted XLM-RoBERTa-large model. To eliminate truncation of entity spans in long charter sentences, training uses O-boundary chunking: sequences that exceed the subword budget are split only at tokens labelled O, so every entity span remains intact and every token is seen during training.

NER hyperparameters:

• —Epochs: 5
• —Learning rate: 2e-5
• —Batch size: 16
• —Max sequence length: 512
• —Warm-up ratio: 10%
• —Weight decay: 0.01
• —Early stopping: patience 3 (on dev F1)

Class imbalance handling:

• —Weighted cross-entropy loss with class weights: 1.0 for the O class and 5.0 for the entity classes, to compensate for the ~10:1 token-level imbalance in the training corpus.

Citation

If you use this model, please cite:

@mastersthesis{Henningsson2086552,
   author = {Henningsson, Pontus},
   institution = {Linnaeus University, Department of Cultural Sciences},
   pages = {84},
   school = {Linnaeus University, Department of Cultural Sciences},
   title = {Old Swedish Entities: Developing Named Entity Recognition for Medieval Charters Written in Old Swedish},
   keywords = {named entity recognition, Old Swedish, natural language processing, medieval Scandinavian, digital humanities},
   abstract = {The Old Swedish charters stored in the Swedish National Archives' Svenskt Diplomatariums huvudkartotek (SDHK) represent some of the most important source collections for studies of medieval Sweden, yet no dedicated open source Named Entity Recognition (NER) model exists for Old Swedish. This thesis evaluates whether such a model can be developed using SDHK charters through entity projection and NER training. An entity projection pipeline is created to automatically annotate entities in Old Swedish charters from 1380-1382. The automated annotations are subsequently manually verified, resulting in an entity-annotated corpus of 417 charters used to train the NER model. Both the pipeline and model are evaluated using F1 scores on a test set of 75 charters from 1375-1382 annotated by domain experts. Inter-annotator agreement is calculated using Krippendorff's alpha and pairwise F1 to evaluate the reliability of the expert annotations and thereby validate the test set. The pipeline and NER model achieve F1 scores of 0.6987 and 0.9764 respectively, and the inter-annotator agreement reaches a Krippendorff's alpha of 0.9812 and pairwise F1 of 0.967, which indicates very high annotator agreement. Beyond these results, the thesis discusses the nature of annotation and mediation of Old Swedish entities, where the developed resources of the thesis are best understood as the result of a chain of mediation and interpretation of the Old Swedish charter texts. This thesis contributes an openly available NER model, entity projection pipeline, and entity-annotated corpus for Old Swedish, while demonstrating how entity annotation and extraction are tinged with interpretative decisions that ultimately reflect what, and how, Old Swedish entities can be represented. },
   year = {2026},
   url  = {https://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-148739}
}

Resources

For more information about the code, data, and pipeline, see the companion GitHub repository. The thesis describing this work is available below.

• —Code & Data: GitHub Repository
• —Domain-adapted model: phenningsson/sdhk-mlm-pretrained-full
• —Thesis: https://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-148739
• —Underlying base model: FacebookAI/xlm-roberta-large

Acknowledgments

Sincere gratitude is extended to the excellent resources and their contributors below, which has made it possible for this academic research and development of open source NER models for Old Swedish to be conducted. Special thanks is given to the expert annotators who graciously annotated Person and Location entities in the 75 charters that form the expert gold test set used for evaluation in this project.

• —SDHK — Svenskt Diplomatariums Huvudkartotek: Riksarkivet
• —TORA — Topografiskt Register: Riksarkivet
• —Diplomatarium Fennicum: Kansallisarkisto
• —Sveriges medeltida personnamn (SMP): Institutet för språk och folkminnen
• —KB-BERT Swedish models: KBLab
• —XLM-RoBERTa-large base model: Conneau et al. 2020

License

This model is released under the GNU General Public License v3.0 (GPL-3.0).

Contact

For questions or issues, please open an issue on the GitHub repository or contact: phenningsson@me.com