latincy/la_core_web_trf

la_core_web_trf: Latin NLP pipeline for spaCy. Trained on harmonized Universal Dependencies treebanks (Perseus, PROIEL, ITTB, LLCT, UDante, CIRCSE) with LASLA data. Includes POS tagging, morphological analysis, lemmatization, dependency parsing, named entity recognition, and sentence segmentation.

<p align="center"><img src="latincy-pipelines-v3_9-logo.png" alt="LatinCy Pipelines v3.9 — Catus" width="500"></p> <p align="center">_{« Catus » — from "De cato seu fele" in Conrad Gesner's <i>Historiae animalium liber primus de quadrupedibus viviparis</i> (1551), p. 318}</p>

What's new in 3.9.8

This is a maintenance release across the LatinCy pipeline family (la_core_web_sm/md/lg/trf). Model weights are unchanged from v3.9.x — no retraining. It also carries the 3.9.7 fix, which was built but never published. Evaluation metrics are not re-measured: the current test set (treebanks 3.10.13) differs from the one these weights were scored on, so the meta.json figures remain the v3.9 measurement.

• —Fixed empty lemmas. The edit-tree lemmatizer can select a delete-everything tree and emit "", which CSV/pandas consumers read back as NaN. The lookup lemmatizer now floors an empty lemma at the lowercased surface form. This model emitted "" for es and i on the dev corpus.
• —Fixed the sentence-boundary cascade after `):` (3.9.7). token_fix's bracket balancing matched whole tokens, so a closer glued to trailing punctuation (): is a single token) was missed, and suppressing one boundary widened the scan for the next — collapsing the rest of a parenthetical passage into one sentence. Bracket depth is now counted per character and the scan reads a snapshot of the original boundaries.
• —Dependency pins. latincy-preprocess>=0.2.0,<0.6.0 (0.4.0–0.5.1 are Greek-only releases; Latin behavior unchanged) and la-latincy-lookups>=1.0.0,<1.1.0 — an upper bound is now declared, because 1.2.x tables regress lemmatization and must not resolve. The duplicate spacy requirement is removed.

See CHANGELOG.md for full version history.

What's new in 3.9.6

This is a maintenance release across the LatinCy pipeline family (la_core_web_sm/md/lg/trf). Model weights are unchanged from v3.9.x — no retraining; the changes are to sentence segmentation and morphological post-processing behavior.

• —Fixed sentence segmentation at a terminal followed by a closing single quote. A closing single quote (' or ’) glued directly after a sentence terminal (., ?, !, ;) was suppressing a boundary the senter places for the closing double quote — so …non potestis?' Moritur… was left as a single sentence. The token_fix corrector now restores the missing boundary, bringing '/’ to parity with "; direct speech ending in a terminal + closing quote is again split from the following clause.
• —Added a deponent-verb tagging fallback. When the statistical tagger labels a known deponent lemma (e.g. sequor, loquor, hortor) as a plain verb, the harmonizer now corrects the fine-grained tag to verb/deponent, so deponents are tagged consistently even when the model misses the subcategory.

See CHANGELOG.md for full version history.

What's new in 3.9.5

This is a maintenance release across the LatinCy pipeline family (la_core_web_sm/md/lg/trf). Model weights are unchanged from v3.9.x — no retraining; the changes are to tokenization and lemmatization behavior.

• —Added ~110 classical abbreviations to the tokenizer. Praenomina (Sp., App., Tib.), nomina gentilicia, Republican voting tribes, magistracy and imperial titulature (Pr., Tr., Praef., Pont., P.M.), and epigraphic funerary formulae (D.M., H.S.E., S.T.T.L., S.P.Q.R.) now tokenize as single tokens with the period attached, so abbreviation periods no longer shatter sentence segmentation. These ship bundled with the model, so they work regardless of the installed spaCy version. Mixed-case and ALL-CAPS (epigraphic) forms only — lowercase is intentionally omitted so a genuine sentence-final word never loses its period.
• —Fixed lemmatizer punctuation handling. Lookalike Unicode punctuation (e.g. U+FE54 small semicolon) is now tagged PUNCT instead of inheriting the model's predicted tag.

See CHANGELOG.md for full version history.

What's new in 3.9.4

This is a maintenance release across the LatinCy pipeline family (la_core_web_sm/md/lg/trf). Model weights are unchanged from v3.9.x — no retraining; the change is to tokenization behavior.

• —Fixed a sentence-segmentation regression at Latin abbreviations. The tokenizer again inherits spaCy lang/la's abbreviation exceptions, so the period after a praenomen or title (M., C., Cn., Q., Ti., Aug., cos., …) stays attached to its token instead of detaching and being read as a sentence boundary. Historical and documentary prose — consular dating formulae and name lists such as M. Messala, M. Pisone consulibus, … — is no longer split into spurious sentences. A regression test now guards this behavior.
• —(trf only) Restored `.similarity()`. Token, Span, and Doc similarity again use the transformer's contextual vectors, via user hooks installed by the trf_vectors component.

See CHANGELOG.md for full version history.

What's new in 3.9.3

This is a maintenance release across the LatinCy pipeline family (la_core_web_sm/md/lg/trf). Model weights are unchanged from v3.9.x — no retraining; the changes are to tokenization behavior and packaging.

• —Fixed sentence segmentation around parentheticals. A tokenizer fix restores correct splitting of word-leading punctuation (e.g. the ( in (mirabile), so parenthetical text like Verba (mirabile dictu!) sunt clara. is no longer split mid-parenthesis. (Internally: the Latin tokenizer now re-asserts its prefix/infix rules after model load, which spaCy's serialization had dropped.)
• —Self-contained install. The token_fix and enclitic_splitter components are now bundled directly in the model, so the wheel installs cleanly from HuggingFace with no extra LatinCy packages required.

See CHANGELOG.md for full version history.

What's new in 3.9.2

This is an alignment release across the LatinCy pipeline family (la_core_web_sm/md/lg/trf). Model weights are unchanged from v3.9.x — no retraining was performed; the changes are to pipeline configuration and packaging.

• —Enclitic splitting is now handled by a dedicated enclitic_splitter component, decoupled from the tokenizer. Splitting of the enclitic -que (e.g. armaque* → arma + que*) is an explicit, configurable step in the pipeline rather than tokenizer behavior.
• —Sentence segmentation upgraded to [`la_senter`](https://huggingface.co/latincy/la_senter) v3.9.2. The upstream LatinCy sentence-segmentation model has been updated to v3.9.2 and is passed through to this pipeline. It adds the token_fix component, which runs after senter to repair sentence boundaries the statistical model mis-splits — keeping sentences intact across parentheticals, dash asides, and closing quotes.

See CHANGELOG.md for full version history.

Label Scheme

<details>

<summary>View label scheme (855 labels for 4 components)</summary>

</details>

Accuracy