ProMeText/aquilign-multilingual-segmenter

1---2license: cc-by-nc-sa-4.03language:4- la5- fr6- en7- pt8- ca9- es10- it11pipeline_tag: token-classification12library_name: transformers13tags:14- medieval-texts15- phrase-segmentation16- multilingual17---18 19 20# Aquilign Multilingual Segmenter21 22**Aquilign Multilingual Segmenter** is a token-classification model for phrase-level segmentation of medieval and historical texts.23 24The model is designed to detect custom segmentation delimiters in multilingual historical corpora and is used as part of the [Aquilign](https://github.com/ProMeText/Aquilign) alignment workflow.25 26## Model Description27 28The segmenter is based on a trainable `BertForTokenClassification` model from Hugging Face’s `transformers` library.29 30It was fine-tuned on historical prose from the [Multilingual Segmentation Dataset](https://github.com/ProMeText/multilingual-segmentation-dataset) to identify phrase-level segmentation boundaries.31 32## Supported Languages33 34- Latin35- French36- Castilian37- Portuguese38- Catalan39- English40- Italian41 42## Intended Use43 44This model is intended for:45 46- phrase-level segmentation of **medieval texts**47- preprocessing parallel corpora before alignment48- multilingual medieval text alignment workflows49- digital philology and computational humanities research50 51It is especially designed to be used with [Aquilign](https://github.com/ProMeText/Aquilign).52 53 54## Related Resources55 56- [Aquilign alignment tool](https://github.com/ProMeText/Aquilign)57- [Multilingual Segmentation Dataset](https://github.com/ProMeText/multilingual-segmentation-dataset)58- [ProMeTEXT GitHub organization](https://github.com/ProMeText)59 60## Citation61 62If you use this model, please cite the related dataset and publication.63 64### Dataset65 66```bibtex67@dataset{ing2025multilingual,68  author       = {Ing, L. and Gille Levenson, M. and Macedo, C.},69  title        = {Multilingual Segmentation Dataset for Historical Prose (13th--16th c.)},70  year         = {2025},71  publisher    = {Zenodo},72  version      = {1.0},73  doi          = {10.5281/zenodo.16992629},74  url          = {https://doi.org/10.5281/zenodo.16992629},75  license      = {Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International}76}77```78 79### Related Publication80```bibtex81@inproceedings{ing-etal-2026-phrase,82  title = {Phrase-Level Segmentation on Medieval Corpora for Aligning Multilingual Texts},83  author = {Ing, Lucence and Gille Levenson, Matthias and Macedo, Carolina},84  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference (LREC 2026)},85  month = {May},86  year = {2026},87  pages = {936--946},88  address = {Palma, Mallorca, Spain},89  publisher = {European Language Resources Association (ELRA)},90  doi = {10.63317/32huzuuokpfr}91}92```