Datasetpublic

uctnlp/mzansi-text

MzansiText MzansiText is a curated multilingual pretraining corpus for all eleven official South African languages. Dataset details Splits: 3,943,584 train rows, 19,379 validation rows, and 19,341 test rows lang values: afr, eng, nbl, nso, sot, ssw, tsn, tso, ven, xho, zul Schema: { "text": "string", "lang": "string" } Validation and test sets are capped at approximately 2M tokens per language to prevent high-resource languages from dominating early… See the full description on the dataset page: https://huggingface.co/datasets/uctnlp/mzansi-text.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

11likes767downloads

Dataset Card

MzansiText

MzansiText is a curated multilingual pretraining corpus for all eleven official South African languages.

![GitHub](https://github.com/Anri-Lombard/sallm) ![Paper](https://arxiv.org/abs/2603.20732) ![Model](https://huggingface.co/uctnlp/mzansilm-125m) ![Collection](https://huggingface.co/collections/anrilombard/mzansilm-69635ca7b60efedb9dfcb09e)

Dataset details

—Splits: 3,943,584 train rows, 19,379 validation rows, and 19,341 test rows
—lang values: afr, eng, nbl, nso, sot, ssw, tsn, tso, ven, xho, zul
—Schema:

json

{
  "text": "string",
  "lang": "string"
}

Validation and test sets are capped at approximately 2M tokens per language to prevent high-resource languages from dominating early stopping.

Release correction

The raw train split was republished on 10 August 2026 after an audit found that an earlier Hub upload had been created from an incomplete input tree. The earlier upload omitted WURA, ParaCrawl, and the isiXhosa/isiZulu mC4 files and contained 2,487,635 train rows. The paper statistics, validation split, test split, and full tokenized training artifact were not affected.

Token distribution (65,536-vocabulary MzansiText BPE tokenizer)

Language	Train tokens	%	Validation tokens	Test tokens
Afrikaans	2,475,913,822	64.96	1,865,255	1,875,605
English	740,994,679	19.44	1,813,651	1,821,803
isiZulu	320,224,015	8.40	2,017,406	2,021,343
isiXhosa	152,212,403	3.99	2,016,503	2,012,000
Sesotho	97,558,939	2.56	2,315,298	2,316,170
Setswana	10,082,930	0.26	1,216,539	1,413,473
Sepedi	6,697,358	0.18	685,425	778,656
Xitsonga	3,013,408	0.08	510,463	319,496
siSwati	1,932,989	0.05	196,247	225,810
Tshivenda	1,852,481	0.05	191,495	243,315
isiNdebele	818,549	0.02	106,224	143,458
Total	3,811,301,573	100	12,934,506	13,171,129

Usage

python

from datasets import load_dataset

dataset = load_dataset("uctnlp/mzansi-text")
print(dataset)

Related releases

—Paper: arXiv:2603.20732
—Model: uctnlp/mzansilm-125m
—Tokenized corpus: uctnlp/mzansi-text-tokenized
—Code and configs: Anri-Lombard/sallm

Citation

bibtex

@misc{lombard2026mzansitextmzansilmopencorpus,
      title={MzansiText and MzansiLM: An Open Corpus and Decoder-Only Language Model for South African Languages},
      author={Anri Lombard and Simbarashe Mawere and Temi Aina and Ethan Wolff and Sbonelo Gumede and Elan Novick and Francois Meyer and Jan Buys},
      year={2026},
      eprint={2603.20732},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2603.20732},
}

License

Apache License 2.0