taqbaylit/tatoeba-en-kab

Tatoeba English-Kabyle Parallel Corpus

A cleaned and aligned English-Kabyle parallel corpus extracted from Tatoeba, with both direct en↔kab links and indirect kab→fra→en links.

Statistics

Source

• —Tatoeba direct en↔kab links
• —Indirect kab→fra→en links (Kabyle linked to French, French linked to English)

Cleaning Pipeline

1. 1.Character standardization: Fixed common character encoding errors:
2. 2.Greek epsilon ε (U+03B5) → Latin open e ɛ (U+025B)
3. 3.Greek capital sigma Σ (U+03A3) → Latin capital open E Ɛ (U+0190)
4. 4.Greek capital gamma Γ (U+0393) → Latin capital gamma Ɣ (U+0194)
5. 5.Greek small gamma γ (U+03B3) → Latin small gamma ɣ (U+0263)
6. 6.Cyrillic reversed ze Ԑ (U+0510) → Latin capital open E Ɛ
7. 7.Romanian/Turkish t-cedilla ţ → Kabyle geminate t
8. 8.Turkish g-breve ğ → Kabyle g-caron ǧ
9. 9.Normalized no-break spaces, zero-width spaces, non-breaking hyphens

1. 1.GlotLID v3 filtering: Language identification using GlotLID v3 (fasttext model model_v3.bin).
2. 2.Retained only sentences predicted as kab_Latn with high confidence
3. 3.Retention rate: 245,247 / 245,747 = 99.80%
4. 4.Average confidence: 0.9990
5. 5.Dropped: 543 sentences predicted as other languages (engLatn, turLatn, itaLatn, deuLatn, catLatn, shiLatn, etc.)

1. 1.Exact deduplication: Removed 4,440 duplicate (en, kab) pairs

1. 1.Quality filtering:
2. 2.Length filter: dropped pairs where either side < 3 chars or > 300 chars
3. 3.Ratio filter: dropped pairs with length ratio > 5×
4. 4.Removed empty, punctuation-only, and identical-sides pairs

Format

from datasets import load_dataset

ds = load_dataset("boffire/tatoeba-en-kab")

# Example:
# ds["train"][0] →
# {
#   "translation": {
#     "en": "When did you graduate?",
#     "kab": "Melmi i d-tewwimt igerdasen-nkent?"
#   }
# }

CLDR Kabyle Alphabet Compliance

The corpus is filtered to use only characters from the Unicode CLDR kab locale main exemplar set:

• —Lowercase: a b c č d ḍ e ɛ f g ǧ ɣ h ḥ i j k l m n p q r ṛ s ṣ t ṭ u w x y z ẓ
• —Uppercase: A B C Č D Ḍ E Ɛ F G Ǧ Ɣ H Ḥ I J K L M N P Q R Ṛ S Ṣ T Ṭ U W X Y Z Ẓ
• —Auxiliary (loanwords): o v

License

CC0 1.0 Universal (Public Domain) — same as Tatoeba source data.

Citation

If you use this dataset, please cite Tatoeba:

@misc{tatoeba,
  title = {Tatoeba: Collection of sentences and translations},
  howpublished = {\url{https://tatoeba.org}},
}

Changelog

• —2026-07-08: Initial release (v1.0)
• —244,954 aligned EN-KAB pairs
• —Extracted from Tatoeba weekly exports
• —Cleaned with GlotLID v3 filtering and character standardization