taqbaylit/kab-en-toponyms-sentences

English-Kabyle Parallel Corpus for Machine Translation

This dataset contains 32,024 grammatically flawless parallel sentence pairs mapping English to literary Kabyle (Taqbaylit kab).

This corpus was synthesized using a linguistically-informed morphosyntactic rule engine paired with clean OpenStreetMap toponym registries from boffire/kabyle-toponyms. It handles complex phonetic mutations natively, making it a state-of-the-art bootstrapping asset for fine-tuning Sequence-to-Sequence models like Meta AI's NLLB-200.

Dataset Structure

The corpus is delivered in a standard .jsonl (JSON Lines) format. Each line contains a single translation object mapped for seq2seq tasks:

{"translation": {"en": "I am in Constantine currently.", "kab": "Aql-iyi di Qsemṭina tura."}}
{"translation": {"en": "He was born in Tighilt Taouraght that year.", "kab": "Ilul di Tiɣilt Tawraɣt deg useggas-nni."}}
{"translation": {"en": "He was born in Ouzellaguen that year.", "kab": "Ilul deg Uzellagen deg useggas-nni."}}
{"translation": {"en": "I have been living in Tasga for a long time.", "kab": "Zzedɣeɣ deg Tesga acḥal aya."}}

Key Linguistic Features

Standard machine translation pipelines typically fail when working with Kabyle due to its intense morphosyntactic variations. This dataset addresses those limitations explicitly by encoding three core Amazigh grammar laws:

1. Prepositional Phonetic Agreement

The engine dynamically checks the initial phoneme of the target toponym to apply correct prefix transitions, preventing raw vowel clashes:

• —Consonant Environment: Maps to di (in) and si (from) → di Qsemṭina, si Wehran.
• —Vowel Environment: Maps to deg (in) and seg (from) → deg Yin Ṣaleḥ, deg Uzellagen.

2. (The Annexed State / L'état d'orientation)

Kabyle nouns mutate their structural boundaries depending on what precedes them. The dataset pipeline accurately alters toponym states on the fly:

• —Feminine Mutation ($Ta- \rightarrow$ $Te-$): Tasga cleanly yields deg Tesga.
• —Masculine Mutation ($A- \rightarrow$ $U-$): Akeffadu cleanly yields deg Ukeffadu.

3. Orthographic Normalization (v) to (b)

To align with standardized modern literary, academic and conventions, all instances of the spirantized soft v sound (e.g., Vgayet) are fully normalized to `b/B` (e.g., Bgayet), preserving consistent tokenization distribution.

How to Load and Use

You can load this dataset instantly into your training environment using the Hugging Face datasets library:

from datasets import load_dataset

# Load from local JSONL file
dataset = load_dataset("json", data_files="en_kab_parallel_corpus.jsonl", split="train")

# Train/Validation Split
dataset_splits = dataset.train_test_split(test_size=0.1, seed=42)
print(dataset_splits)

Intended Use & Fine-Tuning Target

This corpus is optimized specifically to address low-resource performance gaps in encoder-decoder networks. It is primarily built for fine-tuning:

• —NLLB-200 (facebook/nllb-200-distilled-600M or facebook/nllb-200-1.3B) using source asset code eng_Latn and target asset code kab_Latn.
• —AfroXLMR or mT5 architectures adjusted for Tamazight variant mapping.

Acknowledgements

• —Toponym Source Data: Curated via OpenStreetMap extractions in boffire/kabyle-toponyms.
• —Linguistic Blueprint: Tailored according to established literary Kabyle grammar frameworks.