Zauberman/noisy_zarma

Zarma Noisy Dataset

Overview

The Zarma Noisy Dataset is a collection of Zarma sentences with artificially introduced noise to simulate human-like errors. This dataset is designed for tasks such as grammatical error correction (GEC), text denoising, and robustness testing in natural language processing (NLP) for low-resource languages like Zarma. It is derived from a clean monolingual Zarma dataset (monolingual_zarma.jsonl) by applying various types of noise, including character-level and word-level modifications.

Dataset Structure

The dataset is stored in a JSONL file (noisy/zarma_noisy_dataset.jsonl) where each line is a JSON object with the following fields:

• —`original`: The raw input sentence as it appears in the source dataset, preserving its exact form.
• —`cleaned`: A normalized version of the sentence (Unicode NFC normalization, extra spaces removed).
• —`char_swap`: The sentence with adjacent character swaps (e.g., "teh" → "the") within words to mimic typos.
• —`random_char_insertion`: The sentence with up to 2 random character insertions, preferring vowels near vowels for realism.
• —`char_delete`: The sentence with character deletions, avoiding critical positions (first/last in words).
• —`char_substitute`: The sentence with character substitutions, using similar-looking or keyboard-adjacent characters (e.g., 'a' → 's').
• —`word_masking`: The sentence with words replaced by a BLANK token, preferring content words (length > 3).
• —`word_swap`: The sentence wit adjacent word swaps (e.g., "is it" → "it is").

Example Entry

{
    "original": "Yesu Kirisita Tuura Wema TUURA WEMA",
    "cleaned": "Yesu Kirisita Tuura Wema TUURA WEMA",
    "char_swap": "Yseu Kirisita Tuura Wema TUURA WEMA",
    "random_char_insertion": "Yesu Kirisita Tuura Wema TUURA aWEMA",
    "char_delete": "Yesu Kirista Tuura Wema TURA WEMA",
    "char_substitute": "Yesu Kirisita Tuura Wema TUURA WEMs",
    "word_masking": "Yesu Kirisita BLANK Wema TUURA WEMA",
    "word_swap": "Yesu Kirisita Wema Tuura TUURA WEMA"
}

Citation

If you use this dataset in your research, please cite the following paper: code Bibtex

@misc{keita2025grammaticalerrorcorrectionlowresource,
    title={Grammatical Error Correction for Low-Resource Languages: The Case of Zarma},
    author={Mamadou K. Keita and Christopher Homan and Marcos Zampieri and Adwoa Bremang and Habibatou Abdoulaye Alfari and Elysabhete Amadou Ibrahim and Dennis Owusu},
    year={2025},
    eprint={2410.15539},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2410.15539},
}