Syrinesmati/tunisian-dialect-corpus

Dataset Card for Tunisian Dialect Corpus (Cleaned Arabic-Only)

1. Dataset Overview

1.1 Description

This dataset is a cleaned corpus of Tunisian Arabic dialect text, aggregated from multiple public sources on Hugging Face. It is designed for Continual Pretraining (CPT) and general NLP research.

A dedicated preprocessing pipeline was applied to:

• —Normalize text
• —Remove noise and artifacts
• —Filter non-Arabic content
• —Ensure higher overall data quality

The final dataset focuses on Arabic-script Tunisian dialect (ar-TN) with reduced noise and improved consistency.

1.2 Key Information

2. Dataset Sources

This dataset is a merged and cleaned derivative of the following public resources:

3. Intended Uses

3.1 Direct Use

This dataset is suitable for:

• —Continual pretraining of Arabic or multilingual LLMs
• —Domain adaptation for Tunisian dialect
• —Text generation and understanding in Tunisian Arabic
• —Corpus and linguistic analysis
• —Data preparation for downstream NLP tasks

3.2 Out-of-Scope Use

This dataset is not intended for:

• —High-stakes decision-making systems
• —Surveillance or identity profiling
• —Medical, legal, or financial applications without safeguards
• —Claims of full representativeness of Tunisian dialects

4. Dataset Statistics

5. Dataset Structure

The dataset is distributed in Parquet format and contains:

• —text: Cleaned Tunisian Arabic text

Optional metadata fields may be included depending on preprocessing stages.

6. Dataset Creation

The dataset is distributed in Parquet format and contains:

• —text: Cleaned Tunisian Arabic text

Optional metadata fields may be included depending on preprocessing stages.

6. Dataset Creation

6.1 Motivation

Tunisian Arabic is significantly underrepresented in open NLP resources. This dataset aims to:

• —Provide a large-scale, clean corpus
• —Enable reproducible research
• —Improve dialectal Arabic model performance

6.2 Data Collection

The dataset is built by:

• —Aggregating multiple public datasets
• —Merging and standardizing formats
• —Removing redundancy and inconsistencies

6.3 Processing Pipeline

The cleaning pipeline includes:

• —Text normalization (whitespace, formatting)
• —Removal of noisy artifacts (HTML tags, social media patterns)
• —Emoji and symbol cleanup
• —Filtering non-Arabic or mixed-script text
• —Optional removal of digits and hashtags
• —Duplicate and near-duplicate removal
• —Minimum length and word count filtering
• —Arabic-only filtering (final dataset)
• —Data shuffling before export

6.4 Source Data Producers

Original data originates from contributors and maintainers of the upstream datasets. This dataset is a cleaned and merged derivative version.

6.5 Personal and Sensitive Information

As with many web-based corpora:

• —Some personal or sensitive information may remain
• —No guarantee of full anonymization is provided

Users should apply additional filtering if required for sensitive applications.

7. Bias, Risks, and Limitations

• —Source Bias: Reflects biases of original datasets and platforms
• —Language Bias: Focus on Arabic script reduces code-switching diversity
• —Coverage Limitation: Does not fully represent all Tunisian regions or sociolects

8. Citation

8.1 This Dataset

BibTeX:

@dataset{tunisian_dialect_corpus_cleaned_2026,
  title     = {Tunisian Dialect Corpus (Cleaned Arabic-Only)},
  author    = {Syrinesmati},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/Syrinesmati/tunisian-dialect-corpus}
}

APA: Syrinesmati. (2026). Tunisian Dialect Corpus (Cleaned Arabic-Only) [Dataset]. Hugging Face. https://huggingface.co/datasets/Syrinesmati/tunisian-dialect-corpus

8.2 Citations for Included Datasets

Tunisian Derja Dataset

@dataset{linagora2025LLM-tn,
  author = {Wajdi Ghezaiel and Jean-Pierre Lorré},
  title  = {Tunisian Derja Dataset},
  year   = {2025},
  month  = {January},
  url    = {https://huggingface.co/datasets/linagora/Tunisian_Derja_Dataset}
}

Tunisian-English Dialectic Derja

@dataset{Tunisian_English_dialectic_Derja,
  author = {Khaled Bouzaiene},
  title  = {Tunisian-English Dialectic Derja Dataset},
  year   = {2024},
  url    = {https://huggingface.co/datasets/khaled123/Tunisian_English_dialectic_Derja}
}

TunSwitch

@misc{abdallah2023leveraging,
  title         = {Leveraging Data Collection and Unsupervised Learning for Code-switched Tunisian Arabic Automatic Speech Recognition},
  author        = {Ahmed Amine Ben Abdallah and Ata Kabboudi and Amir Kanoun and Salah Zaiem},
  year          = {2023},
  eprint        = {2309.11327},
  archivePrefix = {arXiv},
  primaryClass  = {eess.AS}
}

LinTO Textual Dataset (Tunisian Arabic)

@misc{linagora2024Linto-tn,
  author       = {Hedi Naouara and Jérôme Louradour and Jean-Pierre Lorré},
  title        = {LinTO Audio and Textual Datasets to Train and Evaluate Automatic Speech Recognition in Tunisian Arabic Dialect},
  year         = {2024},
  month        = {October},
  note         = {Good Data Workshop, AAAI 2025},
  howpublished = {\url{https://huggingface.co/linagora/linto-asr-ar-tn-0.1}}
}

Arbi-Houssem STT/TTS Dataset

@dataset{arbi_houssem_stt_tts,
  author = {Arbi Houssem},
  title  = {Tunisian Dataset STT-TTS 15s Filtered Organised Mixed},
  url    = {https://huggingface.co/datasets/Arbi-Houssem/Tunisian_dataset_STT-TTS15s_filtred_organiser_Mixed}
}

LinTO ASR — Tunisian Arabic (linto-asr-ar-tn-0.1)

@misc{linagora2024linto_asr,
  author       = {Hedi Naouara and Jérôme Louradour and Jean-Pierre Lorré},
  title        = {LinTO ASR Tunisian Arabic Dialect Dataset},
  year         = {2024},
  howpublished = {\url{https://huggingface.co/datasets/linagora/linto-asr-ar-tn-0.1}}
}

Tunisian Arabic Dialects Identification (TADI)

@dataset{tadi,
  title = {Tunisian Arabic Dialects Identification (TADI)},
  note  = {Binary classification dataset for Tunisian vs. non-Tunisian Arabic dialect identification}
}

Tunisian Algerian Dialect (TAD) — TunBERT

@misc{tunbert_tad,
  author       = {InstaDeep},
  title        = {Tunisian Algerian Dialect Dataset},
  howpublished = {\url{https://github.com/instadeepai/tunbert}}
}

Tunninjaar — derja.ninja

@dataset{tunninjaar,
  author = {Khaled Bouzaiene},
  title  = {Tunninjaar},
  url    = {https://huggingface.co/datasets/khaled123/tunninjaar}
}

T-HSAB — Tunisian Hate Speech and Abusive Dataset

@inproceedings{mulki2019tsab,
  author    = {Hala Mulki and Hatem Haddad and Chedi Bechikh Ali and Halima Alshabani},
  title     = {T-HSAB: A Tunisian Hate Speech and Abusive Language Dataset},
  booktitle = {Proceedings of the 7th International Conference on Arabic Language Processing},
  year      = {2019}
}

Tunisian Reading Comprehension Dataset

@dataset{tunisian_rc,
  title = {Tunisian Reading Comprehension Dataset},
  note  = {Question-Answering dataset based on the Tunisian constitution; 144 documents, 3 paragraphs each, 3 QA pairs per paragraph}
}

Naim Mhedhbi — Tunisian Dialect Corpus v0

@dataset{mhedhbi_tunisian_v0,
  author = {Naim Mhedhbi},
  title  = {Tunisian Dialect Corpus v0},
  note   = {~40,000 Facebook comments and posts labeled for sentiment}
}

TSAC — Tunisian Sentiment Analysis Corpus

@dataset{tsac,
  title        = {Tunisian Sentiment Analysis Corpus (TSAC)},
  note         = {~17,000 Facebook comments in Tunisian dialect},
  howpublished = {\url{https://paperswithcode.com/dataset/tsac}}
}

khaled123/Testtun

@dataset{testtun,
  author = {Khaled Bouzaiene},
  title  = {Testtun},
  url    = {https://huggingface.co/datasets/khaled123/Testtun}
}

khaled123/Tuniset

@dataset{tuniset,
  author = {Khaled Bouzaiene},
  title  = {Tuniset},
  url    = {https://huggingface.co/datasets/khaled123/Tuniset}
}

9. Dataset Card Authors

Syrinesmati

10. Contact

For questions, issues, or updates, please use the Hugging Face dataset repository discussion/issues page.