datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
arabic-books
Arabic Books
Dataset Summary
The arabic-books dataset contains 8,500 rows of text, each representing the full text of a single Arabic book. These texts were extracted using the arabic-large-nougat model, showcasing the model’s capabilities in Arabic OCR and text extraction. The dataset spans a total of 1.1 billion tokens, calculated using the GPT-4 tokenizer.
This dataset is a testimony to the quality of the Arabic Nougat models and their effectiveness in extracting… See the full description on the dataset page: https://huggingface.co/datasets/MohamedRashad/arabic-books.Mixed-Arabic-Datasets-Repo
Dataset Card for "Mixed Arabic Datasets (MAD) Corpus"
The Mixed Arabic Datasets Corpus : A Community-Driven Collection of Diverse Arabic Texts
Dataset Description
The Mixed Arabic Datasets (MAD) presents a dynamic compilation of diverse Arabic texts sourced from various online platforms and datasets. It addresses a critical challenge faced by researchers, linguists, and language enthusiasts: the fragmentation of Arabic language datasets across the Internet. With MAD, we… See the full description on the dataset page: https://huggingface.co/datasets/M-A-D/Mixed-Arabic-Datasets-Repo.101_billion_arabic_words_dataset
101 Billion Arabic Words Dataset
Updates
Maintenance Status: Actively Maintained
Update Frequency: Weekly updates to refine data quality and expand coverage.
Upcoming Version
More Cleaned Version: A more cleaned version of the dataset is in processing, which includes the addition of a UUID column for better data traceability and management.
Dataset Details
The 101 Billion Arabic Words Dataset is curated by the Clusterlab team and consists of 101… See the full description on the dataset page: https://huggingface.co/datasets/ClusterlabAi/101_billion_arabic_words_dataset.AraMix-HQ
AraMix family: AraMix (minhash and matched) | AraMix-domain-classified (with domain labels) | AraMix-HQ (model-filtered)
AraMix-HQ is a high-quality subset of AraMix-MinHash created using model-based quality scoring. We adapt the approach from FineWeb2-HQ but replace the XLM-Roberta encoder with mmBERT, which provides better Arabic language understanding. We release the model at AdaMLLab/mmBERT-Arabic-Quality-Classifier.
AraMix-HQ outperforms both AraMix-Matched and FineWeb2-HQ… See the full description on the dataset page: https://huggingface.co/datasets/AdaMLLab/AraMix-HQ.fineweb-edu-arabic
fineweb-edu-arabic
Arabic translation of FineWeb-Edu (sample/350BT subset, filtered to language_score > 0.9), translated with Seed-X-PPO-7B using greedy decoding. Documents were split into ~490-token chunks, translated, and reassembled. Each row is one complete document. A companion corpus translated with the same pipeline is available at dclm-pro-arabic.
Details
Documents: 82,840,410 (27.9% of the source subset, uniformly sampled)
Arabic tokens: ~170B (Seed-X… See the full description on the dataset page: https://huggingface.co/datasets/SultanR/fineweb-edu-arabic.AraMix
AraMix family: AraMix (minhash and matched) | AraMix-domain-classified (with domain labels) | AraMix-HQ (model-filtered)
AraMix (https://arxiv.org/abs/2512.18834) is an Arabic pretraining corpus containing 178 billion tokens across 179 million documents (in the minhash subset). Rather than scraping the web again, AraMix combines seven publicly available Arabic datasets, applies Arabic-specific quality filtering, and performs cross-dataset deduplication.We train a 1.4B parameter… See the full description on the dataset page: https://huggingface.co/datasets/AdaMLLab/AraMix.AraMix-domain-classified
AraMix Domain-Classified
AraMix family: AraMix (minhash and matched) | AraMix-domain-classified (with domain labels) | AraMix-HQ (model-filtered)
This is AraMix with per-document domain labels from nvidia/multilingual-domain-classifier.
Usage
from datasets import load_dataset
ds = load_dataset("AdaMLLab/AraMix-domain-classified", "minhash_deduped")
ds = load_dataset("AdaMLLab/AraMix-domain-classified", "sentence_deduped")
Schema
Field… See the full description on the dataset page: https://huggingface.co/datasets/AdaMLLab/AraMix-domain-classified.Japanese-Creative-Writing-39.6k
Japanese-Creative-Writing-39.6k
概要
deepseek-ai/DeepSeek-V3-0324を用いて作成した、約39600件の日本語の小説執筆タスクデータセットです。
全てのデータは2ターンのデータとなっています。また、データセット中の一部データはNSFW表現を含みます。
データの詳細
各データは以下のキーを含みます。
messages: OpenAI messages形式の対話データ
instruction_1: 1ターン目の指示プロンプト
output_1: 1ターン目のアシスタント応答
instruction_2: 2ターン目の指示プロンプト
output_2: 2ターン目のアシスタント応答
1ターン目の指示プロンプトはdeepseek-ai/DeepSeek-V3-0324で合成されています。system promptや2ターン目の指示プロンプトは事前に用意した複数種類からランダムに選択されたものが設定されています。
ライセンス… See the full description on the dataset page: https://huggingface.co/datasets/Aratako/Japanese-Creative-Writing-39.6k.smolkalam-arabic-conversational-sft
SmolKalam
SmolKalam is a quality-filtered Arabic SFT dataset of 1,790,478 examples (~2.45B tokens), built as an ensemble translation of SmolTalk2. It covers multi-turn dialogue (23% of rows), reasoning traces (19% carry <think>), tool and function calling (4.4%), and long context, categories that are underrepresented in existing Arabic post-training data. The SmolTalk2 source mixtures are kept as subsets.
Released with the paper SmolKalam: Ensemble Quality-Filtered Translation… See the full description on the dataset page: https://huggingface.co/datasets/AdaMLLab/smolkalam-arabic-conversational-sft.dclm-pro-arabic
dclm-pro-arabic
Arabic translation of DCLM-Pro (global shards 01 and 05), translated with Seed-X-PPO-7B using greedy decoding. Documents were split into ~490-token chunks at sentence boundaries, translated, and reassembled. Each row is one complete document. A companion corpus translated with the same pipeline is available at fineweb-edu-arabic.
Details
Documents: 33,245,503 (22.7% of the two source shards, uniformly sampled)
Arabic tokens: ~93B (Seed-X… See the full description on the dataset page: https://huggingface.co/datasets/SultanR/dclm-pro-arabic.ArabicText-Large
ArabicText-Large: High-Quality Arabic Corpus for LLM Training
Dataset Summary
ArabicText-Large is a comprehensive, high-quality Arabic text corpus comprising 743,288 articles with over 244 million words, specifically curated for Large Language Model (LLM) training and fine-tuning. This dataset represents one of the largest publicly available Arabic text collections for machine learning research.
This corpus addresses the critical shortage of high-quality Arabic NLP… See the full description on the dataset page: https://huggingface.co/datasets/Jr23xd23/ArabicText-Large.iraqi-arabic-sales-dialogue-dataset
Iraqi Arabic Sales Dialogue Dataset
A large synthetic dataset of Iraqi (Baghdadi-based) Arabic dialogue, centered on
retail sales, haggling, and everyday conversation.
النسخة العربية متوفرة بالكامل بالأسفل — Arabic version available in full below.
What this is
210,832 template-generated conversations, of which 171,601 (81%) are exact-unique
message sequences, spanning 20 topical categories in colloquial Iraqi Arabic. The
core of the dataset (10 categories) is… See the full description on the dataset page: https://huggingface.co/datasets/ameer4wisam/iraqi-arabic-sales-dialogue-dataset.arab-dialects-20-countries-3m
Arab Dialects Dataset - 20 Countries
A large-scale Arabic dialects dataset covering 20 Arab countries, 7 content types per country, 3,000,000 records, 140 JSONL files, 12.07 GB. UTF-8 JSONL, ready for Hugging Face Datasets.
1. Contents
1. Contents
2. Dataset Summary
3. Repository Map
4. Countries Table (20 folders)
5. Data Types Table (7 files)
6. Record Schema
7. Loading and Usage
8. Generation and Reproduction
9. Considerations and Limitations
10. Contributors… See the full description on the dataset page: https://huggingface.co/datasets/ISLAM-PO/arab-dialects-20-countries-3m.documents-Egyptian-Arabic
Egyptian Arabic Mega Corpus (EAMC) — 25M Unified Egyptian Dialect Dataset
The Largest Unified Open Corpus for Egyptian Arabic (Masri / arz)
25.5M Samples | 2.66 GB (Parquet) | 9 Configs | Apache 2.0 | Ready-to-train
Comprehensive coverage: Raw Text · Wikipedia · Conversations · Speech (Whisper) · Parallel Translation (EN↔EGY) · Trilingual QA · Wikipedia Quality Classification · Fake Review / Spam Detection
Dataset Summary
Egyptian Arabic Mega Corpus… See the full description on the dataset page: https://huggingface.co/datasets/ISLAM-PO/documents-Egyptian-Arabic.FineWeb-Edu-Arabic-24M
English
العربية
FineWeb-Edu Arabic 24M
An Arabic-only pretraining corpus of 24,794,425 complete documents, translated from the sample-350BT configuration of FineWeb-Edu. It contains 34.86 billion Arabic tokenizer tokens and preserves the original FineWeb-Edu document IDs, source scores, and detailed translation diagnostics.
Highlight
Saudi architecture shaped by place. A well-translated tour of how builders in Najd, the Gulf coast, Hejaz, and Asir adapted local… See the full description on the dataset page: https://huggingface.co/datasets/nizarun/FineWeb-Edu-Arabic-24M.Arabic-VLM-Full-Pearl
💎 The Arabic VLM Dataset (Full Pearl Edition)
This repository contains the full, unreviewed dataset comprising 309K multimodal examples. This data was generated automatically using the agentic pipeline developed for the Pearl project, as described in our paper.
Disclaimer: This is the raw, synthetic data that has not been subject to human review. It was generated as part of the data creation process and is released for research purposes. It may contain noise, errors, or… See the full description on the dataset page: https://huggingface.co/datasets/MohamedRashad/Arabic-VLM-Full-Pearl.ArabicWeb24
📚 ArabicWeb24
More than 39 billion tokens of high quality Arabic web content 🌐.
What is ArabicWeb24 ?
The ArabicWeb24 dataset consists of more than 28 billion tokens of cleaned and deduplicated Arabic web data from a customized crawl.
This was processed using the large scale data processing library datatrove.
What is being released ?
We are releasing two datasets versions:
ArabicWeb24: dataset version 1 (v1) underwent extensive processing through… See the full description on the dataset page: https://huggingface.co/datasets/lightonai/ArabicWeb24.Shifaa_Arabic_Medical_Consultations
Shifaa Arabic Medical Consultations 🏥📊
Overview 🌍
Shifaa is revolutionizing Arabic medical AI by addressing the critical gap in Arabic medical datasets. Our first contribution is the Shifaa Arabic Medical Consultations dataset, a comprehensive collection of 84,422 real-world medical consultations covering 16 Main Specializations and 585 Hierarchical Diagnoses.
🔍 Why is this dataset important?
First large-scale Arabic medical dataset for AI applications.… See the full description on the dataset page: https://huggingface.co/datasets/Ahmed-Selem/Shifaa_Arabic_Medical_Consultations.ArabCulture
ArabCulture 🇦🇪🇵🇸🇪🇬🇸🇦🇾🇪🇯🇴🇱🇧🇸🇾🇸🇩🇲🇦🇩🇿🇹🇳🇱🇾
Abdelrahman Sadallah and Junior Cedric Tonga and Khalid Almubarak and Saeed Almheiri and Farah Atif and Cahtrine Qwaider and Karima Kadaoui and Sara Shatnawi and Yaser Alesh and Fajri Koto
MBZUAI, SDAIA, Al-Balqa Applied University, Khalifa University
ArabCulture is a culturally grounded commonsense reasoning dataset in Modern Standard Arabic (MSA), covering 13 Arab countries across the… See the full description on the dataset page: https://huggingface.co/datasets/MBZUAI/ArabCulture.arabic-to-code-8-langs-3m
Arabic-to-Code Dataset - 8 Languages, 3M Records
Train your own Arabic-speaking code model: 3,000,000 Arabic instruction-to-code pairs, 8 programming languages, 40 JSONL files, 11.55 GB. Every record has an Arabic instruction, complete runnable code with Arabic comments, and an Arabic explanation. Ready for Hugging Face Datasets.
1. Contents
2. Dataset Summary
3. Repository Map
4. Languages Table
5. Categories Table
6. Record Schema
7. Loading and Training Usage… See the full description on the dataset page: https://huggingface.co/datasets/ISLAM-PO/arabic-to-code-8-langs-3m.101_billion_arabic_words_dataset
101 Billion Arabic Words Dataset
Updates
Maintenance Status: Actively Maintained
Update Frequency: Weekly updates to refine data quality and expand coverage.
Upcoming Version
More Cleaned Version: A more cleaned version of the dataset is in processing, which includes the addition of a UUID column for better data traceability and management.
Dataset Details
The 101 Billion Arabic Words Dataset is curated by the Clusterlab team and consists of 101… See the full description on the dataset page: https://huggingface.co/datasets/muhammadrizo5721/101_billion_arabic_words_dataset.arabic-billion-words
Arabic Billion Words Dataset 🌕
The Abu El-Khair Arabic News Corpus (arabic-billion-words) is a comprehensive collection of Arabic text, encompassing over five million newspaper articles. The corpus is rich in linguistic diversity, containing more than a billion and a half words, with approximately three million unique words. The text is encoded in two formats: UTF-8 and Windows CP-1256, and marked up using two markup languages: SGML and XML.
Data Example
An example… See the full description on the dataset page: https://huggingface.co/datasets/MohamedRashad/arabic-billion-words.Arabic-news-daily
Arabic News Daily 🗞️
A daily-updated, multi-domain Arabic news dataset collected automatically from 15 curated sources.
Unlike other Arabic datasets that are static snapshots, this dataset grows every day — making it ideal for research requiring fresh, current Arabic text across diverse domains.
Sources
Source
Domain
Variety
Al Jazeera Arabic
Politics
MSA
BBC Arabic
Politics
MSA
RT Arabic
Politics
MSA
Al Arabiya
Politics
MSA
AITNews
Tech & AI… See the full description on the dataset page: https://huggingface.co/datasets/unohamza/Arabic-news-daily.Mixed-Arabic-Datasets-Repo
Dataset Card for "Mixed Arabic Datasets (MAD) Corpus"
The Mixed Arabic Datasets Corpus : A Community-Driven Collection of Diverse Arabic Texts
Dataset Description
The Mixed Arabic Datasets (MAD) presents a dynamic compilation of diverse Arabic texts sourced from various online platforms and datasets. It addresses a critical challenge faced by researchers, linguists, and language enthusiasts: the fragmentation of Arabic language datasets across the Internet. With… See the full description on the dataset page: https://huggingface.co/datasets/yrrhall/Mixed-Arabic-Datasets-Repo.aracast_textarabic_billion_wordsAbu El-Khair Corpus is an Arabic text corpus, that includes more than five million newspaper articles.
It contains over a billion and a half words in total, out of which, there are about three million unique words.
The corpus is encoded with two types of encoding, namely: UTF-8, and Windows CP-1256.
Also it was marked with two mark-up languages, namely: SGML, and XML.ArabCulture-Dialogue
ArabCulture-Dialogue: Cultural Benchmarking of LLMs in MSA and Arabic Dialectal Dialogue
📄 Paper (ACL 2026) | 🤗 Dataset
ArabCulture-Dialogue is the first parallel MSA–dialect cultural dialogue dataset, covering 13 Arabic-speaking countries in both Modern Standard Arabic (MSA) and each country's respective dialect, spanning 12 daily-life topics and 54 fine-grained subtopics. It contains 3,471 parallel dialogue pairs (6,942 dialogues, 343,804 words in total), each consisting… See the full description on the dataset page: https://huggingface.co/datasets/Almheiri/ArabCulture-Dialogue.Synthetic-JP-EN-Coding-Dataset-801k
Synthetic-JP-EN-Coding-Dataset-801k
Magpieによって作成したコードSFTデータセットであるAratako/Synthetic-JP-EN-Coding-Dataset-Magpie-69kを元に、Evol-Instructのような手法を用いて複数のinstructionとresonseを生成し拡張して作成した、日英混合801262件のコードSFT用合成データセットです。
日本語: 173849件
英語: 627413件
元のinstructionの作成に利用したモデルは以下の通りです。modelキーに該当レコードの作成に利用したモデル情報があります。
nvidia/Nemotron-4-340B-Instruct
microsoft/Phi-3-medium-4k-instruct
mistralai/Mixtral-8x22B-Instruct-v0.1… See the full description on the dataset page: https://huggingface.co/datasets/Aratako/Synthetic-JP-EN-Coding-Dataset-801k.arabic-dialect-corpus
Arabic Dialect Corpus
A comprehensive collection of Arabic dialectal text, standardized for Natural Language Processing (NLP) model training, evaluation, and linguistic analysis. This corpus has been meticulously processed to ensure high-quality tokenization and consistent metadata.
Dataset Statistics
Metric
Value
Total Records
127,180
Total Tokens
5,802,324
Average Tokens per Record
45.62
Dialect Categories
5
Changelog… See the full description on the dataset page: https://huggingface.co/datasets/dataflare/arabic-dialect-corpus.arabic_tashkil_dataset
Arabic Tashkil (Diacritization) Dataset 📖✨
Dataset Summary
This is a massive, high-quality, Gold-Standard dataset designed explicitly for training Arabic Automatic Diacritization (Tashkil) AI models (such as ByT5, AraT5, or Custom Transformers).
The dataset contains 1,494,228 heavily vocalized pages (~2.47 GB of data) extracted from Classical Arabic and Islamic texts sourced from Thahabi.org.
To ensure the highest possible ground-truth quality, every single page… See the full description on the dataset page: https://huggingface.co/datasets/freococo/arabic_tashkil_dataset.