Datasetpublic

SultanR/fineweb-edu-arabic

fineweb-edu-arabic Arabic translation of FineWeb-Edu (sample/350BT subset, filtered to language_score > 0.9), translated with Seed-X-PPO-7B using greedy decoding. Documents were split into ~490-token chunks, translated, and reassembled. Each row is one complete document. A companion corpus translated with the same pipeline is available at dclm-pro-arabic. Details Documents: 82,840,410 (27.9% of the source subset, uniformly sampled) Arabic tokens: ~170B (Seed-X… See the full description on the dataset page: https://huggingface.co/datasets/SultanR/fineweb-edu-arabic.

sourceHugging Faceupdated 1mo agoView on Hugging Face

1likes1.4kdownloads

Dataset Card

fineweb-edu-arabic

Arabic translation of FineWeb-Edu (sample/350BT subset, filtered to language_score > 0.9), translated with Seed-X-PPO-7B using greedy decoding. Documents were split into ~490-token chunks, translated, and reassembled. Each row is one complete document. A companion corpus translated with the same pipeline is available at dclm-pro-arabic.

Details

—Documents: 82,840,410 (27.9% of the source subset, uniformly sampled)
—Arabic tokens: ~170B (Seed-X tokenizer)
—Shards: 867, zstd compressed, 153 GiB

Column	Type	Description
`doc_id`	int64	Document index (not globally unique, see limitations)
`text`	string	Full Arabic translation

Results

We ran continued pretraining ablations with a 1.46B LLaMA model: 10B tokens on AraMix-HQ, then 20B tokens on the listed mix (30B total). We follow the FineTasks evaluation format: scores are rescaled against a random baseline, then macro averaged over general knowledge (GK), reading comprehension (RC), reasoning (RES), and NLU. FWE is fineweb-edu-arabic, DCLM is dclm-pro-arabic.

Mix	GK	RC	RES	NLU	Aggregate
25% AraMix + 75% FWE	0.1453	0.2006	0.1283	0.2417	0.1790
25% AraMix + 75% DCLM	0.1569	0.1899	0.1171	0.2479	0.1779
33% AraMix + 33% DCLM + 33% FWE	0.1565	0.1892	0.1091	0.2362	0.1727
75% DCLM + 25% FWE	0.1494	0.1719	0.1113	0.2580	0.1726
100% DCLM	0.1412	0.1877	0.1090	0.2492	0.1718
50% AraMix + 50% DCLM	0.1496	0.1868	0.1133	0.2371	0.1717
50% AraMix + 50% FWE	0.1475	0.1727	0.1176	0.2316	0.1673
50% DCLM + 50% FWE	0.1440	0.1752	0.1033	0.2450	0.1669
100% FWE	0.1426	0.1768	0.0993	0.2467	0.1664
100% AraMix-HQ (baseline)	0.1282	0.1890	0.1048	0.2375	0.1649
75% AraMix + 25% DCLM	0.1415	0.1794	0.1092	0.2267	0.1642
25% DCLM + 75% FWE	0.1397	0.1707	0.0962	0.2456	0.1631
75% AraMix + 25% FWE	0.1371	0.1811	0.1080	0.2259	0.1630

25% AraMix + 75% fineweb-edu-arabic was the best result across all 24 runs in the ablation, which also covered Arabic Wikipedia, research, OCR, and math corpora. It also has the best reasoning score of any run. Small fractions of translated data underperform the baseline; it pays off when it dominates the mix.

Usage

python

from datasets import load_dataset

ds = load_dataset("SultanR/fineweb-edu-arabic", split="train", streaming=True)

Limitations

—doc_id restarts at 0 for each of the 20 split workers, so values repeat across unrelated documents. Do not use it as a join or dedup key.
—The decode cap truncated ~4% of chunks, so some documents are missing tail content.
—No automatic quality filtering was applied to the translations.

License

Refer to the original FineWeb-Edu dataset for license information.