riotu-lab/tashkeel-arabic-sentences

This dataset contains Arabic sentences extracted from the ImruQays/Alukah-Arabic dataset. Sentences were filtered based on their 'tashkeel' (Arabic diacritics) ratio, with a minimum ratio of 0.3 (adjustable during extraction).

Source: The original articles were sourced from the ImruQays/Alukah-Arabic dataset on Hugging Face.

Processing:

1. 1.Articles were loaded from ImruQays/Alukah-Arabic.
2. 2.Each article was split into individual sentences using a regex pattern.
3. 3.For each sentence, the ratio of tashkeel characters to total characters was calculated.
4. 4.Sentences with a tashkeel ratio greater than or equal to 0.3 were selected.

This dataset is intended for tasks requiring heavily diacritized Arabic text, such as text-to-speech, diacritization models, or linguistic analysis.