Jamalianpour/persian-wikipedia-instruct

Persian Wikipedia Instruct

A Persian-language instruction-tuning dataset of ~120,000 samples, generated from the Persian Wikipedia (fawiki) article dump. Each article was cleaned to Markdown, chunked by section, and passed to a locally-run Gemma4 model that produced grounded instruction/response pairs across five task types. The result is ready for supervised fine-tuning (SFT) of Persian LLMs.

Heads-up: this is synthetic data. The instructions and answers were written by an LLM from Wikipedia text, then automatically filtered — not human-verified.

Dataset at a glance

Usage

from datasets import load_dataset

ds = load_dataset("Jamalianpour/persian-wikipedia-instruct", split="train")
print(ds[0])

Data fields

Example

{
  "task_type": "qa",
  "instruction": "دیگو مارادونا که بود؟",
  "input": "",
  "output": "یک فوتبالیست افسانه‌ای اهل آرژانتین که از بزرگ‌ترین بازیکنان تاریخ فوتبال به شمار می‌رود.",
  "source_id": 1234,
  "source_title": "دیگو مارادونا",
  "source_url": "https://fa.wikipedia.org/wiki/دیگو_مارادونا",
  "section": "مقدمه"
}

How it was built

1. 1.Corpus extraction. The fawiki pages-articles dump was streamed and each main-namespace, non-redirect article was converted from wikitext to clean Markdown (headings, lists, and tables preserved; infoboxes, references, categories, and back-matter sections such as «منابع» and «جستارهای وابسته» removed). Stub articles were dropped.
2. 2.Sample generation. Each article was split into section-sized chunks and passed to a Gemma4 model running locally via Ollama, prompted to produce diverse, self-contained instruction samples grounded only in the chunk text.
3. 3.Quality filtering. Generated samples were filtered to remove non-Persian or garbled output, source-leaking phrasing (e.g. «بر اساس متن»), near-empty answers, and duplicate instructions.

Task type distribution

Licensing and attribution

The source text is from Persian Wikipedia, licensed under [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/). Because this dataset is derived from that text, it is released under the same license (CC BY-SA 4.0). If you use or build on this dataset:

• —Attribute Wikipedia / the Wikimedia Foundation as the source of the underlying text.
• —Share alike — derivative datasets must also be CC BY-SA 4.0.

Limitations

• —Synthetic and unverified. Samples are model-generated and only automatically filtered. Some answers may contain factual errors or hallucinations not present in the source.
• —Inherited bias. Coverage and viewpoint follow Persian Wikipedia, including its topical and demographic gaps.
• —Not for factual authority. Suitable for teaching instruction-following and Persian fluency; not a reliable knowledge base on its own.

Citation

If you use this dataset, please cite it and the underlying source:

@misc{persian_wikipedia_instruct,
  title        = {Persian Wikipedia Instruct},
  author       = {Jamalianpour},
  year         = {2026},
  howpublished = {https://huggingface.co/datasets/Jamalianpour/persian-wikipedia-instruct},
}