KFUPM-JRCAI/star-dataset-instructions

STAR Instructions

STAR Instructions is a large-scale Arabic instruction-tuning dataset built by rendering the 355 STAR Jinja2 prompt templates against their 87 source datasets, covering 27 raw task labels (20 tasks after merging closely related categories, as reported in the paper). The underlying templates were authored collaboratively using PromptLab. This dataset and the experiments built on it are described in STAR: instruction tuning for Arabic across tasks, datasets, and models.

For the raw templates and experiment design details, see the companion dataset: STAR Templates.

📦 Code: the tuning and evaluation pipelines built on this dataset is accessible at github.com/KFUPM-JRCAI/star-instructions-tuning.

Overview

Each record in this dataset is created by rendering a Jinja2 prompt template against a real data sample, then splitting the result on ||| to separate the instruction input from the expected output. The dataset preserves both the rendered instruction and the original template metadata.

Tasks covered: dialect identification, summarization, sentiment analysis, stance detection, multiple choice, sarcasm detection, machine translation, NLI, question answering, diacritization, review classification, offensive language detection, emotion classification, commonsense validation, topic classification, math solving, era classification, yes/no question answering, claim verification, theme classification, targeted sentiment analysis, text classification, meter classification, named entity recognition, semantic similarity, part-of-speech tagging, and semantic question similarity.

The six tasks used in the paper's fine-tuning experiments (dialect identification, machine translation, NLI, multiple choice, sarcasm detection, summarization) are a subset of these; filter by instruction_template_id against the experimental subset of the templates dataset to isolate them.

Size: 46,263,378 rendered instructions in a single train split.

Usage

from datasets import load_dataset

ds = load_dataset("KFUPM-JRCAI/star-dataset-instructions")

# Filter by task
summarization = ds.filter(lambda x: 'summarization' in x['instruction_tasks'])

# Filter by dataset
xlsum = ds.filter(lambda x: x['dataset_name'] == 'GEM/xlsum')

# Access instruction parts
sample = ds['train'][0]
print(sample['instruction_input'])   # The prompt/question
print(sample['instruction_output'])  # The expected answer
print(sample['instruction_template']) # The raw Jinja2 template

How Merging Works

1. 1.Templates are fetched from the PromptLab API and filtered to approved status
2. 2.Each template's referenced HuggingFace dataset is downloaded
3. 3.For every (template, data sample, split) combination:
4. 4.The Jinja2 template is rendered with the sample's fields as variables
5. 5.The rendered text is split on ||| into input and output parts
6. 6.The record is stored with both the rendered instruction and template metadata
7. 7.Templates that produce invalid splits (multiple ||| occurrences) are skipped

The exact pipeline is a notebook in the repository: `build_merged_instructions.ipynb` renders every approved template against its source dataset and writes the merged Arrow dataset, and `push_merged_instructions.ipynb` uploads the result here. Read them to see exactly how any record in this dataset was produced, or to rebuild it yourself.

Schema

Connection to STAR Templates

This dataset is derived from STAR Templates. Each record here corresponds to a template from that dataset applied to a specific data sample. The instruction_template_id column can be used to join with the templates dataset for experiment-level metadata (e.g., which prompts were selected for tuning vs. evaluation).

Paper

This dataset accompanies [STAR: instruction tuning for Arabic across tasks, datasets, and models](https://link.springer.com/article/10.1007/s10579-026-09942-8), published in Language Resources and Evaluation (2026), which describes how the templates were collected and the experiments they were used for.

Code for the fine-tuning and evaluation experiments: github.com/KFUPM-JRCAI/star-instructions-tuning.

If you use this dataset, please cite:

@article{alshaibani2026star,
  title   = {STAR: instruction tuning for Arabic across tasks, datasets, and models},
  author  = {Al-Shaibani, Maged S. and Alyafeai, Zaid and Ahmad, Irfan},
  journal = {Language Resources and Evaluation},
  volume  = {60},
  number  = {4},
  pages   = {69},
  year    = {2026},
  doi     = {10.1007/s10579-026-09942-8},
  url     = {https://link.springer.com/article/10.1007/s10579-026-09942-8}
}