Datasetpublic

Moealsarraj/arabic-bench-dataset

Arabic Bench Dataset A curated evaluation dataset for benchmarking AI models on Arabic language tasks. Overview 50 test cases across 8 categories Each case includes a prompt, gold-standard reference answer, and a deliberately imperfect AI response Covers Modern Standard Arabic (MSA) and multiple Arabic dialects Designed for evaluating: translation, summarization, Q&A, creative writing, grammar, dialect understanding, legal/formal, and medical/scientific tasks… See the full description on the dataset page: https://huggingface.co/datasets/Moealsarraj/arabic-bench-dataset.

sourceHugging Facemitupdated 6mo agoView on Hugging Face

0likes18downloads

Dataset Card

Arabic Bench Dataset

A curated evaluation dataset for benchmarking AI models on Arabic language tasks.

Overview

—50 test cases across 8 categories
—Each case includes a prompt, gold-standard reference answer, and a deliberately imperfect AI response
—Covers Modern Standard Arabic (MSA) and multiple Arabic dialects
—Designed for evaluating: translation, summarization, Q&A, creative writing, grammar, dialect understanding, legal/formal, and medical/scientific tasks

Category	Arabic	Count
Translation	ترجمة	7
Summarization	تلخيص	6
Question Answering	أسئلة وأجوبة	6
Creative Writing	كتابة إبداعية	6
Grammar & Correction	نحو وإملاء	6
Dialect Understanding	لهجات	6
Legal & Formal	قانوني ورسمي	6
Medical & Scientific	طبي وعلمي	7

Schema

Each test case contains:

Field	Type	Description
`id`	string	Unique identifier (e.g., "tr-01")
`category`	string	Category ID
`difficulty`	string	easy, medium, or hard
`title`	string	English title
`title_ar`	string	Arabic title
`prompt`	string	The task prompt
`reference`	string	Gold-standard reference answer
`ai_input`	string	Deliberately imperfect AI response for evaluation

Usage

python

from datasets import load_dataset
ds = load_dataset("Moealsarraj/arabic-bench-dataset", split="train")

Or load directly:

python

import json
with open("data/test_cases.jsonl") as f:
    cases = [json.loads(line) for line in f]

Live Tool

Try the interactive evaluation tool: Arabic Bench on HuggingFace Spaces

Author