inceptlabs/Arabic_EXAMS-Redux

Arabic_EXAMS-Redux

A corrected and text-repaired version of `OALL/Arabic_EXAMS`, the Arabic subset of the EXAMS multilingual high-school examinations benchmark.

What was fixed

1. 1.Repaired corrupted Arabic text. The upstream benchmark contains widespread PDF-extraction damage to question stems and answer choices: split diacritics, fragmented words, and non-Arabic glyphs replacing standard characters. We restored these to readable Modern Standard Arabic.
2. 2.Corrected the answer keys. We independently re-checked every question and corrected reference answers that were factually wrong, with citations recorded in the audit trail.
3. 3.Preserved attributed religious text verbatim. Quranic verses, hadith, and athar quotations were left untouched except for character-level OCR fixes.

TL;DR

• —562 rows total (537 test + 25 validation), schema-identical to upstream.
• —190 answer-key corrections (33.8% of the benchmark).
• —138 rows had OCR/text damage repaired.
• —Per-row audit trail in audit/{test,validation}.jsonl records every change.

Per-subject answer-key error rate

Examples of factually wrong upstream answers

Schema

Identical to upstream OALL/Arabic_EXAMS:

Audit metadata

Per-row provenance is in the audit/ directory (one JSONL per split). Each row contains:

• —Original and final question / choices / answer
• —Reasoning behind any text repair
• —Reasoning and citations behind any answer-key change
• —Flags for rows with unrecoverable upstream corruption

Limitations

• —A small number of rows have unrecoverable upstream corruption (full font-mapping failure on the question stem) and are preserved as-is from upstream.
• —A few rows touch on curriculum-specific facts where a definitive textbook source could not be located; original keys were preserved in those cases.

Citation

@misc{arabic_exams_redux_2026,
  title  = {Arabic\_EXAMS-Redux: A Corrected Re-Annotation of the Arabic Subset of the EXAMS Multilingual High-School Benchmark},
  author = {Abdulhamid, Ahmed and IL Labs},
  year   = {2026},
  url    = {https://huggingface.co/datasets/inceptlabs/Arabic_EXAMS-Redux}
}

Original sources

This work corrects the OALL Arabic mirror, which is itself based on the EXAMS benchmark:

@misc{oall_arabic_exams,
  title = {Open Arabic Leaderboard - Arabic EXAMS},
  url = {https://huggingface.co/datasets/OALL/Arabic_EXAMS}
}

@inproceedings{hardalov-etal-2020-exams,
  title = {{EXAMS}: A Multi-subject High School Examinations Dataset for Cross-lingual and Multilingual Question Answering},
  author = {Hardalov, Momchil and Mihaylov, Todor and Zlatkova, Dimitrina and Dinkov, Yoan and Koychev, Ivan and Nakov, Preslav},
  booktitle = {EMNLP 2020},
  year = {2020}
}

License

Released under CC-BY-SA-4.0 to match the upstream benchmark.