Hailay/tigqa-extractive-qa

TIGQA — Extractive QA Subset

Dataset Summary

This repository contains a cleaned, extractive-QA-only (SQuAD-style) subset of the TIGQA question–answering data: 1,665 question–answer pairs in Tigrinya, each with a single answer located as an exact character span inside its context passage (answer_start and the span length), ready for direct use in span-extraction QA training and evaluation.

Every row carries a match_status field (exact or fuzzy) recording how confidently the answer span was located during cleaning, so downstream users can filter by confidence if needed.

This is not the official TIGQA release — see "Relationship to the original dataset" below for exactly how it differs.

Credit / Source

This dataset is derived from TIGQA, the expert-annotated Tigrinya machine reading comprehension dataset introduced in:

Hailay Kidu Teklehaymanot, Dren Fazlija, Niloy Ganguly, Gourab K. Patro, and Wolfgang Nejdl. TIGQA: An Expert-Annotated Question-Answering Dataset in Tigrinya. LREC-COLING 2024, Turin, Italy. Paper (ACL Anthology)

Officially released on Zenodo:

TIGQA: An Expert-Annotated Question-Answering Dataset in Tigrinya. Zenodo. https://doi.org/10.5281/zenodo.11423987 · License: CC-BY-4.0

All credit for the underlying data collection, expert annotation, and research goes to the original authors above.

About the original TIGQA dataset

TIGQA is a SQuAD-style reading comprehension dataset built to address the scarcity of annotated QA resources for Tigrinya, a low-resource language spoken by roughly 10 million people in Eritrea and the Tigray region of Ethiopia.

• —2,685 question–answer pairs (dev/test questions have 3+ reference answers each) over 537 context paragraphs, drawn from 455 pages of five Ethiopian Ministry of Education textbooks (four Tigrinya-language books, one Biology book), covering 122 topics.
• —Two subsets by grade band: TIGQA-E (Elementary, grades 4–5) and TIGQA-H (High School, grades 10–11).
• —Official split: train 1,215 questions / 407 paragraphs, dev 200 questions / 65 paragraphs, test 200 questions / 65 paragraphs.
• —Annotated by four expert subject teachers (balanced 2 men / 2 women, different dialects), instructed to reformulate answers rather than copy word-for-word; a secondary shortest-span-selection pass caught and corrected/removed under 2% of questions.
• —Question types: "what"/"why" 59.2%, "where"/"which"/"who" 31.4%, "when"/other 9.4%. Answering requires more than word matching — a 100-example sample showed 27.2% word matching, 26.6% paraphrasing, 24.3% single-sentence reasoning, 21.9% multi-sentence reasoning.
• —Benchmarked models leave a large human/model gap: best model (XLM-R Large) scored 66.56 test EM / 84.34 test F1, versus 92.24 EM / 94.43 F1 for human performance.

Relationship to the original dataset

The file behind both the Zenodo release and this repository is the same underlying annotation table (TIGQA Tigrinya Question Answering dataset.docx). This repo was built by parsing that table into a strict extractive-QA format:

• —Each answer was located as an exact substring span of its context (whitespace-insensitive exact match, or an approximate/fuzzy match when spacing or minor wording differed).
• —Question–answer pairs whose answer could not be confidently located as a span in the context (e.g. answers that paraphrase or infer rather than copy from the passage) were excluded from this subset rather than kept without a valid answer_start.
• —As a result this subset has 1,665 single-answer pairs, not the official 2,685 (which includes multiple reference answers per dev/test question and pairs not reducible to a single extractive span). It does not reproduce the official TIGQA-E/TIGQA-H or train/dev/test splits — everything here is a single, unsplit train set.
• —The grade field is populated only where it was directly recoverable from the source table's cell structure; a large share of rows have an empty grade value due to merged table cells in the original document, not a data error. Cross-reference title/context against the original release if grade-level information is required for a given row.

If you need the complete, officially split dataset with all reference answers, use the Zenodo release or the original paper's data directly.

Dataset Structure

• —Split: train (1,665 rows)
• —Fields:
• —id (string) — unique identifier per QA pair
• —title (string) — topic/lesson title
• —grade (string) — school grade level, where recoverable (see caveat above; otherwise empty)
• —context (string) — the passage the question is asked about
• —question (string) — the question, in Tigrinya
• —answers (dict) — {"text": [str], "answer_start": [int]}, SQuAD-style
• —match_status (string) — "exact" (answer found verbatim, whitespace-insensitive) or "fuzzy" (answer span located via approximate matching; consider spot-checking these before relying on them for evaluation)

Usage

from datasets import load_dataset

ds = load_dataset("Hailay/tigqa-extractive-qa")
print(ds["train"][0])

License

Released under CC-BY-4.0, consistent with the license of the original TIGQA dataset. If you use this data, please cite the original TIGQA paper below.

Citation

@inproceedings{teklehaymanot2024tigqa,
  title     = {{TIGQA}: An Expert-Annotated Question-Answering Dataset in {T}igrinya},
  author    = {Teklehaymanot, Hailay Kidu and Fazlija, Dren and Ganguly, Niloy and Patro, Gourab K. and Nejdl, Wolfgang},
  booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
  year      = {2024},
  address   = {Turin, Italy},
  url       = {https://aclanthology.org/2024.lrec-main.1404.pdf}
}