vincentkoc/tiny_qa_benchmark

1---2license: apache-2.03language:4  - en5pretty_name: Tiny QA Benchmark (Original EN Core for TQB++)6size_categories:7  - n<1K8tags:9  - question-answering10  - evaluation11  - benchmark12  - toy-dataset13  - tqb++-core14task_categories:15  - question-answering16task_ids:17  - extractive-qa18  - closed-book-qa19arxiv: 2505.1205820datasets:21  - vincentkoc/tiny_qa_benchmark_pp22---23 24# Tiny QA Benchmark (Original English Core for TQB++)25 26**This dataset (`vincentkoc/tiny_qa_benchmark`) is the original 52-item English Question-Answering set. It now serves as the immutable "gold standard" core for the expanded [Tiny QA Benchmark++ (TQB++)](https://huggingface.co/datasets/vincentkoc/tiny_qa_benchmark_pp) project.**27 28The TQB++ project builds upon this core dataset by introducing a powerful synthetic generation toolkit, pre-built multilingual datasets, and a comprehensive framework for rapid LLM smoke testing.29 30**For the full TQB++ toolkit, the latest research paper, multilingual datasets, and the synthetic generator, please visit:**31*   **TQB++ Hugging Face Dataset Collection & Toolkit:** [vincentkoc/tiny_qa_benchmark_pp](https://huggingface.co/datasets/vincentkoc/tiny_qa_benchmark_pp)32*   **TQB++ GitHub Repository (Code, Paper & Toolkit):** [vincentkoc/tiny_qa_benchmark_pp](https://github.com/vincentkoc/tiny_qa_benchmark_pp)33 34This original dataset (`vincentkoc/tiny_qa_benchmark`) contains 52 hand-crafted general-knowledge QA pairs covering geography, history, math, science, literature, and more. It remains ideal for quick sanity checks, pipeline smoke-tests, and as a foundational component of TQB++. Each example includes:35 36- **text**: the question prompt  37- **label**: the “gold” answer  38- **metadata.context**: a one-sentence fact  39- **tags**: additional annotations (`category`, `difficulty`)40 41It’s intentionally tiny (<100 KB) so you can iterate on data loading, evaluation scripts, or CI steps in under a second using these specific 52 items.42 43## Supported Tasks and Formats (for this core dataset)44 45- **Tasks**:  46  - Extractive QA  47  - Generative QA  48- **Format**: JSON  49- **Splits**:  50  - `train` (all 52 examples)  51 52## Languages (for this core dataset)53 54- English (`en`)55 56## Dataset Structure57 58### Data Fields59 60Each example in `data/train.json` (as loaded by `datasets`) has:61 62| field               | type   | description                                  |63|---------------------|--------|----------------------------------------------|64| `text`              | string | The question prompt.                         |65| `label`             | string | The correct answer.                          |66| `metadata`          | object | Additional info.                             |67| `metadata.context`  | string | A one-sentence fact supporting the answer.   |68| `tags.category`     | string | Broad question category (e.g. `geography`).  |69| `tags.difficulty`   | string | Rough difficulty level (e.g. `easy`).        |70 71## Data Example72 73```json74[75  {76    "text": "What is the capital of France?",77    "label": "Paris",78    "metadata": {79      "context": "France is a country in Europe. Its capital is Paris."80    },81    "tags": {82      "category": "geography",83      "difficulty": "easy"84    }85  },86  {87    "text": "What is 2 + 2?",88    "label": "4",89    "metadata": {90      "context": "Basic arithmetic: 2 + 2 equals 4."91    },92    "tags": {93      "category": "math",94      "difficulty": "easy"95    }96  }97]98```99*(Note: The actual file on the Hub might be a `.jsonl` file where each line is a JSON object, but `load_dataset` handles this.)*100 101## Data Splits102 103Only one split for this core dataset:104 105- **train**: 52 examples, used for development, quick evaluation, and as the TQB++ core.106 107## Data Creation108 109### Curation Rationale110 111The "Tiny QA Benchmark" (this 52-item set) was originally created to:112 1131. Smoke-test QA pipelines (loading, preprocessing, evaluation).  1142. Demo Hugging Face Datasets integration in tutorials.  1153. Verify model–eval loops run without downloading large corpora.1164. **Serve as the immutable "gold standard" English core for the [Tiny QA Benchmark++ (TQB++)](https://huggingface.co/datasets/vincentkoc/tiny_qa_benchmark_pp) project.**117 118### Source Data119 120Hand-crafted by the dataset creator from well-known, public-domain facts.121It was initially developed as a dataset for sample projects to demonstrate [Opik](https://github.com/comet-ml/opik/) and now forms the foundational English core of TQB++.122 123### Annotations124 125Self-annotated. Each `metadata.context` and `tags` field is manually created for these 52 items.126 127## Usage128 129Load this specific 52-item core dataset with:130 131```python132from datasets import load_dataset133 134ds = load_dataset("vincentkoc/tiny_qa_benchmark")135print(ds["train"][0])136# Expected output:137# {138#   "text": "What is the capital of France?",139#   "label": "Paris",140#   "metadata": {141#     "context": "France is a country in Europe. Its capital is Paris."142#   },143#   "tags": {144#     "category": "geography",145#     "difficulty": "easy"146#   }147# }148```149For accessing the full TQB++ suite, including multilingual packs and the synthetic generator, refer to the [TQB++ Hugging Face Dataset Collection](https://huggingface.co/datasets/vincentkoc/tiny_qa_benchmark_pp).150 151## Considerations for Use152 153*   **Immutable Core for TQB++:** This dataset is the stable, hand-curated English core of the TQB++ project. Its 52 items are not intended to change.154*   **Not a Comprehensive Benchmark (on its own):** While excellent for quick checks, these 52 items are too few for statistically significant model ranking. For broader evaluation, use in conjunction with the TQB++ synthetic generator and its multilingual capabilities found at [vincentkoc/tiny_qa_benchmark_pp](https://huggingface.co/datasets/vincentkoc/tiny_qa_benchmark_pp).155*   **Do Not Train:** Primarily intended for evaluation, smoke-tests, or demos.156*   **No Sensitive Data:** All facts are public domain.157 158## Licensing159 160Apache-2.0. See the `LICENSE` file in the [TQB++ GitHub repository](https://github.com/vincentkoc/tiny_qa_benchmark_pp) for details (as this dataset is now part of that larger project).161 162## Citation163 164If you use this specific 52-item core English dataset, please cite it. You can use the following BibTeX entry, which has been updated to reflect its role:165 166```bibtex167@misc{koctinyqabenchmark_original_core,168	author       = { Vincent Koc },169	title        = { Tiny QA Benchmark (Original 52-item English Core for TQB++) },170	year         = 2025,171	url          = { https://huggingface.co/datasets/vincentkoc/tiny_qa_benchmark },172	doi          = { 10.57967/hf/5417 },173	publisher    = { Hugging Face }174}175```176 177For the complete **Tiny QA Benchmark++ (TQB++)** project (which includes this core set, the synthetic generator, multilingual packs, and the associated research paper), please refer to and cite the TQB++ project directly:178 179```bibtex180@misc{koctinyqabenchmark_pp_dataset,181  author       = {Vincent Koc},182  title        = {Tiny QA Benchmark++ (TQB++) Datasets and Toolkit},183  year         = {2025},184  publisher    = {Hugging Face & GitHub},185  doi          = {10.57967/hf/5531}, /* DOI for the TQB++ collection */186  howpublished = {\\url{https://huggingface.co/datasets/vincentkoc/tiny_qa_benchmark_pp}},187  note         = {See also: \\url{https://github.com/vincentkoc/tiny_qa_benchmark_pp}}188}189```