GaborMadarasz/HuHotpotQA_8k

HuHotpotQA

HuHotpotQA is a Hungarian-language multi-hop question answering dataset designed in the style of HotpotQA. It contains approximately 2,000 question-answer pairs based on articles from Hungarian Wikipedia.

The dataset is designed to evaluate and train models on questions that require combining information from multiple documents rather than retrieving an answer from a single context.

This is a truncated version of the original dataset.

To count the tokens, I used the gemma-3-270-it tokenizer and truncated the examples in the dataset to 8,000 tokens. I only truncated the distracting articles.

Dataset Summary

• —Language: Hungarian
• —Task: Multi-hop Question Answering
• —Domain: General knowledge / Wikipedia
• —Source: Hungarian Wikipedia
• —Format: HotpotQA-style Hugging Face datasets
• —Size: approximately 2,000 examples
• —Question types: Bridge and Comparison
• —Context: Supporting articles plus distractor articles
• —Primary entity source: Wikidata
• —Fallback entity source: Hungarian Wikipedia links

The planned dataset structure follows the HotpotQA representation:

_id
question
answer
type
level
supporting_facts
context

Motivation

Most Hungarian question answering datasets focus primarily on single-hop questions, where the answer can be directly extracted from a single passage.

HuHotpotQA focuses instead on multi-hop reasoning. A model should identify and combine information distributed across multiple Wikipedia articles in order to answer a question.

For Bridge questions, the model must follow an intermediate entity or article connecting two pieces of information. Comparison questions require the model to retrieve information about multiple entities and compare them.

The dataset is therefore intended to provide a Hungarian benchmark for:

• —multi-hop question answering;
• —retrieval-augmented generation;
• —document retrieval and evidence selection;
• —supporting-fact identification;
• —multilingual and Hungarian language models;
• —evaluation of reasoning over multiple documents.

Dataset Structure

Each example contains the following fields:

The context contains the articles necessary for answering the question as well as additional distractor articles. The generation plan specifies 5–7 distractor articles per example.

Question Types

Bridge

A Bridge question requires following an intermediate connection between entities.

Conceptually:

Article A → intermediate entity/article B → Article C

The intermediate information is necessary to connect the information needed to answer the question.

Comparison

A Comparison question requires retrieving information about two entities and comparing them.

The question therefore cannot be reliably answered by examining only one of the relevant articles.

The generation pipeline explicitly distinguishes these two question types and uses separate generation templates for them.

Data Source

The contexts are derived from a frozen snapshot of the Hungarian Wikipedia.

The dataset generation pipeline uses the latest available Hungarian Wikipedia XML dump at generation time. Articles are parsed from the dump and filtered before question generation. Redirects and talk pages are excluded, and articles below the configured minimum length are discarded.

Using a frozen Wikipedia snapshot reduces answer drift between dataset generation and subsequent evaluation.

Entity Graph

The dataset generation process constructs an entity graph using two complementary sources:

1. 1.Wikidata, used as the primary source of entity relationships.
2. 2.Wikipedia links, used to fill gaps where Wikidata relationships are unavailable.

Two-hop paths of the form

A → B → C

are extracted from this graph and used as the basis for multi-hop question generation.

Data Generation

Questions are generated using a combined approach involving:

• —predefined question templates;
• —entity and article relationships;
• —a configurable large language model.

The generation pipeline supports both local LLM inference through Ollama and remote inference through OpenRouter.

The generated output is required to contain:

{
  "question": "...",
  "answer": "...",
  "answer_type": "..."
}

Generation is performed in batches and supports checkpointing so that interrupted generation can be resumed.

Validation

HuHotpotQA applies automated validation followed by LLM (gemini-3.1-flash-lite) review where applicable.

An example is included only if it satisfies the following criteria:

1. 1.The answer appears verbatim in at least one context article.
2. 2.The question requires two or more reasoning hops.
3. 3.The Hungarian question is grammatically acceptable.
4. 4.The question is not a duplicate of an existing example.
5. 5.Supporting facts are correctly identified.
6. 6.The example is accepted during LLM review.

These criteria are defined in the dataset generation plan.

Answer Validation

The validation pipeline checks whether the answer occurs in the supplied context and identifies the article or articles containing the answer.

Multi-Hop Validation

An LLM-as-judge step is used to evaluate whether the question genuinely requires multi-hop reasoning rather than being answerable from a single article.

Duplicate Detection

Questions are normalized and compared against existing questions. Entries exceeding the configured token-overlap threshold are rejected.

The configured duplicate threshold is 0.85.

Difficulty

Each example is assigned a difficulty level based on characteristics including:

• —path length;
• —article length;
• —question complexity.

The exact difficulty distribution depends on the generated and validated dataset.

Recommended Uses

HuHotpotQA can be used for:

• —supervised fine-tuning of Hungarian QA models;
• —evaluation of Hungarian language models;
• —multi-hop reasoning benchmarks;
• —retrieval-augmented QA experiments;
• —retriever and reader evaluation;
• —evidence and supporting-fact prediction;
• —comparison of multilingual and Hungarian-specific models.

Out-of-Scope Uses

The dataset should not be interpreted as a comprehensive representation of Hungarian factual knowledge.

Because the dataset is derived from Wikipedia, its coverage reflects the structure and content of Hungarian Wikipedia at the time of dataset generation.

The dataset is also not intended to measure general reasoning ability independently of language understanding, retrieval, and Wikipedia knowledge.

Biases and Limitations

Potential limitations include:

• —Wikipedia coverage bias: Topics and entities that are poorly represented in Hungarian Wikipedia are less likely to occur.
• —Entity graph bias: The distribution of questions depends on the connectivity of Wikidata and Wikipedia links.
• —LLM generation bias: Generated questions may reflect biases or stylistic characteristics of the model used during generation.
• —Hungarian language quality: Although grammatical quality is part of validation, automatically generated questions may still contain linguistic artifacts.
• —Multi-hop verification: Determining whether a question genuinely requires multiple reasoning steps is inherently difficult, even with automated judging and LLM review.
• —Temporal limitation: Answers and article contents correspond to the frozen Wikipedia snapshot used during generation.
• —LLM Judge bias

The project plan specifically identifies the limited size of Hungarian Wikipedia, local LLM quality, Wikidata query limits, and the difficulty of ensuring genuinely multi-hop questions as relevant risks.

Dataset Creation

The overall pipeline consists of:

1. 1.Hungarian Wikipedia parsing
2. 2.Entity graph construction
3. 3.Two-hop path extraction
4. 4.Question generation
5. 5.Automated validation
6. 6.LLM review
7. 7.Dataset assembly
8. 8.Hugging Face export

The resulting dataset is exported in Hugging Face datasets format.

Citation

If you use HuHotpotQA in your research, please cite the dataset repository:

@dataset{huhotpotqa,
  title        = {HuHotpotQA: Hungarian Multi-Hop Question Answering Dataset},
  year         = {2026},
  language     = {Hungarian},
  publisher    = {Hugging Face}
}

If a formal publication describing the dataset becomes available, this citation should be replaced or supplemented with the corresponding paper citation.

License

The dataset license should be interpreted together with the licensing terms of the underlying Hungarian Wikipedia content and any other external resources used during dataset construction.

Users should verify the applicable Wikimedia/Wikipedia licensing requirements before redistributing the dataset or derived resources.

Acknowledgements

HuHotpotQA is based on content from the Hungarian Wikipedia and entity information obtained from Wikidata.

The dataset generation methodology is inspired by the multi-hop question answering paradigm established by HotpotQA.

Repository

The dataset generation code, configuration, processing pipeline, and dataset artifacts are maintained in the associated project repository.

Future releases may contain modified questions, additional validation, updated source data, or changes to the generation pipeline. Dataset versions should therefore be treated as separate benchmark releases.