Datasetpublic

dyfan/halluhard-seed-questions

This is the seed question set for the benchmark HalluHard We design some difficult seed questions that elicit a citation-grounded multi-turn open-ended generation, covering four domains: legal, research, medical, and coding. Below is the Top15 models on our leaderboard. Please check our website for more models and turn-wise/domain-wise statistics! Rank Model Hallucination Rate Legal Research Medical Coding 1 Claude-Opus-4.5-Web-Search 30.2 33.0 29.6 29.2 29.0 2… See the full description on the dataset page: https://huggingface.co/datasets/dyfan/halluhard-seed-questions.

sourceHugging Facemitupdated 2mo agoView on Hugging Face

2likes52downloads

Dataset Card

This is the seed question set for the benchmark HalluHard

We design some difficult seed questions that elicit a citation-grounded multi-turn open-ended generation, covering four domains: legal, research, medical, and coding.

Below is the Top15 models on our leaderboard. Please check our website for more models and turn-wise/domain-wise statistics!

Rank	Model	Hallucination Rate	Legal	Research	Medical	Coding
1	Claude-Opus-4.5-Web-Search	30.2	33.0	29.6	29.2	29.0
2	GPT-5.4-Thinking-Web-Search	31.9	16.5	54.8	51.9	4.3
3	GPT-5.2-thinking-Web-Search	38.2	35.6	52.6	48.8	15.8
4	GLM-5-Thinking-Web-Search	39.7	26.8	38.9	35.0	58.0
5	GPT-5.3-Web-Search	43.5	26.3	54.0	53.8	39.7
6	GPT-5.4-Thinking	46.8	29.9	72.7	72.9	11.7
7	GPT-5.2-thinking	53.8	33.5	75.3	74.0	32.2
8	GPT-5.3	53.8	34.6	69.1	71.1	40.2
9	Gemini-3.1-Pro	57.1	43.3	78.2	80.0	27.0
10	GPT-5.2	58.8	46.4	79.4	72.7	36.8
11	Kimi-K2.5-Web-Search	59.1	42.5	63.4	67.5	63.2
12	Claude-Opus-4.5	60.0	44.8	84.0	85.6	25.7
13	Claude-Opus-4.6	60.9	49.7	81.6	89.3	23.2
14	Gemini-3-Pro	61.9	46.0	84.1	85.9	31.7
15	Claude-Sonnet-4.6	63.7	49.5	85.8	89.7	29.7

Paper: https://arxiv.org/abs/2602.01031

Code: https://github.com/epfml/halluhard

If you use HalluHard in your research, please cite:

@misc{fan2026halluhardhardmultiturnhallucination,
                            title={HalluHard: A Hard Multi-Turn Hallucination Benchmark}, 
                            author={Dongyang Fan and Sebastien Delsad and Nicolas Flammarion and Maksym Andriushchenko},
                            year={2026},
                            eprint={2602.01031},
                            archivePrefix={arXiv},
                            primaryClass={cs.AI},
                            url={https://arxiv.org/abs/2602.01031}, 
                      }