datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
australian-tax-guidance-retrieval
Australian Tax Guidance Retrieval 🏦
Australian Tax Guidance Retrieval by Isaacus is a novel, diverse, and challenging legal information retrieval evaluation dataset consisting of 112 real-life Australian tax law questions paired with expert-annotated, relevant Australian Government tax guidance and policies.
Uniquely, this dataset sources its real-life tax questions from the posts of everyday Australian taxpayers on the ATO Community forum, with relevant Australian Government… See the full description on the dataset page: https://huggingface.co/datasets/isaacus/australian-tax-guidance-retrieval.concurrentqa-retrievalConcurrentQA is a textual multi-hop QA benchmark to require concurrent retrieval over multiple data-distributions (i.e. Wikipedia and email data). This dataset was constructed by researchers at Stanford and FAIR, following the data collection process and schema of HotpotQA. This benchmark can be used to study generalization in retrieval as well as privacy when reasoning across multiple privacy scopes --- i.e. public Wikipedia documents and private emails.
This dataset is for the Retrieval… See the full description on the dataset page: https://huggingface.co/datasets/stanfordnlp/concurrentqa-retrieval.Retrieval-Synthetic-NVDocs-v1
Dataset Description:
Retrieval-Synthetic-NVDocs-v1 is a synthetic retrieval dataset with question–answer supervision designed to train and evaluate embedding and RAG systems. The dataset was generated on top of NVIDIA's publicly available content using NeMo Data Designer, NVIDIA's open-source framework for generating high-quality synthetic data from scratch or based on seed data.
The dataset contains document chunks paired with semantically rich question-answer pairs across multiple… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Retrieval-Synthetic-NVDocs-v1.mteb-fr-retrieval-syntec-s2p
Syntec dataset for information retrieval
This dataset has been built from the Syntec Collective bargaining agreement. Its purpose is information retrieval.
Dataset Details
The dataset is rather small. It is intended to be used only as a test set, for fast evaluation of models.
It is split into 2 subsets :
queries : it features 100 manually created questions. Each question is mapped to the article that contains the answer.
documents : corresponds to the 90 articles from… See the full description on the dataset page: https://huggingface.co/datasets/lyon-nlp/mteb-fr-retrieval-syntec-s2p.statcan-dialogue-dataset-retrieval
Statcan Dialogue Dataset (Processed for Retrieval Tasks)
This is a variant of the Statcan Dialogue Dataset, which we processed specifically for multilingual retrieval (english, french). It contains everything in CSVs, rather than having metadata hosted separately.
Quickstart
from datasets import load_dataset
repo = 'McGill-NLP/statcan-dialogue-dataset-retrieval'
# load english queries, training split
queries_en = load_dataset(repo, 'queries_english', split='train') #… See the full description on the dataset page: https://huggingface.co/datasets/McGill-NLP/statcan-dialogue-dataset-retrieval.enwikivoyage-retrieval-202605
English Wikivoyage Retrieval 2026-05
I like travel datasets because they are about real places, real constraints,
and the small practical questions people ask before they go somewhere. I am
sharing this English Wikivoyage retrieval corpus in that spirit: as honest
work from a researcher-builder who wants to explore the world, make the
pipeline inspectable, and let other people reuse or challenge the choices.
This is not presented as a finished travel product or a private… See the full description on the dataset page: https://huggingface.co/datasets/vincentss/enwikivoyage-retrieval-202605.regulation-retrieval
Turkish Legal Özelge Corpus Dataset
📊 Dataset Summary
Turkish Legal Özelge Corpus is a comprehensive Information Retrieval dataset consisting of özelge (tax ruling) decisions published by the Turkish Revenue Administration (Gelir İdaresi Başkanlığı - GİB).
Key Features
Format: BEIR (Benchmarking IR) format with corpus-queries-qrels structure
Language: Turkish 🇹🇷
Domain: Tax Law, Administrative Law, Turkish Law
Source: GİB Özelge Decisions
Use Cases:… See the full description on the dataset page: https://huggingface.co/datasets/newmindai/regulation-retrieval.kbmill-brick-retrieval
KBMill Brick Retrieval Demos
Portable, residual-honest knowledge bricks turned into retrieval evaluation sets for KBMill — the public mill at kbmill.com. Shelf packages live in kbmill-brick-library.
These are not unbounded wiki dumps or synthetic QA. They come from real KBMill manufacturing: bounded packages with muted residual junk filtered where applicable, craft notes, security report in the ZIP, and published, re-runnable cosine retrieval evidence.
Config
Queries… See the full description on the dataset page: https://huggingface.co/datasets/CMiller/kbmill-brick-retrieval.kinya-ag-retrieval
Kinyarwanda Agricultural Retrieval Dataset
In Rwanda, many farmers struggle to access timely, personalized agricultural information. Traditional channels - like radio, TV, and online sources - offer limited reach and interactivity, while extension services and a national call center, staffed by only two agents for over two million farmers, face capacity constraints. To address these gaps, we developed a 24/7 AI-enabled Interactive Voice Response (IVR) tool. Accessible via a… See the full description on the dataset page: https://huggingface.co/datasets/C4IR-RW/kinya-ag-retrieval.financial-retrieval
Overview
This dataset contains normalized, long-form records used to benchmark multiple chatbots on financial retrieval QA.
Each row represents a single (ticker, chatbot) pair answering one question.
These records are derived from a verification step that extracts structured fields from each chatbot’s answer.
Columns (normalized dataset)
Column
Type
Description
ticker
string
Company identifier used for the question (e.g., AAPL, 7203:JP).
question
string… See the full description on the dataset page: https://huggingface.co/datasets/daloopa/financial-retrieval.medquad-retrieval-pretriage
MedQuAD Retrieval Pre-Triage Dataset
Dataset Description
This repository contains a processed, retrieval-oriented derivative of the MedQuAD medical question-answering dataset.
It was prepared for contextual medical information retrieval in SortMed, an academic medical pre-triage assistant.
The corpus is not used to train the SortMed triage classifiers. It is used by a separate semantic retrieval component that identifies medically related question-answer entries… See the full description on the dataset page: https://huggingface.co/datasets/cristian-untaru/medquad-retrieval-pretriage.wikipedia-human-retrieval-ja
Japanese Wikipedia Human Retrieval dataset
This is a Japanese question answereing dataset with retrieval on Wikipedia articles
by trained human workers.
Contributors
Yusuke Oda
defined the dataset specification, data structure, and the scheme of data collection.
Baobab, Inc.
operated data collection, data checking, and formatting.
About the dataset
Each entry represents a single QA session:
given a question sentence, the responsible worker tried to search for… See the full description on the dataset page: https://huggingface.co/datasets/baobab-trees/wikipedia-human-retrieval-ja.regulation-retrieval
Not: Bu veri setinin dokümantasyonu Türk yapay zeka topluluğuna katkı sağlamak amacıyla VeriPazarı tarafından Türkçeye çevrilmiştir. Orijinal veri seti newmindai tarafından geliştirilmiş olup, VeriPazarı tarafından Türk AI ekosistemi için arşivlenmiştir.
🔗 Orijinal Kaynak: newmindai/regulation-retrieval
🔗 Derleyen Platform: VeriPazarı
Turkish Legal Özelge Corpus Veri Seti
📊 Veri Seti Özeti
Turkish Legal Özelge Corpus, Gelir İdaresi Başkanlığı (GİB) tarafından… See the full description on the dataset page: https://huggingface.co/datasets/Taklaxbr/regulation-retrieval.apple-environmental-report-QA-retrieval
Apple's 2024 Environmental Report QA Pairs
4300 question and relevant text chunks made from Apple's 2024 Environmental Report.
Chunking was done with a token based recursive chunker at 800 token chunk size with a 400 token overlap resulting in 215 chunks. 20 question labels per chunk were synthetically generated using gpt-4o-mini with the attached prompt and a temperature of 1.0.
Entries were shuffled and split into an 80/20 Train/Validation split resulting in:Training set size:… See the full description on the dataset page: https://huggingface.co/datasets/AdamLucek/apple-environmental-report-QA-retrieval.russian-retrievalBased on Sberquad
Answer converted to human affordable answer.
Context augmented with some pices of texts from wiki accordant to text on tematic and keywords.
This dataset cold be used for training retrieval LLM models or modificators for ability of LLM to retrieve target information from collection of tematic related texts.
Dataset has version with SOURCE data for generating answer with specifing source document for right answer. See file retrieval_dataset_src.jsonl
Dataset consists of… See the full description on the dataset page: https://huggingface.co/datasets/MLNavigator/russian-retrieval.retrieval_grounding
Dino Data Retrieval Grounding Preview
What This Dataset Is
This dataset is a focused retrieval-grounding preview built from four Dino Data capability slices:
search trigger detection
grounded search integration
history search trigger
history search integration
The goal is to train or inspect assistant behavior around two connected problems:
deciding when retrieval or history lookup is needed
generating answers that stay grounded to supplied evidence or prior thread… See the full description on the dataset page: https://huggingface.co/datasets/DinoDS/retrieval_grounding.Philosophical-Triplets-Retrieval
Dataset Card: johnnyboycurtis/Philosophical-Triplets-Retrieval
This dataset is designed for training and evaluating dense retrieval models, specifically for Retrieval Augmented Generation (RAG) systems in complex, subject-matter-heavy domains. It focuses on philosophical texts, demanding high semantic fidelity from retrieval models.
Dataset Details
Feature
Value
Hugging Face Path
johnnyboycurtis/Philosophical-Triplets-Retrieval
Task
Information Retrieval… See the full description on the dataset page: https://huggingface.co/datasets/johnnyboycurtis/Philosophical-Triplets-Retrieval.Indian-Legal-Retrieval-Generation
Indian-Legal-Retrieval-Generation
An expert-verified evaluation set for retrieval-augmented question answering over Indian
court / legal documents. This is the small benchmark used in CourtNav.
Paper: CourtNav: Voice-Guided, Anchor-Accurate Navigation of Long Legal Documents in Courtrooms — Sai Khadloya, Kush Juvekar, Arghya Bhattacharya, Utkarsh Saxena.
Status: work in progress — contents and structure may still evolve.
Overview
21 lawyer-verified question/answer… See the full description on the dataset page: https://huggingface.co/datasets/adalat-ai/Indian-Legal-Retrieval-Generation.passkey-retrieval
ReactiveAI / passkey-retrieval (Interactions Format)
Conversational (in RxLM Interactions Format) retrieval (Passkey / Needle In a Haystack type) dataset, filtered and transformed from grimulkan/passkey-retrieval
Subsets
to-4k - 3-step instruct examples with first (context) message with up to 4k tokens
to-4k-reasoning - 3-step reasoning examples with first (context) query with up to 4k tokens and all the interaction (with reasoning) up to 8k tokens
to-8k - 3-step… See the full description on the dataset page: https://huggingface.co/datasets/ReactiveAI/passkey-retrieval.nanochat-depo-retrieval-copy1-20260715
Nanochat Depo retrieval v1
Each latent 16-node graph yields eight independent, token-aligned, depth-one
query documents. This arm exposes 1 nested edge(s) per
document. Only the answer is supervised in every document; the terminal token
is supervised only for query ordinal 7. This source is separate from and does
not alter Depo-L0 v1.
BuiltBench-retrieval
Data sources
Industry Foundation Classes (IFC) published by buildingSmart International: https://ifc43-docs.standards.buildingsmart.org/
Uniclass product tables published by NBS: https://www.thenbs.com/our-tools/uniclass
License
cc-by-nc-nd-4.0: https://creativecommons.org/licenses/by-nc-nd/4.0/deed.en
How to cite
Research paper on the dataset development and validations: https://arxiv.org/abs/2411.12056
Note: This dataset refers to retrieval-p2p task as… See the full description on the dataset page: https://huggingface.co/datasets/mehrzad-shahin/BuiltBench-retrieval.legalbenchrag_cuad_retrieval_pdf
legalbenchrag_cuad_retrieval_pdf
LegalBenchRAG cuad retrieval dataset
Field
Value
Benchmark
legalbenchrag
Sub-benchmark
cuad
Type
retrieval
Items
4042
Exported from Langfuse.
RLVR-Env-Retrieval-Source-Retrieval-Synthetic-NVDocs-v1
RLVR-Env-Retrieval-Source-Retrieval-Synthetic-NVDocs-v1
RLVR-ready retrieval environment derived from nvidia/Retrieval-Synthetic-NVDocs-v1.
Author: Aman Priyanshu
What Is This
A 100k-row retrieval QA dataset where each row contains a question, ground-truth chunks, and pre-mined distractor chunks (random + semantically similar). Designed for training and evaluating retrieval agents in an RLVR (Reinforcement Learning with Verifiable Rewards) setup — the agent searches… See the full description on the dataset page: https://huggingface.co/datasets/AmanPriyanshu/RLVR-Env-Retrieval-Source-Retrieval-Synthetic-NVDocs-v1.RLVR-Env-Retrieval-Source-code-search-net-javascript
RLVR-Env-Retrieval-Source-code-search-net-javascript
RLVR-ready retrieval environment derived from Nan-Do/code-search-net-javascript.
Author: Aman Priyanshu
What Is This
A 100k-row retrieval QA dataset where each row contains a question, ground-truth chunks, and pre-mined distractor chunks (random + semantically similar). Designed for training and evaluating retrieval agents in an RLVR (Reinforcement Learning with Verifiable Rewards) setup — the agent searches through… See the full description on the dataset page: https://huggingface.co/datasets/AmanPriyanshu/RLVR-Env-Retrieval-Source-code-search-net-javascript.tinyllama-context-retrievallegalbenchrag_contractnli_retrieval_pdf
legalbenchrag_contractnli_retrieval_pdf
LegalBenchRAG contractnli retrieval dataset
Field
Value
Benchmark
legalbenchrag
Sub-benchmark
contractnli
Type
retrieval
Items
977
Exported from Langfuse.
RLVR-Env-Retrieval-Source-code-search-net-python
RLVR-Env-Retrieval-Source-code-search-net-python
RLVR-ready retrieval environment derived from Nan-Do/code-search-net-python.
Author: Aman Priyanshu
What Is This
A 100k-row retrieval QA dataset where each row contains a question, ground-truth chunks, and pre-mined distractor chunks (random + semantically similar). Designed for training and evaluating retrieval agents in an RLVR (Reinforcement Learning with Verifiable Rewards) setup — the agent searches through distractors… See the full description on the dataset page: https://huggingface.co/datasets/AmanPriyanshu/RLVR-Env-Retrieval-Source-code-search-net-python.MoA_Long_RetrievalRetrieval-SFT-Chat
Retrieval-Based Multi-Turn Chat SFT Synthetic Data
A year ago, we released CausalLM/Refined-Anime-Text, a thematic subset of a dataset generated using the then state-of-the-art LLMs. This dataset comprises 1 million entries synthesized through long-context models that rewrote multi-document web text inputs, intended for continued pre-training. We are pleased to note that this data has been employed in various training scenarios and in studies concerning data and internet culture.
In… See the full description on the dataset page: https://huggingface.co/datasets/CausalLM/Retrieval-SFT-Chat.real-math-corpus-questions-with-retrievals
Real Math Corpus - Statement Dependencies and Questions
Dataset Description
This dataset contains a comprehensive collection of mathematical statements and questions extracted from the Real Math Dataset with 207 mathematical papers. The dataset is split into two parts:
Corpus: Statement dependencies and proof dependencies with complete metadata and global ID mapping
Questions: Main statements from papers treated as questions, with dependency mappings to the corpus… See the full description on the dataset page: https://huggingface.co/datasets/AK123321/real-math-corpus-questions-with-retrievals.