datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
natural_questions
Dataset Card for Natural Questions
Dataset Summary
The NQ corpus contains questions from real users, and it requires QA systems to
read and comprehend an entire Wikipedia article that may or may not contain the
answer to the question. The inclusion of real user questions, and the
requirement that solutions should read an entire page to find the answer, cause
NQ to be a more realistic and challenging task than prior QA datasets.
Supported Tasks and Leaderboards… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/natural_questions.nq_open
Dataset Card for nq_open
Dataset Summary
The NQ-Open task, introduced by Lee et.al. 2019,
is an open domain question answering benchmark that is derived from Natural Questions.
The goal is to predict an English answer string for an input English question.
All questions can be answered using the contents of English Wikipedia.
Supported Tasks and Leaderboards
Open Domain Question-Answering,
EfficientQA Leaderboard:… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/nq_open.deepsearchqa
DeepSearchQA
A 900-prompt factuality benchmark from Google DeepMind, designed to evaluate agents on difficult multi-step information-seeking tasks across 17 different fields.
▶ Google DeepMind Release Blog Post▶ DeepSearchQA Leaderboard on Kaggle▶ Technical Report▶ Evaluation Starter Code
Benchmark
DeepSearchQA is a 900-prompt benchmark for evaluating agents on difficult multi-step information-seeking tasks across 17 different fields. Unlike traditional… See the full description on the dataset page: https://huggingface.co/datasets/google/deepsearchqa.xtreme
Dataset Card for "xtreme"
Dataset Summary
The Cross-lingual Natural Language Inference (XNLI) corpus is a crowd-sourced collection of 5,000 test and
2,500 dev pairs for the MultiNLI corpus. The pairs are annotated with textual entailment and translated into
14 languages: French, Spanish, German, Greek, Bulgarian, Russian, Turkish, Arabic, Vietnamese, Thai, Chinese,
Hindi, Swahili and Urdu. This results in 112.5k annotated pairs. Each premise can be associated with the… See the full description on the dataset page: https://huggingface.co/datasets/google/xtreme.tydiqa
Dataset Card for "tydiqa"
Dataset Summary
TyDi QA is a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs.
The languages of TyDi QA are diverse with regard to their typology -- the set of linguistic features that each language
expresses -- such that we expect models performing well on this set to generalize across a large number of the languages
in the world. It contains language phenomena that would not be found in… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/tydiqa.xquad
Dataset Card for "xquad"
Dataset Summary
XQuAD (Cross-lingual Question Answering Dataset) is a benchmark dataset for evaluating cross-lingual question answering
performance. The dataset consists of a subset of 240 paragraphs and 1190 question-answer pairs from the development set
of SQuAD v1.1 (Rajpurkar et al., 2016) together with their professional translations into ten languages: Spanish, German,
Greek, Russian, Turkish, Arabic, Vietnamese, Thai, Chinese, and Hindi.… See the full description on the dataset page: https://huggingface.co/datasets/google/xquad.frames-benchmark
FRAMES: Factuality, Retrieval, And reasoning MEasurement Set
FRAMES is a comprehensive evaluation dataset designed to test the capabilities of Retrieval-Augmented Generation (RAG) systems across factuality, retrieval accuracy, and reasoning.
Our paper with details and experiments is available on arXiv: https://arxiv.org/abs/2409.12941.
Dataset Overview
824 challenging multi-hop questions requiring information from 2-15 Wikipedia articles
Questions span diverse topics… See the full description on the dataset page: https://huggingface.co/datasets/google/frames-benchmark.anle-toaan-gov-vn
Vietnamese Án lệ Corpus — anle.toaan.gov.vn
🇻🇳 Tóm tắt. Bộ dữ liệu các bản án + án lệ Việt Nam thu thập từ cổng
anle.toaan.gov.vn của Tòa án nhân dân tối cao.
Mỗi văn bản đi kèm markdown chuẩn hoá tiếng Việt và một lớp grounding mức câu
(mỗi trích dẫn mang sentence_id + char span trỏ ngược vào markdown). Bộ dữ
liệu là một phần của ViLA common-corpus và ship ba cấu hình HF theo chuẩn
chung: documents (bảng chính) · embeddings (vector 4096-D Nemotron-3-Embed-8B) ·
reduces (toạ… See the full description on the dataset page: https://huggingface.co/datasets/tmquan/anle-toaan-gov-vn.simpleqa-verified
SimpleQA Verified
A 1,000-prompt factuality benchmark from Google DeepMind and Google Research, designed to reliably evaluate LLM parametric knowledge.
▶ SimpleQA Verified Leaderboard on Kaggle▶ Technical Report▶ Evaluation Starter Code
Benchmark
SimpleQA Verified is a 1,000-prompt benchmark for reliably evaluating Large Language Models (LLMs) on short-form factuality
and parametric knowledge. The authors from Google DeepMind and Google Research… See the full description on the dataset page: https://huggingface.co/datasets/google/simpleqa-verified.adaption-charts-p2-gold
Adaption Charts P2 — Gold Chart-QA Dataset
A verified, quality-first chart question-answering dataset built for the
Adaption Labs AutoScientist Challenge (Part 2, Data Visualization track).
Two sources: a programmatically generated synthetic core
(correct-by-construction) and a hand-authored hardset built from real
public dashboards and reports.
At a glance
3803 rows total — 3705 synthetic + 98 hardset
7 chart types — bar, line, grouped_bar, stacked_bar, pie… See the full description on the dataset page: https://huggingface.co/datasets/vinod-anbalagan/adaption-charts-p2-gold.bigbenchThe Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.mgsm-gold
MGSM Gold - Multilingual Grade School Math
This dataset contains the MGSM (Multilingual Grade School Math) benchmark - 250 math word problems translated into 10 languages.
Attribution
This dataset is derived from juletxara/mgsm
Original source: google-research/url-nlp/mgsm
Usage
from datasets import load_dataset
# Load German test set
dataset = load_dataset("alibashir/mgsm-gold", "de")
print(dataset["test"][0])
Languages
Code
Language… See the full description on the dataset page: https://huggingface.co/datasets/alibashir/mgsm-gold.mytown-local-gov-meetings
MyTown — open dataset of US & Canadian local-government meetings
The documents themselves, not just the metadata. Most civic datasets publish meeting
titles, dates and links. This one publishes 2,109,683 full text extractions of
the primary documents — the actual agendas and minutes, pulled out of the PDFs — alongside
11,949,495 per-member roll-call votes and 61,661,080 campaign-finance
transactions, all joinable on the same keys.
That combination is the point: you can go from… See the full description on the dataset page: https://huggingface.co/datasets/jazzypajamas/mytown-local-gov-meetings.tydiqa-goldpTyDi QA is a question answering dataset covering 11 typologically diverse languages with 204K question-answer pairs.
The languages of TyDi QA are diverse with regard to their typology -- the set of linguistic features that each language
expresses -- such that we expect models performing well on this set to generalize across a large number of the languages
in the world. It contains language phenomena that would not be found in English-only corpora. To provide a realistic
information-seeking task and avoid priming effects, questions are written by people who want to know the answer, but
don’t know the answer yet, (unlike SQuAD and its descendents) and the data is collected directly in each language without
the use of translation (unlike MLQA and XQuAD).govbench
GovBench — items, tooling, and every experiment that refuted us
Canon note (2026-08-18). This public snapshot carries 193 items. The frozen,
signed governance bank in the estate corpus holds 237 (the extra items are the
held-out grading splits). When you see both numbers, this is why. Canonical anchor:
10.5281/zenodo.21991104.
193 items · 26 dimensions · 10 models · every item carries grading criteria.
⚠️ Read this before ranking anything
0 of 15 scored… See the full description on the dataset page: https://huggingface.co/datasets/Nicholastempleman/govbench.FACTS-grounding-public
FACTS Grounding 1.0 Public Examples
860 public FACTS Grounding examples from Google DeepMind and Google Research
FACTS Grounding is a benchmark from Google DeepMind and Google Research designed to measure the performance of AI Models on factuality and grounding.
▶ FACTS Grounding Leaderboard on Kaggle▶ Technical Report▶ Evaluation Starter Code▶ Google DeepMind Blog Post
Usage
The FACTS Grounding benchmark evaluates the ability of Large Language Models (LLMs)… See the full description on the dataset page: https://huggingface.co/datasets/google/FACTS-grounding-public.spiqa
SPIQA Dataset Card
Dataset Details
Dataset Name: SPIQA (Scientific Paper Image Question Answering)
Paper: SPIQA: A Dataset for Multimodal Question Answering on Scientific Papers
Github: SPIQA eval and metrics code repo
Dataset Summary: SPIQA is a large-scale and challenging QA dataset focused on figures, tables, and text paragraphs from scientific research papers in various computer science domains. The figures cover a wide variety of plots… See the full description on the dataset page: https://huggingface.co/datasets/google/spiqa.phapdien-moj-gov-vn
Bộ Pháp Điển Việt Nam — phapdien.moj.gov.vn
🇻🇳 Tóm tắt. Bộ ngữ liệu cấp Điều của Bộ Pháp Điển Việt Nam — bộ pháp điển
chính thức do Bộ Tư pháp công bố. Mỗi dòng documents là một Điều kèm toàn văn đã
chuẩn hoá, chương sở thuộc, đề mục và chủ đề. Kèm theo là vector nhúng ngữ nghĩa 4096-D
(embeddings), toạ độ giảm chiều trong không gian chung ViLA (reduces), và từ điển
ontology song ngữ Việt–Anh (chủ đề · đề mục · thuật ngữ).
🇬🇧 One-line. Article-level corpus of the Bộ Pháp… See the full description on the dataset page: https://huggingface.co/datasets/tmquan/phapdien-moj-gov-vn.unpredictable_support-google-comThe UnpredicTable dataset consists of web tables formatted as few-shot tasks for fine-tuning language models to improve their few-shot performance. For more details please see the accompanying dataset card.gspc-gov
GSPC — governance bank (GovBench)
Bank (governance). Frozen split. Live n is the governance row on GET https://councilof.ai/api/gspc, not a Hub leaderboard score. Not a certificate.
Art 50 dates (EUR-Lex): 2 August 2026 live; marking grace 2 December 2026.
Council of AI measurement bank. Measurement, not certification.
Live measurement. This bank stands behind the governance row of the live GSPC board: GET https://councilof.ai/api/gspc?axis=governance (family, kind, status and… See the full description on the dataset page: https://huggingface.co/datasets/csoai/gspc-gov.IndicGenBench_xquad_in
Dataset Card for Dataset Name
This repository contains the XQuAD-IN dataset released as a part of the paper "IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages"
Paper Link: https://arxiv.org/abs/2404.16816
Dataset Details
Overview
IndicGenBench is a multilingual, multi-way parallel benchmark for measuring language generation capabilities across diverse user-facing tasks in 29 Indic languages spanning 13… See the full description on the dataset page: https://huggingface.co/datasets/google/IndicGenBench_xquad_in.SQACThis dataset contains 6,247 contexts and 18,817 questions with their answers, 1 to 5 for each fragment.
The sources of the contexts are:
* Encyclopedic articles from [Wikipedia in Spanish](https://es.wikipedia.org/), used under [CC-by-sa licence](https://creativecommons.org/licenses/by-sa/3.0/legalcode).
* News from [Wikinews in Spanish](https://es.wikinews.org/), used under [CC-by licence](https://creativecommons.org/licenses/by/2.5/).
* Text from the Spanish corpus [AnCora](http://clic.ub.edu/corpus/en), which is a mix from diferent newswire and literature sources, used under [CC-by licence] (https://creativecommons.org/licenses/by/4.0/legalcode).
This dataset can be used to build extractive-QA.reddit_dataset_2025
Bittensor Subnet 13 Reddit Dataset
Miner Data Compliance Agreement
In uploading this dataset, I am agreeing to the Macrocosmos Miner Data Compliance Policy.
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed Reddit data. The data is continuously updated by network miners, providing a real-time stream of Reddit content for various analytical and machine learning tasks.
For more… See the full description on the dataset page: https://huggingface.co/datasets/goldentraversy07/reddit_dataset_2025.cbba-toaan-gov-vn
Vietnamese Bản án Corpus — congbobanan.toaan.gov.vn
🇻🇳 Tóm tắt. Bản án sơ thẩm/phúc thẩm/giám đốc thẩm/tái thẩm của Việt Nam,
thu thập từ cổng công bố bản án
congbobanan.toaan.gov.vn của Tòa án nhân
dân tối cao. Ba cấu hình HF khoá theo doc_name/id: documents (nội dung
siêu dữ liệu + trích dẫn), embeddings (vector 4096-D Nemotron-3-8B),
reduces (toạ độ t-SNE/UMAP trong không gian chung 6 bộ dữ liệu). Tên
cột và giá trị phân loại bằng tiếng Anh; chỉ nội dung pháp lý giữ tiếng… See the full description on the dataset page: https://huggingface.co/datasets/tmquan/cbba-toaan-gov-vn.gspc-jail-goldbank
GSPC — jail bank (GoldBank-Detector)
Council of AI measurement bank. Measurement, not certification.
Bank. Frozen split. Live n is the matching axis on GET https://councilof.ai/api/gspc, not a Hub score. Not a certificate. Art 50 (EUR-Lex): 2 August 2026 live; marking grace 2 December 2026.
Live measurement. This bank stands behind the jail row of the live GSPC board: GET https://councilof.ai/api/gspc?axis=jail (family, kind, status and n are on that row, never typed here; the… See the full description on the dataset page: https://huggingface.co/datasets/csoai/gspc-jail-goldbank.cfq
Dataset Card for "cfq"
Dataset Summary
The Compositional Freebase Questions (CFQ) is a dataset that is specifically designed to measure compositional
generalization. CFQ is a simple yet realistic, large dataset of natural language questions and answers that also
provides for each question a corresponding SPARQL query against the Freebase knowledge base. This means that CFQ can
also be used for semantic parsing.
Supported Tasks and Leaderboards
More Information… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/cfq.qedQED, is a linguistically informed, extensible framework for explanations in question answering. A QED explanation specifies the relationship between a question and answer according to formal semantic notions such as referential equality, sentencehood, and entailment. It is an expertannotated dataset of QED explanations built upon a subset of the Google Natural Questions dataset.unpredictable_support-google-comThe UnpredicTable dataset consists of web tables formatted as few-shot tasks for fine-tuning language models to improve their few-shot performance. For more details please see the accompanying dataset card.M3D-VQA
Dataset Description
Large-Scale 3D Medical Multi-Modal Dataset - Visual Question Answering Dataset (M3D-VQA), consisting of 510K data.
Dataset Introduction
Acquiring Visual Question Answering (VQA) data in medical scenarios is costly, mainly due to the requirement for specialized medical knowledge.
To mitigate these costs, we use publicly available large language models (LLMs) to analyze existing diagnostic reports and generate VQA data.
Additionally, we eliminate noisy… See the full description on the dataset page: https://huggingface.co/datasets/GoodBaiBai88/M3D-VQA.WikiProfile
WikiProfile
WikiProfile is a factual knowledge benchmark for evaluating how well language models encode and recall factual knowledge. It comprises 2,150 facts, each paired with 10 questions, for a total of 21,500 question instances.
Each fact is grounded in the first paragraph (summary) of an English Wikipedia page and is defined as a proposition between two entities, a subject and an object (e.g., "Oasis played their first gig at the Boardwalk club" → subject: Oasis, object:… See the full description on the dataset page: https://huggingface.co/datasets/google/WikiProfile.