datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
IFEval
Dataset Card for IFEval
Dataset Summary
This dataset contains the prompts used in the Instruction-Following Eval (IFEval) benchmark for large language models. It contains around 500 "verifiable instructions" such as "write in more than 400 words" and "mention the keyword of AI at least 3 times" which can be verified by heuristics. To load the dataset, run:
from datasets import load_dataset
ifeval = load_dataset("google/IFEval")
Supported Tasks and… See the full description on the dataset page: https://huggingface.co/datasets/google/IFEval.anle-toaan-gov-vn
Vietnamese Án lệ Corpus — anle.toaan.gov.vn
🇻🇳 Tóm tắt. Bộ dữ liệu các bản án + án lệ Việt Nam thu thập từ cổng
anle.toaan.gov.vn của Tòa án nhân dân tối cao.
Mỗi văn bản đi kèm markdown chuẩn hoá tiếng Việt và một lớp grounding mức câu
(mỗi trích dẫn mang sentence_id + char span trỏ ngược vào markdown). Bộ dữ
liệu là một phần của ViLA common-corpus và ship ba cấu hình HF theo chuẩn
chung: documents (bảng chính) · embeddings (vector 4096-D Nemotron-3-Embed-8B) ·
reduces (toạ… See the full description on the dataset page: https://huggingface.co/datasets/tmquan/anle-toaan-gov-vn.govreport-summarization
GovReport dataset for summarization
Dataset for summarization of long documents.Adapted from this repo and this paperThis dataset is compatible with the run_summarization.py script from Transformers if you add this line to the summarization_name_mapping variable:
"ccdv/govreport-summarization": ("report", "summary")
Data Fields
id: paper id
report: a string containing the body of the reportsummary: a string containing the summary of the report
Data Splits… See the full description on the dataset page: https://huggingface.co/datasets/ccdv/govreport-summarization.SWE-smith-go
SWE-smith Dataset
Code
•
Paper
•
Site
As of 12/14/202, SWE-smith: Golang contains 8212 task instances from 87 GitHub repositories
The SWE-smith Dataset is the largest open source dataset for training software engineering agents.
All SWE-smith task instances come with an executable environment.
To learn more about how to use this dataset to train Language Models for Software Engineering, please refer to the documentation.
open_government
Open Government Dataset
Open Government is the largest agregation of governement text and data made available as part of open data programs.
In total, the dataset contains approximately 380B tokens. While Open Government aims to become a global resource, in its current state it mostly features open datasets from the US, France, European and international organizations.
The dataset comprises 16 collections curated through two different initiaties: Finance commons and Legal commons.… See the full description on the dataset page: https://huggingface.co/datasets/AgentPublic/open_government.mgsm-gold
MGSM Gold - Multilingual Grade School Math
This dataset contains the MGSM (Multilingual Grade School Math) benchmark - 250 math word problems translated into 10 languages.
Attribution
This dataset is derived from juletxara/mgsm
Original source: google-research/url-nlp/mgsm
Usage
from datasets import load_dataset
# Load German test set
dataset = load_dataset("alibashir/mgsm-gold", "de")
print(dataset["test"][0])
Languages
Code
Language… See the full description on the dataset page: https://huggingface.co/datasets/alibashir/mgsm-gold.google-code-archive
Google Code Archive Dataset
Dataset Description
This dataset was compiled from the Google Code Archive, a preserved snapshot of projects hosted on Google Code, Google's open-source project hosting service that operated from 2006 to 2016. Google Code was one of the major code hosting platforms of its era, hosting hundreds of thousands of open-source projects before its shutdown. The archive provides a unique historical record of open-source development during a formative… See the full description on the dataset page: https://huggingface.co/datasets/nyuuzyou/google-code-archive.msi-corpus
Main Street Independent Corpus
The complete Main Street Independent archive as open data: 917 news articles and 0 opinion columns, each with full text and metadata. AI-generated news and opinion, dedicated to the public domain under CC0 — no rights reserved. Refreshed daily.
Load it
from datasets import load_dataset
ds = load_dataset("golfplan18/msi-corpus") # 'news' and 'opinion' splits
print(ds["news"][0]["headline"], ds["news"][0]["text"][:200])… See the full description on the dataset page: https://huggingface.co/datasets/golfplan18/msi-corpus.BetterDataset-12M
Dataset
Mixed pretraining dataset built from:
Source
Config
Weight
HuggingFaceTB/smollm-corpus
fineweb-edu-dedup
20%
openbmb/Ultra-FineWeb-L3
Ultra-FineWeb-L3-en-Multi-Style-Synthetic
10%
HuggingFaceTB/dclm-edu
—
20%
HuggingFaceFW/finewiki
en
20%
HuggingFaceTB/cosmopedia
stories
2%
HuggingFaceTB/cosmopedia
stanford
2%
HuggingFaceFW/finephrase
all
6%
HuggingFaceTB/finemath
finemath-l4
5%
nampdn-ai/tiny-math-textbooks
—
5%
HuggingFaceTB/cosmopedia… See the full description on the dataset page: https://huggingface.co/datasets/GODELEV/BetterDataset-12M.phapdien-moj-gov-vn
Bộ Pháp Điển Việt Nam — phapdien.moj.gov.vn
🇻🇳 Tóm tắt. Bộ ngữ liệu cấp Điều của Bộ Pháp Điển Việt Nam — bộ pháp điển
chính thức do Bộ Tư pháp công bố. Mỗi dòng documents là một Điều kèm toàn văn đã
chuẩn hoá, chương sở thuộc, đề mục và chủ đề. Kèm theo là vector nhúng ngữ nghĩa 4096-D
(embeddings), toạ độ giảm chiều trong không gian chung ViLA (reduces), và từ điển
ontology song ngữ Việt–Anh (chủ đề · đề mục · thuật ngữ).
🇬🇧 One-line. Article-level corpus of the Bộ Pháp… See the full description on the dataset page: https://huggingface.co/datasets/tmquan/phapdien-moj-gov-vn.code_x_glue_cc_code_completion_token
Dataset Card for "code_x_glue_cc_code_completion_token"
Dataset Summary
CodeXGLUE CodeCompletion-token dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/CodeCompletion-token
Predict next code token given context of previous tokens. Models are evaluated by token level accuracy.
Code completion is a one of the most widely used features in software development through IDEs. An effective code completion tool could improve software… See the full description on the dataset page: https://huggingface.co/datasets/google/code_x_glue_cc_code_completion_token.unpredictable_support-google-comThe UnpredicTable dataset consists of web tables formatted as few-shot tasks for fine-tuning language models to improve their few-shot performance. For more details please see the accompanying dataset card.gspc-gov
GSPC — governance bank (GovBench)
Bank (governance). Frozen split. Live n is the governance row on GET https://councilof.ai/api/gspc, not a Hub leaderboard score. Not a certificate.
Art 50 dates (EUR-Lex): 2 August 2026 live; marking grace 2 December 2026.
Council of AI measurement bank. Measurement, not certification.
Live measurement. This bank stands behind the governance row of the live GSPC board: GET https://councilof.ai/api/gspc?axis=governance (family, kind, status and… See the full description on the dataset page: https://huggingface.co/datasets/csoai/gspc-gov.reddit_dataset_2025
Bittensor Subnet 13 Reddit Dataset
Miner Data Compliance Agreement
In uploading this dataset, I am agreeing to the Macrocosmos Miner Data Compliance Policy.
Dataset Summary
This dataset is part of the Bittensor Subnet 13 decentralized network, containing preprocessed Reddit data. The data is continuously updated by network miners, providing a real-time stream of Reddit content for various analytical and machine learning tasks.
For more… See the full description on the dataset page: https://huggingface.co/datasets/goldentraversy07/reddit_dataset_2025.govdocs1-by-extension
govdocs1 Dataset: By File Extension
[!NOTE]
PDFs from govdocs1 are at this repo in "raw" file form - no simple "mostly correct" way to convert to text
Markdown-parsed versions of documents in govdocs1 with light filtering.
Usage
Load specific file formats (e.g., .doc files) parsed to markdown with pandoc:
from datasets import load_dataset
# Replace "doc" with desired config name
dataset = load_dataset("BEE-spoke-data/govdocs1-by-extension", "doc")… See the full description on the dataset page: https://huggingface.co/datasets/BEE-spoke-data/govdocs1-by-extension.cbba-toaan-gov-vn
Vietnamese Bản án Corpus — congbobanan.toaan.gov.vn
🇻🇳 Tóm tắt. Bản án sơ thẩm/phúc thẩm/giám đốc thẩm/tái thẩm của Việt Nam,
thu thập từ cổng công bố bản án
congbobanan.toaan.gov.vn của Tòa án nhân
dân tối cao. Ba cấu hình HF khoá theo doc_name/id: documents (nội dung
siêu dữ liệu + trích dẫn), embeddings (vector 4096-D Nemotron-3-8B),
reduces (toạ độ t-SNE/UMAP trong không gian chung 6 bộ dữ liệu). Tên
cột và giá trị phân loại bằng tiếng Anh; chỉ nội dung pháp lý giữ tiếng… See the full description on the dataset page: https://huggingface.co/datasets/tmquan/cbba-toaan-gov-vn.godot-gdscript-dataset
Godot GDscript Code Dataset
This dataset contains GDScript code from 5k+ github repositories. Data from each repo has been extracted into a text file. Each text file contains the code from all .gd files & README.md text (if the README was not empty in the original repo).
Original forum post:
https://diffused.to/Thread-Godot-GDscript-Code-Dataset-5k
Dataset collection date
June 2025
Dataset structure:
📂 files/
├── repo-name-1.txt
├── repo-name-2.txt… See the full description on the dataset page: https://huggingface.co/datasets/wallstoneai/godot-gdscript-dataset.gspc-jail-goldbank
GSPC — jail bank (GoldBank-Detector)
Council of AI measurement bank. Measurement, not certification.
Bank. Frozen split. Live n is the matching axis on GET https://councilof.ai/api/gspc, not a Hub score. Not a certificate. Art 50 (EUR-Lex): 2 August 2026 live; marking grace 2 December 2026.
Live measurement. This bank stands behind the jail row of the live GSPC board: GET https://councilof.ai/api/gspc?axis=jail (family, kind, status and n are on that row, never typed here; the… See the full description on the dataset page: https://huggingface.co/datasets/csoai/gspc-jail-goldbank.MathOlympiadBenchThis repository contains the MathOlympiadBench dataset, which is introduced in the paper Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction.
Project Page: https://blog.goedel-prover.com
Code Repository: https://github.com/Goedel-LM/Goedel-Prover-V2
MathOlympiadBench (Math Olympiad) comprises human-verified formalizations of Olympiad-level mathematical competition problems, sourced from Compfiles and IMOSLLean4 repository. MathOlympiadBench… See the full description on the dataset page: https://huggingface.co/datasets/Goedel-LM/MathOlympiadBench.fattah-golden-superset
Fattah Golden
Fattah Golden is a large-scale, model-agnostic supervised fine-tuning (SFT) superset built by Nomeda Labs to train the Fattah family of coding and agentic coding models.
The dataset is designed as a labeled superset with no baked-in training ratios. This means the stored dataset is the complete cleaned and annotated corpus. Researchers and practitioners choose their own mixture at training time by filtering on the boolean capability columns.
Stats… See the full description on the dataset page: https://huggingface.co/datasets/nomeda-lab/fattah-golden-superset.google-code-archive
Google Code Archive Dataset
Dataset Description
This dataset was compiled from the Google Code Archive, a preserved snapshot of projects hosted on Google Code, Google's open-source project hosting service that operated from 2006 to 2016. Google Code was one of the major code hosting platforms of its era, hosting hundreds of thousands of open-source projects before its shutdown. The archive provides a unique historical record of open-source development during a formative… See the full description on the dataset page: https://huggingface.co/datasets/Carrillo16/google-code-archive.unpredictable_support-google-comThe UnpredicTable dataset consists of web tables formatted as few-shot tasks for fine-tuning language models to improve their few-shot performance. For more details please see the accompanying dataset card.google-code-archive
Google Code Archive Dataset
Dataset Description
This dataset was compiled from the Google Code Archive, a preserved snapshot of projects hosted on Google Code, Google's open-source project hosting service that operated from 2006 to 2016. Google Code was one of the major code hosting platforms of its era, hosting hundreds of thousands of open-source projects before its shutdown. The archive provides a unique historical record of open-source development during a formative… See the full description on the dataset page: https://huggingface.co/datasets/0xzanuee/google-code-archive.google-code-archive
Google Code Archive Dataset
Dataset Description
This dataset was compiled from the Google Code Archive, a preserved snapshot of projects hosted on Google Code, Google's open-source project hosting service that operated from 2006 to 2016. Google Code was one of the major code hosting platforms of its era, hosting hundreds of thousands of open-source projects before its shutdown. The archive provides a unique historical record of open-source development during a formative… See the full description on the dataset page: https://huggingface.co/datasets/Mgmgrand420/google-code-archive.code_x_glue_cc_code_completion_line
Dataset Card for "code_x_glue_cc_code_completion_line"
Dataset Summary
CodeXGLUE CodeCompletion-line dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/CodeCompletion-line
Complete the unfinished line given previous context. Models are evaluated by exact match and edit similarity.
We propose line completion task to test model's ability to autocomplete a line. Majority code completion systems behave well in token level completion, but fail in… See the full description on the dataset page: https://huggingface.co/datasets/google/code_x_glue_cc_code_completion_line.go-swe-bench-v0
go_swe_bench v0 — real Go bug fixes, verified by the Go toolchain
246 tasks from 79 real Go repositories. Each task is a bug-fix commit whose co-committed test is red on the
parent and green on the fix. No LLM anywhere in the build.
Mined on 2026-09-19 from the GuildLM Go mining pipeline by inverting the filter that had thrown the tests
away (the pipeline was built for SFT data; a benchmark needs the opposite). Every task was verified twice
with go test: green at the commit (≥ 1… See the full description on the dataset page: https://huggingface.co/datasets/guildlm/go-swe-bench-v0.Goodreads-Books
Dataset Card for "BrightData/Goodreads-Books"
Dataset Summary
Explore a collection of millions of books with the Goodreads dataset, comprising over 6.3M structured records and 14 data fields updated and refreshed regularly.
Each entry includes all major data points such as URLs, book IDs, titles, authors, ratings, number of ratings, reviews, summaries, genres, publication dates, author details and prices.
For a complete list of data points, please refer to the full "Data… See the full description on the dataset page: https://huggingface.co/datasets/BrightData/Goodreads-Books.air_dialogue
Dataset Card for air_dialogue
Dataset Summary
AirDialogue, is a large dataset that contains 402,038 goal-oriented conversations. To collect this dataset, we create a contextgenerator which provides travel and flight restrictions. Then the human annotators are asked to play the role of a customer or an agent and interact with the goal of successfully booking a trip given the restrictions.
News in v1.3:
We have included the test split of the AirDialogue dataset.
We have… See the full description on the dataset page: https://huggingface.co/datasets/google/air_dialogue.wildchat-mixed-1k
wildchat-mixed-1k
Real-world chat requests for end-to-end LLM inference benchmarking in fastkernels — Scenario A, the bulk-throughput workload used to saturate continuous batching with a realistic mix of short/long prompts and short/long responses.
What it's for
One dataset that replaces separate prefill-heavy / balanced / decode-heavy splits: its natural length distribution puts prefill-bound and decode-bound requests in the same batch, so a single run yields a… See the full description on the dataset page: https://huggingface.co/datasets/sfc-gh-goliaro/wildchat-mixed-1k.code_x_glue_cc_cloze_testing_all
Dataset Card for "code_x_glue_cc_cloze_testing_all"
Dataset Summary
CodeXGLUE ClozeTesting-all dataset, available at https://github.com/microsoft/CodeXGLUE/tree/main/Code-Code/ClozeTesting-all
Cloze tests are widely adopted in Natural Languages Processing to evaluate the performance of the trained language models. The task is aimed to predict the answers for the blank with the context of the blank, which can be formulated as a multi-choice classification problem.
Here we… See the full description on the dataset page: https://huggingface.co/datasets/google/code_x_glue_cc_cloze_testing_all.