datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fineweb-edu-fortified
Fineweb-Edu-Fortified
The composition of fineweb-edu-fortified, produced by automatically clustering a 500k row sample in
Airtrain
What is it?
Fineweb-Edu-Fortified is a dataset derived from
Fineweb-Edu by applying exact-match
deduplication across the whole dataset and producing an embedding for each row. The number of times
the text from each row appears is also included as a count column. The embeddings were produced
using TaylorAI/bge-micro
Fineweb and… See the full description on the dataset page: https://huggingface.co/datasets/airtrain-ai/fineweb-edu-fortified.epoch_ai_swebench_verified
Epoch AI SWE-bench Verified Traces
Complete public trace archives and an analysis-ready Parquet conversion of Epoch AI's SWE-bench Verified evaluations.
Contents
34 published evaluation runs covering 16,456 traces (484 SWE-bench instances per run).
data/: loadable Parquet data, one exact trace per row.
original/: the byte-identical .eval archives published by Epoch AI.
run_metadata/: non-sample files from each .eval archive (header.json, summaries, reductions… See the full description on the dataset page: https://huggingface.co/datasets/naderalfares/epoch_ai_swebench_verified.ai-ecosystem-daily
TensorFeed AI Ecosystem Daily
Daily snapshots of the AI ecosystem: news, model pricing, benchmarks, service status, GPU rental prices, MCP registry growth, LLM endpoint latency probes, agent traffic, and the AFTA adopter directory. Captured once per day from the public tensorfeed.ai API and committed to this repo as JSONL.
Each daily snapshot lives in a YYYY-MM-DD/ subfolder with one JSONL file per feed plus a manifest.json summarizing what was captured.
What's in… See the full description on the dataset page: https://huggingface.co/datasets/tensorfeed/ai-ecosystem-daily.indic-align
IndicAlign
A diverse collection of Instruction and Toxic alignment datasets for 14 Indic Languages. The collection comprises of:
IndicAlign - Instruct
Indic-ShareLlama
Dolly-T
OpenAssistant-T
WikiHow
IndoWordNet
Anudesh
Wiki-Conv
Wiki-Chat
IndicAlign - Toxic
HHRLHF-T
Toxic-Matrix
We use IndicTrans2 (Gala et al., 2023) for the translation of the datasets.
We recommend the readers to check out our paper on Arxiv for detailed information on the curation process of these… See the full description on the dataset page: https://huggingface.co/datasets/ai4bharat/indic-align.panda-bench
PandaBench
PandaBench is a comprehensive benchmark for evaluating Large Language Model (LLM) safety, focusing on jailbreak attacks, defense mechanisms, and evaluation methodologies.
The PandaGuard framework architecture illustrating the end-to-end pipeline for LLM safety evaluation. The system connects three key components: Attackers, Defenders, and Judges.
Dataset Description
This repository contains the benchmark results from extensive evaluations of various… See the full description on the dataset page: https://huggingface.co/datasets/Beijing-AISI/panda-bench.fulg
❄️FuLG
The FuLG dataset is a comprehensive Romanian language corpus comprising 150 billion tokens, carefully
extracted from Common Crawl. This extensive dataset is the result of rigorous filtering and deduplication
processes applied to 95 Common Crawl snapshots. The compressed dataset has 289 GB.
For more details, check the arXiv preprint.
How do I download this?
Using 🤗 Datasets
from datasets import load_dataset
# Full dataset
dataset =… See the full description on the dataset page: https://huggingface.co/datasets/faur-ai/fulg.AIME_2000_2026_Kimi_K3
AIME 2000–2026 — Kimi K3 reasoning traces
🔄 Changelog
2026-08-08 — full re-generation. All reasoning traces were regenerated from scratch and re-verified against the official answer key.
New schema — added gen_attempts_low, gen_attempts_high; renamed gen_parsed_answer → gen_answer_int and answer_note → problem_note; removed gen_effort, gen_pass1.
New generation — only use the bare problem (v1 appended an "ANSWER:" format instruction), so traces are cleaner.… See the full description on the dataset page: https://huggingface.co/datasets/bevangelista/AIME_2000_2026_Kimi_K3.xRouteBench
xRouteBench — LLM Routing Benchmark
xRouteBench is a benchmark for training and evaluating LLM routers — systems that pick the best LLM from a candidate pool for each incoming query, trading off performance vs. price cost.
Every query in each scenario was executed against all 18 candidate LLMs, recording each model's response, task performance, token usage, and latency. A router learns from the train split which model to pick, and is evaluated on test.
Scenarios… See the full description on the dataset page: https://huggingface.co/datasets/ulab-ai/xRouteBench.Sujet-Finance-Instruct-177k
Sujet Finance Dataset Overview
The Sujet Finance dataset is a comprehensive collection designed for the fine-tuning of Language Learning Models (LLMs) for specialized tasks in the financial sector. It amalgamates data from 18 distinct datasets hosted on HuggingFace, resulting in a rich repository of 177,597 entries. These entries span across seven key financial LLM tasks, making Sujet Finance a versatile tool for developing and enhancing financial applications of AI.… See the full description on the dataset page: https://huggingface.co/datasets/sujet-ai/Sujet-Finance-Instruct-177k.carbon-tokenized-corpus
Dataset Summary
AINovice2005/carbon-tokenized-corpus is the tokenized sample of AINovice2005/carbon-cpu-enriched-sequences-sampled .
Schema
The current dataset contains the following fields:
Field
Type
Description
record_id
string
Source/reference sequence identifier
start
int64
Start coordinate of the sequence interval
end
int64
End coordinate of the sequence interval
token_ids
list
Integer token IDs produced by the tokenizer
token_mask
list… See the full description on the dataset page: https://huggingface.co/datasets/AINovice2005/carbon-tokenized-corpus.DecodingTrust
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Overview
This repo contains the source code of DecodingTrust. This research endeavor is designed to help researchers better understand the capabilities, limitations, and potential risks associated with deploying these state-of-the-art Large Language Models (LLMs). See our paper for details.
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Boxin Wang, Weixin Chen, Hengzhi… See the full description on the dataset page: https://huggingface.co/datasets/AI-Secure/DecodingTrust.RepoExec
RepoExec: Evaluate Code Generation with a Repository-Level Executable Benchmark
Dataset Summary
RepoExec is a novel benchmark designed to evaluate code generation at the repository level with a focus on executability and correctness. This benchmark addresses the gaps in existing systems by emphasizing real-world applicability and providing a comprehensive assessment of code functionality. It aims to provide a comprehensive evaluation of code functionality and alignment… See the full description on the dataset page: https://huggingface.co/datasets/Fsoft-AIC/RepoExec.python-text-copilot-training-instruct-ai-research-2024-02-03
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Agora Open Source AI Research Lab:
Agora GitHub Organization
Agora Hugging Face
This dataset is the 2024-02-03 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/matlok/python-text-copilot-training-instruct-ai-research-2024-02-03.qwen3.8-max-glm5.2-kimi-k3-distillation
Multi-Teacher Distillation Dataset (57,937 traces)
A quality-filtered, deduplicated, multi-teacher SFT corpus combining traces from three frontier models across math, code, reasoning, instruction-following, tool-use, science, long-context, multilingual, and creative dialogue domains.
Teachers
Teacher
Provider
Traces
Qwen3.8-Max-Preview
Alibaba Cloud Model Studio
48,283
GLM-5.2
Z.AI Coding Plan
5,307
Kimi Code K3
Moonshot AI (Kimi)
4,347… See the full description on the dataset page: https://huggingface.co/datasets/inferenceport-ai/qwen3.8-max-glm5.2-kimi-k3-distillation.carbon-cpu-enriched-sequences
carbon-cpu-enriched-sequences
A CPU-enriched subset of the carbon pretraining corpus (eukaryote_generator), combining original source fields with normalized sequences
and row-level features for quality analysis, GPU enrichment and embedding generation.
Information of Features
Feature
Type
Description
record_id
string
NCBI Identifier linking the row back to the source genomic record. It provides the primary record-level identity.
begin_of_sequence… See the full description on the dataset page: https://huggingface.co/datasets/AINovice2005/carbon-cpu-enriched-sequences.carbon-likelihood-stats
Dataset Summary
Dataset Summary
AINovice2005/carbon-likelihood-stats is a model-enriched dataset derived from the Carbon genomic corpus, containing sequence-level likelihood statistics for 251,427 sequence regions.
Each row links a source sequence record to its corresponding region in the processed/tokenized Carbon corpus using record_id, start, and end. The dataset summarizes both the model's overall likelihood of the observed sequence and the variation in… See the full description on the dataset page: https://huggingface.co/datasets/AINovice2005/carbon-likelihood-stats.tiny-textbooks
Textbook-like Dataset: A High-Quality Resource for Small Language Models
The idea is simply inspired by the Textbooks Are All You Need II: phi-1.5 technical report paper. The source texts in this dataset have been gathered and carefully select the best of the falcon-refinedweb and minipile datasets to ensure the diversity, quality while tiny in size. The dataset was synthesized using 4x3090 Ti cards over a period of 500 hours, thanks to Nous-Hermes-Llama2-13b finetuned model.
Why… See the full description on the dataset page: https://huggingface.co/datasets/nampdn-ai/tiny-textbooks.danbooru-tags-2024
danbooru-tags-2024
from datasets import load_dataset
ds = load_dataset(
"isek-ai/danbooru-tags-2024",
# revision="202412-at20250122", # optional
split="train",
)
Last updated: since 2005 to 2024/12/31, collected at 2025/01/22
AgentJudgeBench
AgentJudgeBench: Evaluating LLM Judge Reliability on Agentic Tool-Calling
A benchmark for systematically evaluating how reliably LLM judges assess
agentic tool-calling workflows across structured, dependency-driven tasks.
Why this benchmark?
AgentJudgeBench measures how reliably LLM judges assess agentic tool-calling outputs. It provides 3,808 benchmark records spanning six DAG topologies and three difficulty… See the full description on the dataset page: https://huggingface.co/datasets/ServiceNow-AI/AgentJudgeBench.smol-worldcup
🏟️ Smol AI WorldCup — SHIFT Benchmark
The world's first 5-axis evaluation framework for small language models.
Not just "how smart?" — but "how honest? how fast? how small? how efficient?"
🏟️ Leaderboard
huggingface.co/spaces/ginigen-ai/smol-worldcup
📊 Dataset
huggingface.co/datasets/ginigen-ai/smol-worldcup
🏅 ALL Bench
huggingface.co/spaces/FINAL-Bench/all-bench-leaderboard
🏆 Official Ranking: WCS (WorldCup Score)
WCS = √( SHIFT × PIR_norm )… See the full description on the dataset page: https://huggingface.co/datasets/ginigen-ai/smol-worldcup.python-text-training-instruct-ai
Python Copilot Instructions on How to Code using Alpaca and Yaml
Training and test datasets for building coding multimodal models that understand how to use the open source GitHub projects for the Agora Open Source AI Research Lab:
Agora GitHub Organization
Agora Hugging Face
This dataset is the 2024-02-03 update for the matlok python copilot datasets. Please refer to the Multimodal Python Copilot Training Overview for more details on how to use this dataset.
Details… See the full description on the dataset page: https://huggingface.co/datasets/DevShubham/python-text-training-instruct-ai.r15-ai-search-metamerism
R15: AI Search Metamerism — Cross-Cultural Brand Perception Dataset
Citation: Zharnikov, D. (2026v) | DOI: 10.5281/zenodo.19422427 | Version: v3.2.0
Dataset Summary
This dataset contains the full session logs, aggregated results, and analysis outputs from the R15 large-scale experiment testing whether Large Language Models systematically collapse multi-dimensional brand perception into Economic and Experiential dimensions ("spectral metamerism"). It comprises 21… See the full description on the dataset page: https://huggingface.co/datasets/spectralbranding/r15-ai-search-metamerism.RuHeritage-Corpus
RuHeritage-Corpus
🇬🇧 English Description
RuHeritage-Corpus is a high-quality, curated dataset of Russian classical literature, specifically designed for the pre-training and continued pre-training (CPT) of Large Language Models (LLMs).
The corpus focuses on the Golden and Silver Ages of Russian literature, providing models with exposure to rich vocabulary, complex syntactic structures, and stylistically flawless Russian text, acting as a "quality anchor"… See the full description on the dataset page: https://huggingface.co/datasets/zait-ai/RuHeritage-Corpus.EvoAgentBench
EvoAgentBench
EvoAgentBench is a benchmark for evaluating AI agent self-evolution — the ability of agents to improve their performance by learning from past experiences. It provides standardized train/test splits across five diverse task domains, enabling reproducible comparison of skill extraction and experience reuse methods.
Benchmark Overview
Domain
Base Dataset
Train
Test
Task Format
Information Retrieval
BrowseComp-Plus
154
65
Multi-constraint… See the full description on the dataset page: https://huggingface.co/datasets/EverMind-AI/EvoAgentBench.story-imprinting
Story Imprinting — training datasets
Datasets accompanying Story Imprinting: AI Assistants Absorb Traits from Human Characters They Resemble.
Paper · Code
Contents
Paper section
Folder
Data
3.1 — Sabotage
3_1_sabotage/
Three training mixtures and separate sabotage/clean story pools
3.2 — Narration preferences
3_2_narration_preferences/
Six training mixtures and 12 story pools
4 — Affinity
4_selectivity/
Opposing-pair training datasets and raw… See the full description on the dataset page: https://huggingface.co/datasets/truthful-ai/story-imprinting.RAIL-HH-10K RAIL-HH-10K: Multi-Dimensional Safety Alignment Dataset
The first large-scale safety dataset with 99.5% multi-dimensional annotation coverage across 8 ethical dimensions.
📖 Read Blog •
📖 Paper (Coming Soon) •
🚀 Quick Start •
🔌 RAIL API •
💻 Examples
🌟 What Makes RAIL-HH-10K Special?
🎯 Near-Complete Coverage
99.5% dimension coverage across all 8 ethical dimensions
Most existing datasets: 40-70% coverage
RAIL-HH-10K: 98-100%… See the full description on the dataset page: https://huggingface.co/datasets/responsible-ai-labs/RAIL-HH-10K.seas-coevolve
seas-coevolve
Per-round trajectory + per-category taxonomy from an automated red-teaming co-evolution loop (attacker -> target -> harm-judge) on Qwen2.5 (7B attacker+judge, 3B target), seeded with JailbreakBench behaviors. This dataset is the co-evolution (attacker AND target train each round) arm: held-in ASR oscillates ~0.48 across 5 rounds with no trend - no arms race emerges.
Study question: does adversarial co-evolution ignite an arms race, and does safety-hardening… See the full description on the dataset page: https://huggingface.co/datasets/yavuz-ai/seas-coevolve.cc100-nepali
CC-100 Nepali — Cleaned & Deduplicated
Cleaned, language-filtered, and deduplicated Nepali monolingual text derived from
CC-100, suitable for transformer pretraining.
Originally published at himalaya-ai/cc100-nepali.Dataset contents replaced with the cleaned version from Titung/cc100-nepali-cleaned.
Statistics
Split
Sentences
train
4,736,157
validation
48,328
test
48,329
total
4,832,814
Token Statistics (train split)
Metric
Value… See the full description on the dataset page: https://huggingface.co/datasets/himalaya-ai/cc100-nepali.moltbook-data
Moltbook Social Interaction Dataset
Moltbook is consists of individual "claws"
Dataset Summary
The Moltbook Social Interaction Dataset captures large-scale social interactions from Moltbook, a persistent online platform populated entirely by LLM-powered agents. Each agent autonomously creates posts, writes comments, upvotes/downvotes content, and engages in threaded discussions across community-organized "submolts" (topic-based groups similar to subreddits).… See the full description on the dataset page: https://huggingface.co/datasets/AIcell/moltbook-data.reward-hacking-olmo3.1-32b-kl0.02-seed2-rollouts
Reward-Hacking Training Rollouts — OLMo-3.1-32B (β=0.02, seed 2)
GRPO reinforcement-learning training rollouts from a reward-hackable competitive-programming environment, part of the Science of Model Organisms (mt-somo) study of natural emergent misalignment from reward hacking.
Companion to the checkpoint repo ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.02-seed2. With a small KL penalty (β=0.02) the policy stays closer to the base model, yet it still learns to exploit… See the full description on the dataset page: https://huggingface.co/datasets/ai-safety-institute/reward-hacking-olmo3.1-32b-kl0.02-seed2-rollouts.