datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
cybergymsts12-sts
STS12
An MTEB dataset
Massive Text Embedding Benchmark
SemEval-2012 Task 6.
Task category
t2t
Domains
Encyclopaedic, News, Written
Reference
https://www.aclweb.org/anthology/S12-1051.pdf
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STS12"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how to… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts12-sts.swe-bench-dummy-test-datasetFineFineWeb-sample
FineFineWeb: A Comprehensive Study on Fine-Grained Domain Web Corpus
arXiv: Coming Soon
Project Page: Coming Soon
Blog: Coming Soon
Data Statistics
Domain (#tokens/#samples)
Iteration 1 Tokens
Iteration 2 Tokens
Iteration 3 Tokens
Total Tokens
Iteration 1 Count
Iteration 2 Count
Iteration 3 Count
Total Count
aerospace
5.77B
261.63M
309.33M
6.34B
9100000
688505
611034
10399539
agronomy
13.08B
947.41M
229.04M
14.26B
15752828
2711790
649404
19114022
artistic… See the full description on the dataset page: https://huggingface.co/datasets/m-a-p/FineFineWeb-sample.tiny-supervised-datasetCodeAlpaca-20kforecast-news
Forecast News
Deduplicated daily news corpus used by forecast-sim and future-sim.
Snapshot
31,859,020 articles
3,463 daily partitions
Coverage: 2016-08-26 through 2026-08-31
Snapshot published: 2026-09-18
Stored data size: approximately 158.5 GiB
The Parquet files are the canonical complete representation. The repository also
contains daily JSONL files where available and compact headline JSON files used
by article-browsing workflows.
Layout
Files… See the full description on the dataset page: https://huggingface.co/datasets/shash42/forecast-news.xlam-function-calling-60k
APIGen Function-Calling Datasets
Paper | Website | Models
This repo contains 60,000 data collected by APIGen, an automated data generation pipeline designed to produce verifiable high-quality datasets for function-calling applications. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness.
We conducted human evaluation over 600 sampled data points… See the full description on the dataset page: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k.synthesized_datasetsts22-crosslingual-sts
STS22.v2
An MTEB dataset
Massive Text Embedding Benchmark
SemEval 2022 Task 8: Multilingual News Article Similarity. Version 2 filters updated on STS22 by removing pairs where one of entries contain empty sentences.
Task category
t2t
Domains
News, Written
Reference
https://competitions.codalab.org/competitions/33835
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts22-crosslingual-sts.cc-traces-weka-062126
semianalysisai/cc-traces-weka-062126
WekaTrace corpus derived from SemiAnalysis Claude Code proxy traces. Built 2026-06-21 17:48:24 UTC via utils/agentic/build_weka_hf_dataset.py.
Filters
Trace version: exactly v7
min Anthropic requests per session: 20
Claude Code CLI ≥ 2.1.139 (every row)
peak concurrent sub-agent groups ≤ 10
Non-image rows only (image content excluded at source)
Classifier calls excluded (max_tokens<=64 AND no tools → SUGGESTION MODE, title-gen… See the full description on the dataset page: https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126.the-stack-smol
Dataset Description
A small subset (~0.1%) of the-stack dataset, each programming language has 10,000 random samples from the original dataset. The dataset has 2.6GB of text (code).
Languages
The dataset contains 30 programming languages:
"assembly", "batchfile", "c++", "c", "c-sharp", "cmake", "css", "dockerfile", "fortran", "go", "haskell", "html", "java",
"javascript", "julia", "lua", "makefile", "markdown", "perl", "php", "powershell", "python", "ruby", "rust"… See the full description on the dataset page: https://huggingface.co/datasets/bigcode/the-stack-smol.UltraData-SFT-2605
UltraData-SFT-2605
📦 UltraData Collection |
🌐 UltraData |
🤗 MiniCPM5 Series
English |
中文
📚 Introduction
UltraData-SFT-2605 is the full set of core-domain SFT data used in the post-training of MiniCPM5-1B-SFT within the MiniCPM5-1B series, and a key representative of L3 refined data in the UltraData L0-L4 tiered data management framework. It covers math, code, knowledge, instruction following, and other core domains, containing over 15 million Deep… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraData-SFT-2605.soc-ratchakitcha
Royal Gazette Thailand (Ratchakitcha) Dataset
ชุดข้อมูลราชกิจจานุเบกษา (แบบ Machine Readable)
โครงการ Open Law Data Thailand ร่วมกับคณะกรรมาธิการการพาณิชย์และการอุตสาหกรรม วุฒิสภา ได้รับความอนุเคราะห์ข้อมูลจาก สำนักเลขาธิการคณะรัฐมนตรี (สลค.) เพื่อเผยแพร่ข้อมูลกฎหมายไทยสู่สาธารณะในรูปแบบที่ประมวลผลได้ด้วยคอมพิวเตอร์ (Machine Readable) เพื่อส่งเสริมนวัตกรรม Legal Tech และ AI ของประเทศไทย
Dataset Description
ชุดข้อมูลนี้รวบรวมรายการประกาศในราชกิจจานุเบกษา… See the full description on the dataset page: https://huggingface.co/datasets/open-law-data-thailand/soc-ratchakitcha.sickr-sts
SICK-R
An MTEB dataset
Massive Text Embedding Benchmark
Semantic Textual Similarity SICK-R dataset
Task category
t2t
Domains
Web, Written
Reference
https://aclanthology.org/L14-1314/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["SICK-R"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how to run models… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sickr-sts.sts13-sts
STS13
An MTEB dataset
Massive Text Embedding Benchmark
SemEval STS 2013 dataset.
Task category
t2t
Domains
Web, News, Non-fiction, Written
Reference
https://www.aclweb.org/anthology/S13-1004/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STS13"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more about how… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts13-sts.sts14-sts
STS14
An MTEB dataset
Massive Text Embedding Benchmark
SemEval STS 2014 dataset. Currently only the English dataset
Task category
t2t
Domains
Blog, Web, Spoken
Reference
https://www.aclweb.org/anthology/S14-1002
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STS14"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To… See the full description on the dataset page: https://huggingface.co/datasets/mteb/sts14-sts.scifact
SciFact
An MTEB dataset
Massive Text Embedding Benchmark
SciFact verifies scientific claims using evidence from the research literature containing scientific paper abstracts.
Task category
t2t
Domains
Academic, Medical, Written
Reference
https://github.com/allenai/scifact
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["SciFact"])
evaluator = mteb.MTEB(task)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/scifact.scidocs
SCIDOCS
An MTEB dataset
Massive Text Embedding Benchmark
SciDocs, a new evaluation benchmark consisting of seven document-level tasks ranging from citation prediction, to document classification and recommendation.
Task category
t2t
Domains
Academic, Written, Non-fiction
Reference
https://allenai.org/data/scidocs
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task =… See the full description on the dataset page: https://huggingface.co/datasets/mteb/scidocs.biosses-sts
BIOSSES
An MTEB dataset
Massive Text Embedding Benchmark
Biomedical Semantic Similarity Estimation.
Task category
t2t
Domains
Medical
Reference
https://tabilab.cmpe.boun.edu.tr/BIOSSES/DataSet.html
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["BIOSSES"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)
evaluator.run(model)
To learn more… See the full description on the dataset page: https://huggingface.co/datasets/mteb/biosses-sts.medical-o1-reasoning-SFT
News
[2025/04/22] We split the data and kept only the medical SFT dataset (medical_o1_sft.json). The file medical_o1_sft_mix.json contains a mix of medical and general instruction data.
[2025/02/22] We released the distilled dataset from Deepseek-R1 based on medical verifiable problems. You can use it to initialize your models with the reasoning chain from Deepseek-R1.
[2024/12/25] We open-sourced the medical reasoning dataset for SFT, built on medical verifiable problems and an… See the full description on the dataset page: https://huggingface.co/datasets/FreedomIntelligence/medical-o1-reasoning-SFT.sst5
Stanford Sentiment Treebank - Fine-Grained
Stanford Sentiment Treebank with 5 labels: very positive, positive, neutral, negative, very negative
Splits are from:
https://github.com/AcademiaSinicaNLPLab/sentiment_dataset/tree/master/data
Training data is on sentence level, not on phrase level!
UltraData-SFT-Agent-2609
UltraData-SFT-Agent-2609
📦 UltraData Collection |
🌐 UltraData |
🤗 MiniCPM5 Series
English |
中文
📚 Introduction
UltraData-SFT-Agent-2609 is the L3 refined data for Agent instruction-tuning within UltraData's L0-L4 tiered data management framework. Built for the post-training of MiniCPM5-2B, it complements UltraData-SFT-2605 (core-domain SFT) with executable Agent trajectories. The release contains approximately 500,000 samples spanning tool use… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609.cc-traces-weka-062126-256k
semianalysisai/cc-traces-weka-062126-256k
WekaTrace corpus derived from SemiAnalysis Claude Code proxy traces. Built 2026-06-21 17:49:45 UTC via utils/agentic/build_weka_hf_dataset.py.
Derived from semianalysisai/cc-traces-weka-062126 by applying the 256k per-request cap and preserving the surviving requests' relative timestamps.
Filters
Trace version: exactly v7
min Anthropic requests per session: 20
Claude Code CLI ≥ 2.1.139 (every row)
peak concurrent… See the full description on the dataset page: https://huggingface.co/datasets/semianalysisai/cc-traces-weka-062126-256k.stsbenchmark-sts
STSBenchmark
An MTEB dataset
Massive Text Embedding Benchmark
Semantic Textual Similarity Benchmark (STSbenchmark) dataset.
Task category
t2t
Domains
Blog, News, Written
Reference
https://github.com/PhilipMay/stsb-multi-mt/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import mteb
task = mteb.get_tasks(["STSBenchmark"])
evaluator = mteb.MTEB(task)
model = mteb.get_model(YOUR_MODEL)… See the full description on the dataset page: https://huggingface.co/datasets/mteb/stsbenchmark-sts.cadgenbench-submissions
CADGenBench Submissions
Submissions and evaluation results for the CADGenBench leaderboard.
Leaderboard Space: HuggingAI4Engineering/CADGenBench.
Benchmark code: github.com/huggingface/cadgenbench.
Fixture inputs: HuggingAI4Engineering/cadgenbench-data.
Ground truth: HuggingAI4Engineering/cadgenbench-data-gt (private).
Contents
Path
What it is
results.jsonl
One row per submitted + evaluated submission. The leaderboard table reads from this.… See the full description on the dataset page: https://huggingface.co/datasets/HuggingAI4Engineering/cadgenbench-submissions.wrf-250m-severe-weather
WRF 250m Severe Weather Simulations — Raw WRF Output
2 TB of raw WRF-ARW v4.7.1 output from 18 high-impact weather events at 250m resolution. Three nested domains (3km / 1km / 250m), 81 vertical levels, 216 variables per wrfout file, 1-minute surface output via auxhist2. This is the full simulation output — nothing removed, nothing derived.
If you want pre-extracted surface fields in a simpler format, see wrf-250m-severe-weather-netcdf4 (16 fields, ~150 GB).
This dataset is… See the full description on the dataset page: https://huggingface.co/datasets/deepguess/wrf-250m-severe-weather.SciCodeThis dataset was presented in SciCode: A Research Coding Benchmark Curated by Scientists.
security-auditsA collection of agent traces generated with Swival (not Claude Code, despite what the HF interface currently shows), an agent designed for open-source models.
These traces focus on security audits of opensource software.
Sharing traces with Swival
Swival can export full conversation traces with --trace-dir, which writes one <session_id>.jsonl file per session:
swival "Fix the login bug" --trace-dir traces/
Those JSONL files use Swival's Claude Code compatible trace export, and… See the full description on the dataset page: https://huggingface.co/datasets/jedisct1/security-audits.PKU-SafeRLHF
Dataset Card for PKU-SafeRLHF
Warning: this dataset contains data that may be offensive or harmful. The data are intended for research purposes, especially research that can make models less harmful. The views expressed in the data do not reflect the views of PKU-Alignment Team or any of its members.
[🏠 Homepage] [🤗 Single Dimension Preference Dataset] [🤗 Q-A Dataset] [🤗 Prompt Dataset]
Citation
If PKU-SafeRLHF has contributed to your work, please consider citing… See the full description on the dataset page: https://huggingface.co/datasets/PKU-Alignment/PKU-SafeRLHF.