datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ClimbLab-JaJapanese / 日本語版
ClimbLab-Ja
ClimbLab-Ja is a high-quality 300-billion-token Japanese corpus with 20 clusters. It is a Japanese adaptation of the nvidia/Nemotron-ClimbLab approach. Based on LLM-jp Corpus v4, we semantically reorganized and filtered the dataset into 20 distinct clusters, resulting in a high-quality 300-billion-token corpus. Specifically, we first grouped the data into 1,000 groups based on topic information. Then we assigned six scores from 0 to 5 to each group… See the full description on the dataset page: https://huggingface.co/datasets/KantaHayashiAI/ClimbLab-Ja.Nemotron-ClimbMix
ClimbMix Dataset
🚀 Creating the highest-quality pre-training datasets for LLMs 🌟
📄 PAPER
🤗 CLIMBLAB
🤗 CLIMBMIX
🏠 HOMEPAGE
Figure 1: Continuously training a 1B model yields a 2.0% improvement over Llama-3.2-1B, demonstrating a more efficient scaling trend compared to prior models.
Figure 2: Pre-training a 1B model from scratch on ClimbMix shows better scaling effects than training on other datasets.… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/Nemotron-ClimbMix.ClimbMixClimbMix is a high-quality pre-training corpus released by NVIDIA. Here is the description:
ClimbMix is a compact yet powerful 400-billion-token dataset designed for efficient pre-training that delivers superior performance under an equal token budget. It was introduced in this paper.
We proposed a new algorithm to filter and mix the dataset. First, we grouped the data into 1,000 groups based on topic information. Then we applied two classifiers: one to detect advertisements and another to… See the full description on the dataset page: https://huggingface.co/datasets/OptimalScale/ClimbMix.cli-commands-explained
Overview
This dataset is a collection of 16,098 command line instructions sourced from Commandlinefu and Cheatsheets. It includes an array of commands, each with an id, title, description, date, url to source, author, votes, and flag indicating if the description is AI generated. The descriptions are primarily authored by the original contributors, for entries where descriptions were absent, they have been generated using NeuralBeagle14-7B. Out of the total entries, 10,039… See the full description on the dataset page: https://huggingface.co/datasets/b-mc2/cli-commands-explained.CLI-Bench
CLI-Bench: Benchmarking AI Agents on Command-Line Tool Orchestration
Abstract
CLI-Bench is an evaluation benchmark for measuring AI agents' ability to learn and use command-line interface (CLI) tools to complete real-world tasks. Unlike existing benchmarks that test general coding ability or narrow tool-use scenarios, CLI-Bench evaluates tool-agnostic CLI orchestration -- the capacity to read tool documentation, plan multi-step workflows, execute commands… See the full description on the dataset page: https://huggingface.co/datasets/ChengyiX/CLI-Bench.devin-cli-reasoning-distillation
Devin CLI Reasoning Distillation Dataset
A distillation dataset built from Devin CLI session traces, containing the model's internal
reasoning traces (chain-of-thought / thinking), user prompts, assistant answers, and tool calls.
The dataset is formatted to be directly compatible with SFT training pipelines that expect
OpenAI-style message lists with a reasoning_content field.
Dataset Summary
Total rows
2,632 (2,507 train / 125 validation)
Rows with… See the full description on the dataset page: https://huggingface.co/datasets/Akahsizrr/devin-cli-reasoning-distillation.SERA-KimiK3-Django-SWEAgent-Cliff32k-T1
SERA Kimi-K3 Django SWE-Agent — Cliff-chunked T1 (first rollout)
572 training records built from 210 Kimi-K3 SWE-agent trajectories on
Django, split to fit a 32,768-token context with
CliffCompaction instead of being truncated.
Why chunked
A 100+ step agent rollout does not fit a 32k training window — 76% of the
source T1 trajectories exceed it. Truncating them throws away most of the
supervision, and trains the model on a context format it never sees at… See the full description on the dataset page: https://huggingface.co/datasets/thientrangngv/SERA-KimiK3-Django-SWEAgent-Cliff32k-T1.SERA-KimiK3-Django-SWEAgent-Cliff32k-T2
SERA Kimi-K3 Django SWE-Agent — Cliff-chunked T2 (second rollout)
227 training records built from 137 Kimi-K3 SWE-agent trajectories on
Django, split to fit a 32,768-token context with
CliffCompaction instead of being truncated.
Why chunked
A 100+ step agent rollout does not fit a 32k training window — 27% of the
source T2 trajectories exceed it. Truncating them throws away most of the
supervision, and trains the model on a context format it never sees at… See the full description on the dataset page: https://huggingface.co/datasets/thientrangngv/SERA-KimiK3-Django-SWEAgent-Cliff32k-T2.finject
FInject Dataset Card
FInject is a financial unanswerability benchmark built by transforming answerable financial reasoning problems into controlled unanswerable variants. Each row preserves the original question and pairs an answerable original context with a perturbed context that is no longer sufficient to support a unique answer.
Dataset Summary
Seed source: 78 answerable hard problems from FinanceReasoning.
Final release size: 426 unanswerable variants.… See the full description on the dataset page: https://huggingface.co/datasets/pnu-clink/finject.ClimbMix-sample
Unofficial NVIDIA Nemotron-ClimbMix (Subsampled)
This dataset is a curated, subsampled version of OptimalScale/ClimbMix, which itself is a detokenized version of NVIDIA's official pretraining dataset, nvidia/Nemotron-ClimbMix.
It is designed for researchers and developers looking for a smaller, well-shuffled slice of the Nemotron pretraining data for quick experimentation, testing, or ablation studies.
Processing Method
To create this streamlined version, the… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/ClimbMix-sample.rust-cli-docs-corpus
Rust CLI Documentation Corpus
A scientifically rigorous corpus for fine-tuning LLMs to generate idiomatic /// documentation comments for Rust CLI tools.
Dataset Description
This corpus follows the Toyota Way principles and Popperian falsification methodology.
Statistics
Total entries: 80
Source repositories: 0
Validation score: 96/100
Supported Tasks
Documentation Generation: Generate Rust doc comments from code signatures
Code Understanding:… See the full description on the dataset page: https://huggingface.co/datasets/paiml/rust-cli-docs-corpus.climbs
Climbs Dataset
This dataset contains climbing route data. It was created using the BoardLib (unofficial) API.
This api pulls publicly available climbs from the following Aurora climbing boards:
Kilter
Tension
Decoy
Aurora
Grasshopper
Touchstone
Soill
Usage
from datasets import load_dataset
dataset = load_dataset("mattzcarey/climbs")
License
not sure yet. Also not sure how legal this is, please don't sue me.
stride-preproc-climbmix
STRIDE: Preprocessed ClimbMix
Tokenized ClimbMix sequences used for STRIDE's pretraining-data attribution experiments. There is one training pool per released nanochat depth (d12, d16, d20, d24) and one held-out test set shared across depths. These are the corpora the released nanochat operators attribute over, so STRIDE's column indices line up with the lines of these files.
Files
File
Sequences
Size
Contents
climbmix_train_d12.jsonl
1,317,003
3.8 GB
training… See the full description on the dataset page: https://huggingface.co/datasets/CausalNLP/stride-preproc-climbmix.Med-ART_Clinical_Agent_EHR_Dataset
ART — Action-based Reasoning Tasks (Subset)
120-task stratified sample from the ART benchmark introduced in:
ART: Action-based Reasoning Task Benchmarking for Medical AI Agents
Ananya Mantravadi, Shivali Dalmia, Abhishek Mukherji
arXiv:2601.08988
ART is a programmatically generated clinical decision benchmark built on real FHIR patient data. It targets three dominant error categories in medical AI reasoning — retrieval failures, aggregation errors, and conditional logic… See the full description on the dataset page: https://huggingface.co/datasets/CentificAIResearch/Med-ART_Clinical_Agent_EHR_Dataset.Galaxy_clients
Galaxy Datasets
The Galaxy Datasets are a collection of four synthetic datasets created for NLP experiments, featuring client records in a nursing home setting. Each dataset serves a specific purpose and builds upon the previous one, providing a useful resource for various NLP tasks.
Galaxy_clients contains profiles of nursing home residents, including additional information such as the number of months before data generation for each client and the complications they experienced… See the full description on the dataset page: https://huggingface.co/datasets/ekrombouts/Galaxy_clients.ClimateMBERT-syn-qwen3-30b-a3b-fp8-10k-seed42
ClimateMBERT Synthetic Qwen3 30B A3B FP8 10K Seed42
Synthetic continuation dataset generated from WxChat/ClimateMBERT_syn train split.
Source dataset: WxChat/ClimateMBERT_syn
Source split: train
Sampling: shuffled with random seed 42, ranks 0..9999
Rows: 10,000
Generator: Qwen/Qwen3-30B-A3B-Instruct-2507-FP8
Inference: vLLM on Clariden GH200 GPUs, tensor parallel size 2, non-eager mode
Max tokens: 4096
Generation config: temperature 0.7, top_p 0.8, top_k 20, min_p 0.0… See the full description on the dataset page: https://huggingface.co/datasets/JingweiNi/ClimateMBERT-syn-qwen3-30b-a3b-fp8-10k-seed42.clinical_confidence_collapse_detection_v0.1Clinical Confidence Collapse Detection
PurposeDetect when a prior working diagnosis should lose confidence fast due to new evidence.
You receive:
working_diagnosis
confidence_before
new_evidence
proposed_next_step
You output one JSON object:
confidence_collapseyes or no
new_confidencefloat 0 to 1
correct_actionone sentence
Scoring
confidence_collapse_accuracy
new_confidence_score
correct_action_similarity
format_pass_rate
Run scoringpython scorer.py --predictions… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/clinical_confidence_collapse_detection_v0.1.ClimateMBERT-syn-qwen35-122b-fp8-10k-seed42
ClimateMBERT Synthetic Qwen3.5 FP8 10K Seed42
Synthetic continuation dataset generated from WxChat/ClimateMBERT_syn train split.
Source dataset: WxChat/ClimateMBERT_syn
Source split: train
Sampling: shuffled with random seed 42, ranks 0..9999
Rows: 10,000
Generator: Qwen/Qwen3.5-122B-A10B-FP8
Inference: vLLM on Clariden GH200 GPUs, tensor parallel size 4, non-eager mode
Max tokens: 4096
No-thinking mode: chat_template_kwargs={"enable_thinking": false}
Generation config: temperature… See the full description on the dataset page: https://huggingface.co/datasets/JingweiNi/ClimateMBERT-syn-qwen35-122b-fp8-10k-seed42.ClimbMix-split
ClimbMix Split
climbmix-split reorganizes the detokenized NVIDIA ClimbMix source corpus into
source-oriented splits. ClimbMix is described as being built from Nemotron-CC
and SmolLM-Corpus. Because SmolLM-Corpus is the smaller and directly
identifiable component, we used exact normalized-text matching against
SmolLM-Corpus to recover the SmolLM-derived portions. The remaining rows are
provided as the residual nemotron-cc split.
The data rows are unchanged from the detokenized… See the full description on the dataset page: https://huggingface.co/datasets/AIML-TUDA/ClimbMix-split.