datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Ultra-FineWeb
Ultra-FineWeb
📜 Technical Report |
📦 UltraData Collection |
🌐 UltraData |
🤗 MiniCPM4 Series |
🤗 MiniCPM5 Series
English |
中文
📚 Introduction
Ultra-FineWeb is a large-scale, high-quality, and efficiently-filtered dataset. We use the proposed efficient verification-based high-quality filtering pipeline to the FineWeb and Chinese FineWeb datasets (source data from Chinese FineWeb-edu-v2, which includes IndustryCorpus2, MiChao, WuDao, SkyPile… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/Ultra-FineWeb.UltraData-Math
UltraData-Math
🤗 Dataset | 💻 Source Code | 🇨🇳 中文 README
UltraData-Math is a large-scale, high-quality mathematical pre-training dataset totaling 290B+ tokens across three progressive tiers—L1 (170.5B tokens web corpus), L2 (33.7B tokens quality-selected), and L3 (88B tokens multi-format refined)—designed to systematically enhance mathematical reasoning in LLMs. It has been applied to the mathematical pre-training of the MiniCPM Series models.
It was introduced in… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraData-Math.UltraData-Code
UltraData-Code
📦 UltraData Collection |
🌐 UltraData |
🤗 MiniCPM5 Series |
📖 Tech Report (Coming Soon) |
🤗 UltraData-Code-L2 Classifier
English | 中文
📚 Introduction
UltraData-Code is a complete implementation of the UltraData L0-L4 tiered data management framework. It covers four code data states from L0 through L3, with each level corresponding to a distinct construction stage. The pipeline starts from approximately 192 million public GitHub… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraData-Code.Ultra-FineWeb-L3
Ultra-FineWeb-L3
📜 Ultra-FineWeb Technical Report |
📦 UltraData Collection |
🌐 UltraData |
🤗 MiniCPM5 Series
English |
中文
📚 Introduction
Ultra-FineWeb-L3 is the L3 refined data for general high-quality web data within UltraData's L0-L4 tiered data management framework. Moving beyond L2 quality selection, it transforms high-value web corpora into structured, high-learnability training data with clearer reasoning signals and richer educational… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/Ultra-FineWeb-L3.UltraData-SFT-2605
UltraData-SFT-2605
📦 UltraData Collection |
🌐 UltraData |
🤗 MiniCPM5 Series
English |
中文
📚 Introduction
UltraData-SFT-2605 is the full set of core-domain SFT data used in the post-training of MiniCPM5-1B-SFT within the MiniCPM5-1B series, and a key representative of L3 refined data in the UltraData L0-L4 tiered data management framework. It covers math, code, knowledge, instruction following, and other core domains, containing over 15 million Deep… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraData-SFT-2605.Ultra-FineWeb-L1
Ultra-FineWeb-L1
📜 Ultra-FineWeb Technical Report |
📦 UltraData Collection |
🌐 UltraData
English |
中文
📚 Introduction
Ultra-FineWeb-L1 is a large-scale English web corpus built from Common Crawl snapshots. Within UltraData's L0-L4 tiered data management framework, it serves as the L1 filtered layer for general web data and provides the foundation for subsequent L2 selection and L3 refinement. Building on the FineWeb processing pipeline, we perform… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/Ultra-FineWeb-L1.UltraData-SFT-Agent-2609
UltraData-SFT-Agent-2609
📦 UltraData Collection |
🌐 UltraData |
🤗 MiniCPM5 Series
English |
中文
📚 Introduction
UltraData-SFT-Agent-2609 is the L3 refined data for Agent instruction-tuning within UltraData's L0-L4 tiered data management framework. Built for the post-training of MiniCPM5-2B, it complements UltraData-SFT-2605 (core-domain SFT) with executable Agent trajectories. The release contains approximately 500,000 samples spanning tool use… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraData-SFT-Agent-2609.UltraX-Preview
UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing
📜 Paper |
💻 Code |
🤖 Models |
📦 UltraData Collection
English |
中文
📚 Introduction
UltraX is a function-calling refinement framework for large-scale pre-training data that adaptively generates and executes editing functions for efficient instance-wise refinement. Unlike rule-based or end-to-end LLM rewriting methods, UltraX trains a lightweight… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraX-Preview.UltraFeedback
Introduction
GitHub Repo
UltraRM-13b
UltraCM-13b
UltraFeedback is a large-scale, fine-grained, diverse preference dataset, used for training powerful reward models and critic models. We collect about 64k prompts from diverse resources (including UltraChat, ShareGPT, Evol-Instruct, TruthfulQA, FalseQA, and FLAN). We then use these prompts to query multiple LLMs (see Table for model lists) and generate 4 different responses for each prompt, resulting in a total of 256k samples.
To… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraFeedback.UltraChat
Dataset Card for Dataset Name
Dataset Description
An open-source, large-scale, and multi-round dialogue data powered by Turbo APIs. In consideration of factors such as safeguarding privacy, we do not directly use any data available on the Internet as prompts.
To ensure generation quality, two separate ChatGPT Turbo APIs are adopted in generation, where one plays the role of the user to generate queries and the other generates the response.
We instruct the user model with… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/UltraChat.MA-ProofBench
MA-ProofBench: A Two-Tiered Evaluation of LLMs for Theorem Proving in Mathematical Analysis
English | 中文
We introduce MA-ProofBench, to the best of our knowledge, the first formal benchmark for evaluating large language models (LLMs) on theorem proving in Mathematical Analysis. It contains 200 rigorously formalized theorem-proving problems in Lean 4 + Mathlib (v4.28.0), split into two difficulty tiers:
Tier
Description
Source
Count
Level I
Undergraduate… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/MA-ProofBench.FormalVerse
MathForm: Scaling Mathematical Autoformalization with Knowledge Retrieval and Verification-Guided Refinement
FormalVerse is a verified Lean 4 autoformalization dataset released with the
paper MathForm: Scaling Mathematical Autoformalization with Knowledge Retrieval
and Verification-Guided Refinement.
Every example is produced by the MathForm pipeline, which retrieves relevant
Mathlib knowledge before generation and refines each candidate using Lean
compiler diagnostics and… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/FormalVerse.RLHF-V-Dataset
Dataset Card for RLHF-V-Dataset
Project Page | Paper | GitHub
Updates
[2024.05.28] 📃 Our RLAIF-V paper is accesible at arxiv now!
[2024.05.20] 🎉 We release a new feedback dataset, RLAIF-V-Dataset, which is a large-scale diverse-task multimodal feedback dataset constructed using open-source models. You can download the corresponding dataset and models (7B, 12B) now!
[2024.04.11] 🔥 Our data is used in MiniCPM-V 2.0, an end-side multimodal large language model that… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/RLHF-V-Dataset.RLPR-Train-Dataset
Dataset Card for RLPR-Train-Dataset
GitHub | Paper
News:
[2025.06.23] 📃 Our paper detailing the RLPR framework and this dataset is accessible at here.
Dataset Summary
The RLPR-Train-Dataset is a curated collection of 77k high-quality reasoning prompts specifically designed for enhancing Large Language Model (LLM) capabilities in the general domain (non-mathematical).
This dataset is derived from the comprehensive collection of prompts from WebInstruct. We… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/RLPR-Train-Dataset.RLPR-Evaluation
Dataset Card for RLPR-Evaluation
GitHub | Paper
News:
[2025.06.23] 📃 Our paper detailing the RLPR framework and its comprehensive evaluation using this suite is accessible at here!
Dataset Summary
We include the following seven benchmarks for evaluation of RLPR:
Mathematical Reasoning Benchmarks:
MATH-500 (Cobbe et al., 2021)
Minerva (Lewkowycz et al., 2022)
AIME24
General Domain Reasoning Benchmarks:
MMLU-Pro (Wang et al., 2024): A multitask language… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/RLPR-Evaluation.DensingLaw-ScalingBench
DensingLaw-ScalingBench
This dataset was created to enable a more accurate performance scaling law estimation of Large Language Models (LLMs).
This dataset is released as part of our paper, Densing Law of LLMs.
📜 Paper
💡 Overview
This repository contains the open-source dataset used for calculating conditional loss in our LLM density evaluation framework.
LLM density is defined as the ratio of effective parameter size to actual parameter size, where effective… See the full description on the dataset page: https://huggingface.co/datasets/openbmb/DensingLaw-ScalingBench.curated-openbmb-code-math
Curated OpenBMB Code/Math Post-Training Data
English code/math-focused post-training data derived from curated OpenBMB UltraData rows.
Contents
Config
Rows
Schema
Purpose
sft_no_think
25,891
prompt, response
Direct code/math SFT plus necessary technical instruction-following/alignment
sft_think
6,018
prompt, response
Code/math reasoning SFT with <think>...</think> traces
Total rows: 31,909.
Curation
The SFT split keeps English code… See the full description on the dataset page: https://huggingface.co/datasets/josephmayo/curated-openbmb-code-math.