datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
dclm-baseline-1.0
DCLM-baseline
DCLM-baseline is a 4T token / 3B document pretraining dataset that achieves strong performance on language model benchmarks.
Below are comparisions of model trained on DCLM-baseline with other models in the 7B regime.
Model
Params
Tokens
Open dataset?
CORE
MMLU
EXTENDED
Open weights, closed datasets
Llama2
7B
2T
✗
49.2
45.8
34.1
DeepSeek
7B
2T
✗
50.7
48.5
35.3
Mistral-0.3
7B
?
✗
57.0
62.7
45.1
QWEN-2
7B
?
✗
57.5
71.9
50.5
Llama3
8B
15T
✗… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0.dcvlm-baseline-200b
DCVLM-Baseline (200B tokens)
DCVLM-Baseline is the reference training mixture from our DataComp-VLM paper.
It is a pre-mixed, decontaminated, ready-to-train multimodal pretraining dataset, materialized as flat
WebDataset tar shards so it can be consumed by any training
stack.
This is a 200B-token dataset release consisting of 103,985,276 samples, curated from our DCVLM-large data pool.
A smaller 6.25B-token version is also available.
⚠️ NOTE: The training data is the WebDataset… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations/dcvlm-baseline-200b.knowledge-base
RL-for-LLMs Wiki
An expert-level, citation-backed knowledge base on reinforcement learning for
large language models — RLHF, DPO and offline preference optimization, reward
modeling, RLVR and reasoning, training systems, and the failure modes — built
collaboratively by autonomous agents. Each topic article is a deep dive written
so you can learn the topic from it without reading the underlying papers, with
every non-obvious claim cited to a source. Every change lands through a… See the full description on the dataset page: https://huggingface.co/datasets/rl-llm-wiki/knowledge-base.cmp-v6-base108-renderbite-baseline
bite-baseline — artifacts for extreme (ternary) quantization of Qwen3.6-35B-A3B
Companion dataset for ihavespoons/bite — an open
pipeline for compressing a Mixture-of-Experts LLM (Qwen/Qwen3.6-35B-A3B, 35B total / ~3B
active, 256 experts) toward ternary {-1,0,+1} weights (1.71 bpw) via PTQ init +
quantization-aware distillation. See the repo's docs/report-extreme-quant-moe.md for the
full technical report.
Contents
Path
What it is
baseline.json… See the full description on the dataset page: https://huggingface.co/datasets/ihavespoons/bite-baseline.LET-Base-Dataset
LET:Full-Size Humanoid Robot Real-World Dataset
中文| [English]
LET Dataset is collected based on the full-size humanoid robot Kuavo 4 Pro covering real-world multi-task data across multiple scenarios and operation types. It is designed for robot manipulation, mobility, and interaction tasks, supporting scalable robot learning in real environments.
📋 Table of Contents
Key Features
Hardware Platform
Usage Guide
Tool… See the full description on the dataset page: https://huggingface.co/datasets/LejuRobotics/LET-Base-Dataset.dclm-baseline-1.0-parquet
DCLM-baseline
Note: this is an identical copy of https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0, where all the files have been mapped to a parquet format.
DCLM-baseline is a 4T token / 3B document pretraining dataset that achieves strong performance on language model benchmarks.
Below are comparisions of model trained on DCLM-baseline with other models in the 7B regime.
Model
Params
Tokens
Open dataset?
CORE
MMLU
EXTENDED
Open weights, closed datasets… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations/dclm-baseline-1.0-parquet.rbo_oxe_base_language_table_lerobot
Language Table (LeRobot) — Task-Pruned, Reindexed Subset
This release is a task-pruned subset of the original
IPEC-COMMUNITY/language_table_lerobot.
We subsampled by task text and rebuilt the package so it remains internally consistent
(indices, splits, stats, paths).
Robot: xArm
Modality: RGB video + states + actions
FPS / Resolution: 10 FPS, 360×640, AV1
License: apache-2.0 (inherits from source)
What’s different in this subset
Kept ~0.85% of unique tasks… See the full description on the dataset page: https://huggingface.co/datasets/saaduddinM/rbo_oxe_base_language_table_lerobot.vtok101-distr-attribution-baselines
vtok101 attribution baselines, with a hard negative beside every document
Data-attribution scores over lamsheeper-data-attribution/Qwen3.5-4B-d0-vtok101-distr-lora-seeds:
3 function counts x 7 document counts x 4 seeds,
scored by 12 methods.
Each training document defines one synthetic constant function, and each query
asks for one function's value. The ground truth for a query is the set of
documents describing its function, so a method is measured by how far up its
ranking… See the full description on the dataset page: https://huggingface.co/datasets/lamsheeper-data-attribution/vtok101-distr-attribution-baselines.route-attribution-baselines
vtok101 attribution baselines
Data-attribution scores over lamsheeper-data-attribution/Qwen3.5-4B-route-ab-l8-lora-scale:
3 function counts x 7 document counts x 4 seeds,
scored by 12 methods.
Each training document defines one synthetic constant function, and each query
asks for one function's value. The ground truth for a query is the set of
documents describing its function, so a method is measured by how far up its
ranking those documents come.
Every document in the pool… See the full description on the dataset page: https://huggingface.co/datasets/lamsheeper-data-attribution/route-attribution-baselines.vigil-jailbreak-all-mpnet-base-v2
Vigil: LLM Jailbreak all-mpnet-base-v2
Repo: github.com/deadbits/vigil-llm
Vigil is a Python framework and REST API for assessing Large Language Model (LLM) prompts against a set of scanners to detect prompt injections, jailbreaks, and other potentially risky inputs.
This repository contains all-mpnet-base-v2 embeddings for all "jailbreak" prompts used by Vigil.
You can use the parquet2vdb.py utility to load the embeddings in the Vigil chromadb instance, or use them in your own… See the full description on the dataset page: https://huggingface.co/datasets/deadbits/vigil-jailbreak-all-mpnet-base-v2.vtok101-attribution-baselines
vtok101 attribution baselines
Data-attribution scores over lamsheeper-data-attribution/Qwen3.5-4B-d0-vtok101-lora-seeds:
3 function counts x 7 document counts x 4 seeds,
scored by 12 methods.
Each training document defines one synthetic constant function, and each query
asks for one function's value. The ground truth for a query is the set of
documents describing its function, so a method is measured by how far up its
ranking those documents come.
Every document in the pool… See the full description on the dataset page: https://huggingface.co/datasets/lamsheeper-data-attribution/vtok101-attribution-baselines.wit_base
Dataset Card for WIT
Dataset Summary
Wikimedia's version of the Wikipedia-based Image Text (WIT) Dataset, a large multimodal multilingual dataset.
From the official blog post:
The core training data is taken from the Wikipedia Image-Text (WIT) Dataset, a large curated set of more than 37 million image-text associations extracted from Wikipedia articles in 108 languages that was recently released by Google Research.
The WIT dataset offers extremely valuable data about the… See the full description on the dataset page: https://huggingface.co/datasets/wikimedia/wit_base.include-base-44
INCLUDE-base (44 languages)
Dataset Description
Paper: http://arxiv.org/abs/2411.19799
Dataset Summary
INCLUDE is a comprehensive knowledge- and reasoning-centric benchmark across 44 languages that evaluates multilingual LLMs for performance in the actual language environments where they would be deployed.
It contains 22,637 4-option multiple-choice-questions (MCQ) extracted from academic and professional exams, covering 57 topics, including… See the full description on the dataset page: https://huggingface.co/datasets/CohereLabs/include-base-44.Bench2Drive-VL-base
Bench2Drive-VL: Full-Stack Software for Closed-Loop Autonomous Driving with Vision Language Models
Project Page | GitHub | Paper
Bench2Drive-VL is a comprehensive closed-loop benchmark for Vision-Language Models in Autonomous Driving (VLM4AD). It extends the Bench2Drive benchmark by introducing closed-loop evaluation and the DriveCommenter expert model for automated annotation.
This repository contains the natural language annotations for the Bench2Drive-Base1000 dataset. These… See the full description on the dataset page: https://huggingface.co/datasets/Telkwevr/Bench2Drive-VL-base.based-fdaThis dataset is adapted from the paper Language Models Enable Simple Systems for Generating
Structured Views of Heterogeneous Data Lakes. You can learn more about the data collection process there.
Please consider citing the following if you use this task in your work:
@article{arora2024simple,
title={Simple linear attention language models balance the recall-throughput tradeoff},
author={Arora, Simran and Eyuboglu, Sabri and Zhang, Michael and Timalsina, Aman and Alberti, Silas and… See the full description on the dataset page: https://huggingface.co/datasets/hazyresearch/based-fda.based-swdeThis dataset is adapted from the paper Language Models Enable Simple Systems for Generating
Structured Views of Heterogeneous Data Lakes. You can learn more about the data collection process there.
Please consider citing the following if you use this task in your work:
@article{arora2024simple,
title={Simple linear attention language models balance the recall-throughput tradeoff},
author={Arora, Simran and Eyuboglu, Sabri and Zhang, Michael and Timalsina, Aman and Alberti, Silas and… See the full description on the dataset page: https://huggingface.co/datasets/hazyresearch/based-swde.rbo_oxe_base_droid_lerobotThis dataset is a pruned, one-camera subset of the original
IPEC-COMMUNITY/droid_lerobot:
Removed episodes with empty task text (e.g., "tasks": [""]).
Grouped by task string and kept ≤ 4 episodes per unique task.
For each kept episode, kept exactly one camera stream, chosen at random from{observation.images.exterior_image_1_left, observation.images.exterior_image_2_left, observation.images.wrist_image_left},
and unified it under the single key observation.images.image.
Reindexed episodes to… See the full description on the dataset page: https://huggingface.co/datasets/saaduddinM/rbo_oxe_base_droid_lerobot.dcvlm-baseline-6_25b
DCVLM-Baseline (6.25B tokens)
DCVLM-Baseline is the reference training mixture from our DataComp-VLM paper.
It is a pre-mixed, decontaminated, ready-to-train multimodal pretraining dataset, materialized as flat
WebDataset tar shards so it can be consumed by any training
stack.
This dataset version is a small 6.25B-token (small-pool) release consisting of 3,253,356 samples.
⚠️ NOTE: The training data is the WebDataset shards under shards/. The preview
config shown in the Dataset… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations/dcvlm-baseline-6_25b.based-squadPlease consider citing the following if you use this task in your work:
@inproceedings{Rajpurkar2018SQuAD2,
title={Know What You Don't Know: Unanswerable Questions for SQuAD},
author={Pranav Rajpurkar and Jian Zhang and Percy Liang},
booktitle={ACL 2018},
year={2018}
}
@article{arora2024simple,
title={Simple linear attention language models balance the recall-throughput tradeoff},
author={Arora, Simran and Eyuboglu, Sabri and Zhang, Michael and Timalsina, Aman and Alberti, Silas… See the full description on the dataset page: https://huggingface.co/datasets/hazyresearch/based-squad.ga420-adobe-baselines-questQwen3.5-4B-Base
juiceb0xc0de/Qwen3.5-4B-Base
A brain atlas for Qwen/Qwen3.5-4B-Base, a 32-layer hybrid that runs linear attention on 24 layers and full attention on the other 8. This is not a chat dataset or a benchmark. It is an internal-mechanics map built by running activations through a corpus of prompts and scoring what each layer, component, head, and feature direction is doing.
This is a base model, before any instruction tuning, so whatever structure shows up here was put there by… See the full description on the dataset page: https://huggingface.co/datasets/juiceb0xc0de/Qwen3.5-4B-Base.GITQA-Base-Pruned-LegacyGITQA-Base-Legacyso101-pi05-base-dataset
SO-101 / SO-100 unified manipulation dataset
A single LeRobot dataset built from 150 public SO-101 / SO-100 teleoperation
datasets collected from the Hugging Face Hub, plus 6 recorded by the author.
The sources were screened for action-space conformity, deduplicated, and
rewritten into one schema with a fixed three-camera layout.
github.com/jinnymo/so101-pi05-base — the
scripts that built this, stage by stage: the Hub crawl, the screening criteria, the
action-convention check… See the full description on the dataset page: https://huggingface.co/datasets/dongyoonkim/so101-pi05-base-dataset.knowledge-base
Attention Wiki — a living knowledge base on LLM attention
A citation-backed tree of knowledge about attention in large language
models, built collaboratively by autonomous agents. Agents read papers,
blogs, and model cards; distill them into structured, provenance-tracked pages;
and reconcile where sources agree, disagree, or leave a question open. Every
change lands through a reviewed Pull Request — so the canonical wiki is
curated, not just accumulated.
Contributing? Read… See the full description on the dataset page: https://huggingface.co/datasets/attention-wiki/knowledge-base.GUI_BASED_PLATFORMnanochat-climbmix-arithmetic-base10
nanochat ClimbMix + Base-10 Arithmetic
This dataset contains the first 170 shuffled ClimbMix training shards
used by nanochat's speedrun. The deterministic base-10 arithmetic corpus is
mixed into shards 00000..00149; the final
20 train shards are unchanged web-only padding.
The original validation shard (shard_06542.parquet) is also
copied unchanged.
Arithmetic corpus
Family
Examples
a + b = c (all ordered pairs 0..2000, two exposures)
8,008,002
a + b… See the full description on the dataset page: https://huggingface.co/datasets/Yujivus/nanochat-climbmix-arithmetic-base10.Sci-Base
Sci-Base: The Largest AI-Ready Scientific Foundation Dataset
🌌 The Sciverse Data Foundation
Sciverse is a comprehensive, multi-layered scientific data foundation designed to provide the ultimate data infrastructure for the AI for Science (AI4S) community. As scientific research becomes increasingly data-driven, Sciverse supplies the essential, high-quality data resources required to build robust scientific knowledge systems and accelerate research.
Sciverse… See the full description on the dataset page: https://huggingface.co/datasets/opendatalab/Sci-Base.jepa-qwen3-32b-pure-baselines-2026-05-25
JEPA-Align: Qwen3-32B Safety Defense Matrix
The complete 11-condition Qwen3-32B experiment for Predictive Representation
Alignment (PRA), the paired-view objective introduced in Predictive
Representation Alignment Improves Generalization in LLM Safety.
PRA aligns adversarially rewritten prompts with clean prompts expressing the
same intent. This release contains trained adapters, attack traces, benign
capability evaluations, machine-readable results, and paper-ready tables for… See the full description on the dataset page: https://huggingface.co/datasets/memo-ozdincer/jepa-qwen3-32b-pure-baselines-2026-05-25.