datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
OpenThoughts-1k-sample
[!NOTE]
We have released a paper for OpenThoughts! See our paper here.
Open-Thoughts-1k-sample
This is a 1k sample of the OpenThoughts-114k dataset.
Open synthetic reasoning dataset with high-quality examples covering math, science, code, and puzzles!
Inspect the content with rich formatting with Curator Viewer.
Available Subsets
default subset containing ready-to-train data used to finetune the OpenThinker-7B and OpenThinker-32B models:
ds =… See the full description on the dataset page: https://huggingface.co/datasets/ryanmarten/OpenThoughts-1k-sample.FineFineWeb-sample
FineFineWeb: A Comprehensive Study on Fine-Grained Domain Web Corpus
arXiv: Coming Soon
Project Page: Coming Soon
Blog: Coming Soon
Data Statistics
Domain (#tokens/#samples)
Iteration 1 Tokens
Iteration 2 Tokens
Iteration 3 Tokens
Total Tokens
Iteration 1 Count
Iteration 2 Count
Iteration 3 Count
Total Count
aerospace
5.77B
261.63M
309.33M
6.34B
9100000
688505
611034
10399539
agronomy
13.08B
947.41M
229.04M
14.26B
15752828
2711790
649404
19114022
artistic… See the full description on the dataset page: https://huggingface.co/datasets/m-a-p/FineFineWeb-sample.evolkit-logprobs-pipeline-75k-v2-sampleessential-web-1t-sample-fdc-partitioned
🌐 Essential-Web: FDC Level-2 Partitioned Dataset
📋 Dataset Description
This dataset contains a 1 trillion token sample from Essential-Web, partitioned by Free Decimal Correspondence (FDC) level-2 categories. Essential-Web is a 24-trillion-token web dataset with extensive document-level metadata designed to enable rapid dataset curation through SQL-like filtering.
🔍 Free Decimal Correspondence (FDC)
The FDC taxonomy is an open classification system… See the full description on the dataset page: https://huggingface.co/datasets/Research-EAI/essential-web-1t-sample-fdc-partitioned.audio_samples_1kbabilong-1k-samples
BABILong (1000 samples) : a long-context needle-in-a-haystack benchmark for LLMs
Preprint is on arXiv and code for LLM evaluation is available on GitHub.
BABILong Leaderboard with top-performing long-context models.
bAbI + Books = BABILong
BABILong is a novel generative benchmark for evaluating the performance of NLP models in
processing arbitrarily long documents with distributed facts.
It contains 9 configs, corresponding to different sequence lengths in tokens: 0k… See the full description on the dataset page: https://huggingface.co/datasets/RMT-team/babilong-1k-samples.common-crawl-sample
Common Crawl sample
A small unofficial random subset of the famous Common Crawl dataset.
60 random segment WET files were downloaded from Common Crawl on 2024-05-12.
Lines between 500 and 5000 characters long (inclusive) were kept.
Only unique texts were kept.
No other filtering.
Languages
Each text was assigned to one of the language codes using the GCLD3 Python package.
The Chinese texts were classified as either simplified, traditional, or Cantonese using the… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/common-crawl-sample.swebench-verified-sample-100-Qwen3-30B-evalrpj-v2-sampleThis is a mirror of the sample-10B subset of RedPajama-Data-V2 which we have re-uploaded in order to resolve issues with the original download script.
Getting Started
RedPajama-V2 is an open dataset for training large language models. The dataset includes over 100B text
documents coming from 84 CommonCrawl snapshots and processed using
the CCNet pipeline. Out of these, there are 30B documents in the corpus
that additionally come with quality signals. In addition, we also provide the… See the full description on the dataset page: https://huggingface.co/datasets/EleutherAI/rpj-v2-sample.owt-sampleThese files were created with the following script:
from datasets import load_dataset
from tqdm import tqdm
import io
dataset = load_dataset("Skylion007/openwebtext")['train']
split_dataset = dataset.train_test_split(train_size=2400000, test_size=60000, seed=0)
with io.open('data/owt_train.txt','w') as fopen:
listout = []
for data in tqdm(split_dataset['train']):
listout.append(data['text']+'<|endoftext|>')
if len(listout) > 1000:
_ =… See the full description on the dataset page: https://huggingface.co/datasets/stanford-cs336/owt-sample.Qwen-Image-2512_samplesThis dataset is a highly diverse set of high quality images generated with Qwen Image 2512.
Possible uses
Regularization images for training models based on Qwen Image 2512
Quality testing
Data source
The images were created in ComfyUI with the
bf16 version
of Qwen Image 2512. For each prompt were four images generated, all are (without any cherry picking) included in the corresponding dataset directories.
bf16 - full model weights
1328x1328 pixels - native resolution… See the full description on the dataset page: https://huggingface.co/datasets/stablellama/Qwen-Image-2512_samples.mc4-es-sampled50 million documents in Spanish extracted from mC4 applying perplexity sampling via mc4-sampling: "https://huggingface.co/datasets/bertin-project/mc4-sampling". Please, refer to BERTIN Project. The original dataset is the Multlingual Colossal, Cleaned version of Common Crawl's web crawl corpus (mC4), based on the Common Crawl dataset: "https://commoncrawl.org", and processed by AllenAI.vibevoice_samplesSource: https://github.com/vibevoice-community/VibeVoice/tree/main/demo
dolma3-6t-sample-10000-docs
dolma3-6t-sample-10000-docs
Materialized stratified sample of 10K docs per bin (5.68M total docs, 10.5B tokens). Seed 42. This is the basis for the SOC-156 TrackStar gradient index.
Provenance
This dataset was renamed on 2026-05-25 as part of the HCAI-Lab HF naming convention cleanup (PR 3). See docs/HCAI_LAB_NAMING_CONVENTION.md in the project repo for the convention.
Field
Value
Previous name
HCAI-Lab/dolma3_6T_sample_10000_docs
Renamed
2026-05-25… See the full description on the dataset page: https://huggingface.co/datasets/HCAI-Lab-GT/dolma3-6t-sample-10000-docs.wiki-sample
Loading dataset without vector embeddings
You can load the raw dataset without vectors, like this:
from datasets import load_dataset
dataset = load_dataset("weaviate/wiki-sample", split="train", streaming=True)
Loading dataset with vector embeddings
You can also load the dataset with vectors, like this:
from datasets import load_dataset
dataset = load_dataset("weaviate/wiki-sample", "weaviate-snowflake-arctic-v2", split="train", streaming=True)
for item in dataset:… See the full description on the dataset page: https://huggingface.co/datasets/weaviate/wiki-sample.RedPajama-Data-1T-Sample-BackupSampleToHiyori
SampleToHiyori
'모모세 히요리(桃瀬 ひより)' 페르소나 학습용 한국어 데이터셋. config 두 개로 이루어진다.
config
split
행 수
내용
default
train
4,837
단일 턴 한국어 페르소나 대화 (instruction / response)
tools
train / eval
5,495 / 322
도구 호출(function calling) 대화
히요리는 상대를 항상 "오빠" 라고 부르고, 일인칭은 "히요리", 말투는 "인걸" / "인거야" 다.
tools
OpenMascotAI 마스코트의 자비스 모드(윈도우를 실제로 조작하는 모드)에서 쓰기 위한 도구 호출
학습 데이터. 페르소나 LoRA를 얹으면 베이스 모델이 도구를 전혀 호출하지 않게 되는 현상을 고치려고
만들었다. 시나리오(도구·인자·결과·브리프)는 자매 데이터셋 MelissaJ/ProjectLucia_Hera… See the full description on the dataset page: https://huggingface.co/datasets/MelissaJ/SampleToHiyori.minipile_100_samplespractice-radar-behavioral-health-npi-sample
New behavioral-health organization NPIs — weekly NPPES sample
A 15-row public sample from a weekly, reproducible selection of newly enumerated Type 2 behavioral-health organizations in the U.S. Centers for Medicare & Medicaid Services National Plan and Provider Enumeration System (NPPES).
Edition at a glance
Measured period: July 6–12, 2026
New Type 2 organizations screened: 2,722
Behavioral-health organizations selected: 486
States and territories represented:… See the full description on the dataset page: https://huggingface.co/datasets/unitedideas/practice-radar-behavioral-health-npi-sample.Krea-2-Raw_samples_Best_ofThis dataset is a highly diverse set of high quality images generated with Krea 2 Raw.
NOTE: Raw is not intended for image generation, so do not use these images to judge the quality of the model.
Raw is intended for training, as are the samples in this dataset as they can be used for regularization.
Possible uses
Regularization images for training models based on Krea 2 Raw
Quality testing
Data source
This dataset is derived from… See the full description on the dataset page: https://huggingface.co/datasets/stablellama/Krea-2-Raw_samples_Best_of.dolma3_300B_sampleredbajama-sampledOpenThoughts-1k-SampledFLUX.2-klein-base-9B_samples_Best_ofThis dataset is a highly diverse set of high quality images generated with FLUX.2 [klein] 9B Base.
NOTE: The Base is not intended for image generation, so do not use these images to judge the quality of the model.
Base is intended for training, as are the samples in this dataset as they can be used for regularization.
Possible uses
Regularization images for training models based on FLUX.2 [klein] 9B Base
Quality testing
Data source
This dataset is derived from… See the full description on the dataset page: https://huggingface.co/datasets/stablellama/FLUX.2-klein-base-9B_samples_Best_of.omni-dreams-samples
AlpaDreams Samples
Curated single-view driving sequences for evaluating the
nvidia/alpadreams-dit world model.
Layout
data/
└── single_view/
├── <clip-id>/
| ├── <clip-id_...>.mp4 # ground truth video
│ ├── <clip-id_..._hdmap>.mp4 # HD-map rasterized conditioning video
│ ├── first_frame.png # RGB first frame, extracted from ground truth video
│ └── prompt.txt # text prompt
└──… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/omni-dreams-samples.RedPajama-Data-1T-Samplebabilong-train-5k-samples
BABILong (5k train samples) : a long-context needle-in-a-haystack benchmark for LLMs
Preprint is on arXiv
bAbI + Books = BABILong
BABILong is a novel generative benchmark for evaluating the performance of NLP models in
processing arbitrarily long documents with distributed facts.
It contains 10 configs, each corresponding to its bAbI task. Each config has spltis corresponding to different sequence lengths in tokens: '4k', '32k', '128k', '256k', '512k', '1M'
Solving tasks… See the full description on the dataset page: https://huggingface.co/datasets/RMT-team/babilong-train-5k-samples.Amazon_Sample_Metadata_2023
Dataset Card for Dataset Name
Original datasets can be found on: https://amazon-reviews-2023.github.io/
Dataset Details
This dataset was made as sample of several datasets from the link above.
Dataset Description
This dataset is a curated sample derived from seven filtered Amazon product category datasets(Amazon All Beauty, Amazon Fashion, Sports and Outdoors,
Health and Personal Care, Amazon Clothing Shoes and Jewlery,
Baby Products and Beauty and Personal… See the full description on the dataset page: https://huggingface.co/datasets/smartcat/Amazon_Sample_Metadata_2023.FLUX.2-klein-base-9B_samplesThis dataset is a highly diverse set of high quality images generated with FLUX.2 [klein] 9B Base.
NOTE: The Base is not intended for image generation, so do not use these images to judge the quality of the model.
Base is intended for training, as are the samples in this dataset as they can be used for regularization.
Possible uses
Regularization images for training models based on FLUX.2 [klein] 9B Base
Quality testing
Data source
The images were created in ComfyUI… See the full description on the dataset page: https://huggingface.co/datasets/stablellama/FLUX.2-klein-base-9B_samples.MedHEB-Bench_Sample
Note for reviewers: every JSON file in this repo is a test-only
sub-task. We declare three configs (2D, 3D, Text) and route all
files to the test split explicitly so the viewer does not mis-group
them based on filename heuristics. Image fields (qry_img_path,
tgt_img_path) are relative paths into the matching 2D_Images/ /
3D_Images/ folders.
MedHEB-Bench — Reviewer Sample
This repository is a small, self-contained sample of the full MedHEB-Bench
benchmark, hosted here so… See the full description on the dataset page: https://huggingface.co/datasets/MEDHEB/MedHEB-Bench_Sample.