datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fineweb-edu-zh-chengyu-cpt
Fineweb-Edu Chinese — Chengyu-Tagged Continued-Pretraining Corpus
A 3.74M-document Chinese corpus (~7.8B tokens) for continued pretraining on
cultural knowledge in figurative language, built from the highest-quality
tier of opencsg/Fineweb-Edu-Chinese-V2.1.
Each document is educational Chinese text containing at least one culturally
vetted chengyu, with an appended 【成语注释】 knowledge block listing every
matched idiom's figurative meaning(s) and classical source citation.
This is a… See the full description on the dataset page: https://huggingface.co/datasets/jiviteshjn/fineweb-edu-zh-chengyu-cpt.fineweb-edufineweb_edu_10B_for_crypto-LLMfineweb-edu-climateFineweb-Edu-Chinese-V2.3
Chinese Fineweb Edu Dataset V2.3
中文 | English
OpenCSG 社区 | GitHub | 数据集许可协议
数据集简介
Chinese Fineweb Edu Dataset V2.3 是 OpenCSG 面向中文教育、知识问答、指令微调和文本生成场景构建的高质量中文教育 SFT 数据集。
该版本包含 23.04 万条高质量中文教育 QA pairs，并将同一批问答对发布为 Alpaca、Messages、Messages-no-system 三种训练格式。三种格式面向不同训练模板，建议训练时按模型和框架选择其中一种格式使用，而不是将不同格式简单相加作为独立知识规模。
V2.3 是在 V2.2 基础上的质量升级版本。针对 V2.2 社区反馈和内部质量审计中出现的重复模式、异常中英文混入、噪声片段、弱证据支撑回答和低质量合成输出等问题，V2.3 提高了源文本进入生成环节的门槛，并优化了问答生成与过滤逻辑。
在数据构建上，V2.3 从约 2.3T… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V2.3.fineweb-edu-v2-FastT
FineWeb-Edu v2 - FastText Climate Filtered
A climate and environment-focused subset of sraj/finewebedu-climate-v2, further filtered using a trained FastText binary classifier.
Overview
This dataset applies a supervised FastText climate classifier to the FineWeb-Edu climate v2 dataset. Each record includes a climate probability score from the classifier, providing a confidence measure for climate relevance.
Pipeline
Source: sraj/finewebedu-climate-v2… See the full description on the dataset page: https://huggingface.co/datasets/Michaelyya/fineweb-edu-v2-FastT.fineweb-edu-dedup6mfineweb-edu-pretokenized-llama3-100b
FineWeb-Edu Pretokenized with Llama 3.1 (100B)
This repository contains the sample/100BT subset of FineWeb-Edu, pretokenized for Megatron-LM-style training with meta-llama/Meta-Llama-3.1-8B.
It is a derived, pretokenized version of the upstream data, not an official Hugging Face FineWeb release.
Dataset summary
140 indexed shards
97,270,686 non-empty documents
97,458,793,013 tokens
English web text from FineWeb-Edu sample/100BT
Source dataset revision:… See the full description on the dataset page: https://huggingface.co/datasets/ZhuofengLi/fineweb-edu-pretokenized-llama3-100b.fineweb-edufineweb-edu-100BT-samples-not-in-10BTFineWeb-Edu-Sample-BT100-1-4The 1-4 parquet files of the fineweb-edu sample bt100 mixed up and turned up to 1 jsonl file. This is for users who dont have a NASA PC.
fineweb-edu-1m
train splitt = 1M
validation nsplitt = 1024
fineweb-edu-100KCode used to create this:
import json
from itertools import islice
from datasets import load_dataset
from tqdm import tqdm
DATASET_NAME = "HuggingFaceFW/fineweb-edu"
DATASET_CONFIG = "sample-10BT"
NUM_SAMPLES = 100_000
BATCH_SIZE = 5_000
OUTPUT_PATH = "pretrain_100k.jsonl"
dataset = load_dataset(
DATASET_NAME,
DATASET_CONFIG,
split="train",
streaming=True,
)
dataset_iter = iter(dataset)
written = 0
with open(OUTPUT_PATH, "w", encoding="utf-8") as f:
progress = tqdm(… See the full description on the dataset page: https://huggingface.co/datasets/mlx-community/fineweb-edu-100K.fineweb-edu-100b-smollmv2-tokenizedFineWeb-Edu-Analytic
FineWeb-Edu-Analytic (v1)
FineWeb-Edu-Analytic (v1) is an English-language dataset containing 9908 documents, intended as a resource for training language models.
The dataset was generated by taking text sequences from the FineWeb-Edu dataset (CC-MAIN-2025-26 subset) to serve as a source. Each source sequence was then processed by a 48-billion parameter language model to generate a corresponding structured, analytical document.
Disclaimer: This dataset is not affiliated with the… See the full description on the dataset page: https://huggingface.co/datasets/MultivexAI/FineWeb-Edu-Analytic.fineweb_edu_10btfinewebedu-conversation
FineWebEdu-Conversation
FineWebEdu-Conversation is a dataset containing excerpts from HuggingFaceFW/fineweb-edu, enriched with generated conversations between an ordinary person and an expert. The format simulates a natural interview-style dialogue.
Dataset Creation
Source: FineWebEdu excerpts.
Methodology:
Chunking: Segmented into chunks of approximately 1950 Llama 3 tokens.
Conversation Generation: Used agentlans/Llama3.1-LexiHermes-SuperStorm
and a model distilled… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/finewebedu-conversation.1M-finewebedu-samples256tTotal tokens in matching entries: 196_670_428
Average tokens per entry: 196.67
finewebedu-guru
FineWebEdu-Guru
A high-quality dataset collection for training interactive expert large language models (LLMs)
These are general educational web content with no specific focus
To specialize the LLMs for your own data, you'll need other models to generate the training data such as
agentlans/Qwen2.5-1.5B-Refiner
agentlans/Qwen2.5-1.5B-Instruct-Conversation-Maker
agentlans/Qwen2.5-1.5B-Instruct-Multiple-Choice-Maker
agentlans/Qwen2.5-1.5B-Instruct-Short-Answer-Maker
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/finewebedu-guru.finewebedu-nli
Dataset Card for Fineweb-edu NLI Dataset
Dataset Description
The Fineweb-edu NLI Dataset consists of pairs of sentences, where each pair includes a "premise" and a "hypothesis." The goal is to classify the relationship between the premise and hypothesis into three categories:
Label 0: The hypothesis is supported by the premise (entailment).
Label 1: The hypothesis is related but not directly supported by the premise (neutral).
Label 2: The hypothesis contradicts the… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/finewebedu-nli.FineWeb-Edu-Spanish
High Quality Spanish Corpus
This dataset contains a sample of a large collection of high-quality Spanish text data with their metadata.
To access the full data please visit Token Haven
Creation
The dataset was created by filtering all English common crawl data for high-quality text using the FineWeb-Edu classifier with education score of 4 or higher over 5.
The data is source from the v1.0.0 of the HuggingFaceFW/fineweb-edu dataset which corresponds to… See the full description on the dataset page: https://huggingface.co/datasets/TokenHaven/FineWeb-Edu-Spanish.FineWeb-Edu-Arabic
High Quality Arabic Corpus
This dataset contains a large collection of high-quality Arabic text data with their metadata.
To access the full data please visit Token Haven
Creation
The dataset was created by filtering all English common crawl data for high-quality text using the FineWeb-Edu classifier with education score of 4 or higher over 5.
The data is source from the v1.0.0 of the HuggingFaceFW/fineweb-edu dataset which corresponds to CC-MAIN-2024-10 from… See the full description on the dataset page: https://huggingface.co/datasets/TokenHaven/FineWeb-Edu-Arabic.FineWeb-Edu-1BTA subset of FineWeb-Edu randomly sampled from the whole dataset of around 1B gpt2 tokens.
This dataset is created for illustration purpose in retrieval-scaling. Please do not distribute.
FineWeb-Edu-1MTA subset of FineWeb-Edu randomly sampled from the whole dataset of around 1M gpt2 tokens.
This dataset is created for illustration purpose in retrieval-scaling. Please do not distribute.
fineweb_edu_10bt_shuffledfineweb-edu-500k200K samples
100k-finewebedu-samples2048tTotal tokens in matching entries: 140_782_625
Average tokens per entry: 1407.83
1M-finewebedu-samples1024tTotal tokens in matching entries: 732318957
Average tokens per entry: 732.32
1M-finewebedu-samples2048tTotal tokens in matching entries: 1_392_312_785
Average tokens per entry: 1392.31
finewebedu-sft
FineWeb-Edu Supervised Finetuning Dataset
Model Description
This dataset is designed for training language models to generate supervised finetuning data from raw text. It consists of text passages and corresponding question-answer pairs in JSONLines format.
Intended Use
The primary purpose of this dataset is to enable large language models (LLMs) to generate high-quality supervised finetuning data from raw text inputs, useful for creating custom datasets for… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/finewebedu-sft.