datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fineweb-edu-zh-chengyu-cpt
Fineweb-Edu Chinese — Chengyu-Tagged Continued-Pretraining Corpus
A 3.74M-document Chinese corpus (~7.8B tokens) for continued pretraining on
cultural knowledge in figurative language, built from the highest-quality
tier of opencsg/Fineweb-Edu-Chinese-V2.1.
Each document is educational Chinese text containing at least one culturally
vetted chengyu, with an appended 【成语注释】 knowledge block listing every
matched idiom's figurative meaning(s) and classical source citation.
This is a… See the full description on the dataset page: https://huggingface.co/datasets/jiviteshjn/fineweb-edu-zh-chengyu-cpt.Primus-FineWeb
PRIMUS: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Training
🤗 Primus-FineWeb
The Primus-FineWeb dataset is constructed by filtering cybersecurity-related text from FineWeb, a refined version of Common Crawl. We began by leveraging Primus-Seed, a high-quality dataset of manually curated cybersecurity text, as positive samples. We then sampled ten times the amount of data from FineWeb as negative samples and trained a binary cybersecurity… See the full description on the dataset page: https://huggingface.co/datasets/trendmicro-ailab/Primus-FineWeb.fineweb_2_samples_hq
fineweb_2_samples_hq
FINEWEB2-HQ dataset
Dataset Structure
This dataset contains 5 JSONL files with a total size of 26415.31 MB.
Files:
ukr_Cyrl_sample_001.jsonl: 6163.40 MB
ron_Latn_sample_001.jsonl: 3739.69 MB
kor_Hang_sample_001.jsonl: 4120.89 MB
hin_Deva_sample_001.jsonl: 6681.96 MB
heb_Hebr_sample_001.jsonl: 5709.37 MB
Usage
from datasets import load_dataset
dataset = load_dataset("path/to/this/dataset")
Loading specific files… See the full description on the dataset page: https://huggingface.co/datasets/Mayank6255/fineweb_2_samples_hq.Fineweb-Edu-Chinese-V2.3
Chinese Fineweb Edu Dataset V2.3
中文 | English
OpenCSG 社区 | GitHub | 数据集许可协议
数据集简介
Chinese Fineweb Edu Dataset V2.3 是 OpenCSG 面向中文教育、知识问答、指令微调和文本生成场景构建的高质量中文教育 SFT 数据集。
该版本包含 23.04 万条高质量中文教育 QA pairs，并将同一批问答对发布为 Alpaca、Messages、Messages-no-system 三种训练格式。三种格式面向不同训练模板，建议训练时按模型和框架选择其中一种格式使用，而不是将不同格式简单相加作为独立知识规模。
V2.3 是在 V2.2 基础上的质量升级版本。针对 V2.2 社区反馈和内部质量审计中出现的重复模式、异常中英文混入、噪声片段、弱证据支撑回答和低质量合成输出等问题，V2.3 提高了源文本进入生成环节的门槛，并优化了问答生成与过滤逻辑。
在数据构建上，V2.3 从约 2.3T… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V2.3.smollm-corpus-fineweb-edu-enPurified-openai-messages
📖 smollm-corpus-fineweb-edu-enPurified-openai-messages
smollm-corpus-fineweb-edu-enPurified is a highly curated, "prose-first" subset of the fineweb-edu-dedup subset found in HuggingFaceTB/smollm-corpus.
The enPurified collection is built on a specific philosophy: Specialization. While the original dataset is excellent for general pre-training, high-quality fluent English prose often gets diluted when mixed with syntax-heavy code, rigid math formulas, or low-information web junk.… See the full description on the dataset page: https://huggingface.co/datasets/enPurified/smollm-corpus-fineweb-edu-enPurified-openai-messages.epfml-FineWeb2-HQ-sample
epfml/FineWeb2-HQ
A curated subset of the epfml/FineWeb2-HQ dataset featuring high-quality multilingual text.
Details
First 25 000 rows per config (language and script pair)
Duplicates removed
Texts truncated to 512 LLaMA 3.1 tokens
Scores transformed with log10
Rows shuffled and 20% of the rows split into the test set (stratified by config)
Example
{
"text": "爵士大师Tim Garland 深圳专场 - [jazz]\nTim Garland Lighthouse… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/epfml-FineWeb2-HQ-sample.AraToken-FineWeb2-HQ-ar
AraToken FineWeb2-HQ Arabic splits
These are the exact document splits used in
AraToken (code). They were drawn from the arb_Arab part of
epfml/FineWeb2-HQ and are redistributed
under its ODC-By 1.0 license.
A document goes to a split by blake2b(id, digest_size=8) mod 1000, so the splits are disjoint by
construction.
split
buckets
role
documents
characters
lm-train
0–599
LEP and CPT adaptation
662,685
2.00 B
tok-train
600–899
tokenizer training and pruning
159,826… See the full description on the dataset page: https://huggingface.co/datasets/mariklolik/AraToken-FineWeb2-HQ-ar.fineweb2-id-filtered-10k
FineWeb2-ID Filtered (min 10.000 karakter)
Dataset ini adalah subset turunan dari HuggingFaceFW/fineweb-2,
subset bahasa Indonesia (ind_Latn), yang disaring dengan syarat:
Panjang teks (setelah strip whitespace) minimal 10.000 karakter per dokumen
Diambil secara streaming dari split train, urutan asli (tanpa shuffle)
Jumlah sampel: 2,097,152 dokumen, tersebar di 80 file JSONL.
Sumber & Lisensi
Seluruh isi teks berasal dari FineWeb-2 (lisensi ODC-By), yang pada… See the full description on the dataset page: https://huggingface.co/datasets/tampakwilll/fineweb2-id-filtered-10k.finewebedu-guru
FineWebEdu-Guru
A high-quality dataset collection for training interactive expert large language models (LLMs)
These are general educational web content with no specific focus
To specialize the LLMs for your own data, you'll need other models to generate the training data such as
agentlans/Qwen2.5-1.5B-Refiner
agentlans/Qwen2.5-1.5B-Instruct-Conversation-Maker
agentlans/Qwen2.5-1.5B-Instruct-Multiple-Choice-Maker
agentlans/Qwen2.5-1.5B-Instruct-Short-Answer-Maker
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/finewebedu-guru.FineWeb-Edu-Spanish
High Quality Spanish Corpus
This dataset contains a sample of a large collection of high-quality Spanish text data with their metadata.
To access the full data please visit Token Haven
Creation
The dataset was created by filtering all English common crawl data for high-quality text using the FineWeb-Edu classifier with education score of 4 or higher over 5.
The data is source from the v1.0.0 of the HuggingFaceFW/fineweb-edu dataset which corresponds to… See the full description on the dataset page: https://huggingface.co/datasets/TokenHaven/FineWeb-Edu-Spanish.FineWeb-Edu-Arabic
High Quality Arabic Corpus
This dataset contains a large collection of high-quality Arabic text data with their metadata.
To access the full data please visit Token Haven
Creation
The dataset was created by filtering all English common crawl data for high-quality text using the FineWeb-Edu classifier with education score of 4 or higher over 5.
The data is source from the v1.0.0 of the HuggingFaceFW/fineweb-edu dataset which corresponds to CC-MAIN-2024-10 from… See the full description on the dataset page: https://huggingface.co/datasets/TokenHaven/FineWeb-Edu-Arabic.anima-corpus-ko-fineweb2-broad
anima-corpus-ko-fineweb2-broad
🇰🇷 한국어 broad (일반) 코퍼스 for anima conv 303M byte-level pretraining.
anima chat register 표준(a_chat_registers)의 4칸 {ko·en} × {일반·SNS} 중 ko-일반 칸을 메우기 위한 데이터셋이다. 기존 ko-일반 source 가 ~1.7MB 로 빈약(en ~202MB 대비)했던 갭을 FineWeb-2 한국어로 보강한다.
Source
Upstream: HuggingFaceFW/fineweb-2, config kor_Hang (한국어 한글 스크립트).
Extracted from: train parquet data/kor_Hang/train/000_00000.parquet (1 of 25 shards).
Field: text 컬럼만 추출 (raw UTF-8, byte-vocab256… See the full description on the dataset page: https://huggingface.co/datasets/dancinlab/anima-corpus-ko-fineweb2-broad.finewebedu-sft
FineWeb-Edu Supervised Finetuning Dataset
Model Description
This dataset is designed for training language models to generate supervised finetuning data from raw text. It consists of text passages and corresponding question-answer pairs in JSONLines format.
Intended Use
The primary purpose of this dataset is to enable large language models (LLMs) to generate high-quality supervised finetuning data from raw text inputs, useful for creating custom datasets for… See the full description on the dataset page: https://huggingface.co/datasets/agentlans/finewebedu-sft.MixtureVitae-fineweb-permissive-multilingual-2m
MixtureVitae Fineweb-Permissive-Multilingual-2M: 2 Million Translated Documents Of Permissive Text From Fineweb-edu-2
Dataset Summary
This is a translation of a small subset of the Fineweb-edu-2 dataset. We have filtered to find websites with what we believe are government domain names, international organization domain names like the UN and europa.eu, and creative commons licensed data. While we strongly believe that fair use protects machine learning on webcrawled data… See the full description on the dataset page: https://huggingface.co/datasets/ontocord/MixtureVitae-fineweb-permissive-multilingual-2m.FineWeb-Edu-Norwegian
High Quality Norwegian Corpus
This dataset contains a large collection of high-quality Norwegian text data with their metadata.
To access the full data please visit Token Haven
Creation
The dataset was created by filtering all English common crawl data for high-quality text using the FineWeb-Edu classifier with education score of 4 or higher over 5.
The data is source from the v1.0.0 of the HuggingFaceFW/fineweb-edu dataset which corresponds to CC-MAIN-2024-10… See the full description on the dataset page: https://huggingface.co/datasets/TokenHaven/FineWeb-Edu-Norwegian.fineweb_10k
fineweb_10k
10,000 text samples from HuggingFaceFW/fineweb (train split), streamed and
saved as-is (no shuffling needed beyond the source stream's own order).
Companion single-source dataset to
vuhaian/24_collected
(23,000 examples across 23 sources) — used as the single-source comparison
baseline in continue-pretraining experiments.
Files
fineweb_10k.jsonl -- raw text, one JSON object per line:
{"text": ..., "source": "fineweb", "repo_id":… See the full description on the dataset page: https://huggingface.co/datasets/vuhaian/fineweb_10k.