CoolFace
30 shown

datasets

Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.

modalityAllTextImageAudioTabularVideo
taskAllText generationQuestion answeringImage classificationSpeech
sizeAllUnder 1K1K – 10K100K – 1M1M – 10M
formatAllParquetJSONCSVWebDataset
sortTrendingMost likedMost usedRecently updatedNewest
Clear all
01ReliableAI /irish_fineweb_eduData translation project of https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu, sample-10BT subset. Data are translated from English to Irish using NLLB-3.3B. tabular100K<n<1M1 likes6.7k downloads2y agoHugging Face02coms4705-hewitt /fineweb-linuxliketext100K<n<1M0 likes2.8k downloads1y agoHugging Face03alexkstern /fineweb-nanochatbpe-100M fineweb-nanochatbpe-100M FineWeb-Edu (from karpathy/fineweb-edu-100b-shuffle), pre-tokenized with the nanochatbpe tokenizer (vocab 65,536) and packaged as flat uint16 token-id .bin files for fast memmap training. This is a 100-million-token slice for data-constrained experiments. The train.bin is the byte-exact first 100,000,000 tokens (bytes [0, 200000000)) of the parent alexkstern/fineweb-nanochatbpe-20B train.bin. The val.bin is byte-identical to the parent's val.bin.… See the full description on the dataset page: https://huggingface.co/datasets/alexkstern/fineweb-nanochatbpe-100M.tabularn<1K0 likes2.3k downloads3mo agoHugging Face04jiviteshjn /fineweb-edu-zh-chengyu-cpt Fineweb-Edu Chinese — Chengyu-Tagged Continued-Pretraining Corpus A 3.74M-document Chinese corpus (~7.8B tokens) for continued pretraining on cultural knowledge in figurative language, built from the highest-quality tier of opencsg/Fineweb-Edu-Chinese-V2.1. Each document is educational Chinese text containing at least one culturally vetted chengyu, with an appended 【成语注释】 knowledge block listing every matched idiom's figurative meaning(s) and classical source citation. This is a… See the full description on the dataset page: https://huggingface.co/datasets/jiviteshjn/fineweb-edu-zh-chengyu-cpt.tabulartext-generation1M<n<10M1 likes1.8k downloads2mo agoHugging Face05alexkstern /fineweb-nanochatbpe-20B fineweb-nanochatbpe-20B FineWeb-Edu (from karpathy/fineweb-edu-100b-shuffle), pre-tokenized with the nanochatbpe tokenizer (vocab 65,536) and packaged as flat uint16 token-id .bin files for fast memmap training. file split tokens train.bin train 20,000,000,000 val.bin val 52,336,096 train and val are disjoint held-out partitions. Each .bin is a raw little-endian uint16 stream (no header); token count = filesize / 2, and train.meta.json / val.meta.jsoncarry the full… See the full description on the dataset page: https://huggingface.co/datasets/alexkstern/fineweb-nanochatbpe-20B.tabularn<1K0 likes952 downloads4mo agoHugging Face06RioYokotaLab /fineweb-edutabular100M<n<1B0 likes826 downloads1y agoHugging Face07trendmicro-ailab /Primus-FineWebgated PRIMUS: A Pioneering Collection of Open-Source Datasets for Cybersecurity LLM Training 🤗 Primus-FineWeb The Primus-FineWeb dataset is constructed by filtering cybersecurity-related text from FineWeb, a refined version of Common Crawl. We began by leveraging Primus-Seed, a high-quality dataset of manually curated cybersecurity text, as positive samples. We then sampled ten times the amount of data from FineWeb as negative samples and trained a binary cybersecurity… See the full description on the dataset page: https://huggingface.co/datasets/trendmicro-ailab/Primus-FineWeb.texttext-generation1M<n<10M27 likes806 downloads1y agoHugging Face08yoheikobashi /fineweb_edu_10B_for_crypto-LLMtext10M<n<100M0 likes666 downloads2y agoHugging Face09Mayank6255 /fineweb_2_samples_hq fineweb_2_samples_hq FINEWEB2-HQ dataset Dataset Structure This dataset contains 5 JSONL files with a total size of 26415.31 MB. Files: ukr_Cyrl_sample_001.jsonl: 6163.40 MB ron_Latn_sample_001.jsonl: 3739.69 MB kor_Hang_sample_001.jsonl: 4120.89 MB hin_Deva_sample_001.jsonl: 6681.96 MB heb_Hebr_sample_001.jsonl: 5709.37 MB Usage from datasets import load_dataset dataset = load_dataset("path/to/this/dataset") Loading specific files… See the full description on the dataset page: https://huggingface.co/datasets/Mayank6255/fineweb_2_samples_hq.tabulartext-generation10M<n<100M0 likes518 downloads1y agoHugging Face10Michaelyya /fineweb-edu-climatetext1M<n<10M0 likes446 downloads9mo agoHugging Face11opencsg /Fineweb-Edu-Chinese-V2.3 Chinese Fineweb Edu Dataset V2.3 中文 | English OpenCSG 社区 | GitHub | 数据集许可协议 数据集简介 Chinese Fineweb Edu Dataset V2.3 是 OpenCSG 面向中文教育、知识问答、指令微调和文本生成场景构建的高质量中文教育 SFT 数据集。 该版本包含 23.04 万条高质量中文教育 QA pairs，并将同一批问答对发布为 Alpaca、Messages、Messages-no-system 三种训练格式。三种格式面向不同训练模板，建议训练时按模型和框架选择其中一种格式使用，而不是将不同格式简单相加作为独立知识规模。 V2.3 是在 V2.2 基础上的质量升级版本。针对 V2.2 社区反馈和内部质量审计中出现的重复模式、异常中英文混入、噪声片段、弱证据支撑回答和低质量合成输出等问题，V2.3 提高了源文本进入生成环节的门槛，并优化了问答生成与过滤逻辑。 在数据构建上，V2.3 从约 2.3T… See the full description on the dataset page: https://huggingface.co/datasets/opencsg/Fineweb-Edu-Chinese-V2.3.texttext-generation100K<n<1M1 likes369 downloads3mo agoHugging Face12enPurified /smollm-corpus-fineweb-edu-enPurified-openai-messages 📖 smollm-corpus-fineweb-edu-enPurified-openai-messages smollm-corpus-fineweb-edu-enPurified is a highly curated, "prose-first" subset of the fineweb-edu-dedup subset found in HuggingFaceTB/smollm-corpus. The enPurified collection is built on a specific philosophy: Specialization. While the original dataset is excellent for general pre-training, high-quality fluent English prose often gets diluted when mixed with syntax-heavy code, rigid math formulas, or low-information web junk.… See the full description on the dataset page: https://huggingface.co/datasets/enPurified/smollm-corpus-fineweb-edu-enPurified-openai-messages.texttext-generation10M<n<100M1 likes343 downloads8mo agoHugging Face13data-is-better-together /fineweb-c-progresstabularn<1K9 likes307 downloads5mo agoHugging Face14JQL-AI /Fineweb_2_500k_removedtabular10M<n<100M0 likes298 downloads2y agoHugging Face15tok-project /fineweb_blttext10M<n<100M0 likes287 downloads6mo agoHugging Face16Michaelyya /fineweb-edu-v2-FastT FineWeb-Edu v2 - FastText Climate Filtered A climate and environment-focused subset of sraj/finewebedu-climate-v2, further filtered using a trained FastText binary classifier. Overview This dataset applies a supervised FastText climate classifier to the FineWeb-Edu climate v2 dataset. Each record includes a climate probability score from the classifier, providing a confidence measure for climate relevance. Pipeline Source: sraj/finewebedu-climate-v2… See the full description on the dataset page: https://huggingface.co/datasets/Michaelyya/fineweb-edu-v2-FastT.texttext-classification100K<n<1M0 likes267 downloads8mo agoHugging Face17NanoMatriX /fineweb-edu-dedup6mtext1M<n<10M0 likes259 downloads8mo agoHugging Face18ZhuofengLi /fineweb-edu-pretokenized-llama3-100b FineWeb-Edu Pretokenized with Llama 3.1 (100B) This repository contains the sample/100BT subset of FineWeb-Edu, pretokenized for Megatron-LM-style training with meta-llama/Meta-Llama-3.1-8B. It is a derived, pretokenized version of the upstream data, not an official Hugging Face FineWeb release. Dataset summary 140 indexed shards 97,270,686 non-empty documents 97,458,793,013 tokens English web text from FineWeb-Edu sample/100BT Source dataset revision:… See the full description on the dataset page: https://huggingface.co/datasets/ZhuofengLi/fineweb-edu-pretokenized-llama3-100b.tabularn<1K0 likes255 downloads2mo agoHugging Face19secmlr /fineweb-edutabular10M<n<100M0 likes252 downloads2mo agoHugging Face20tyzhu /fineweb_score4_sortedtabular10M<n<100M0 likes246 downloads6d agoHugging Face21dahara1 /FineWeb2-HQ-ja-20B元のデータセットFineWeb2-HQ 元のデータセットは多言語で巨大なため、扱いやすい用に日本語データを約200GBだけ抽出したデータセットです wc 結果 1763269 38541549 5370473709 fineweb_jpn_Jpan_chunk_0.jsonl 1784158 37430170 5370514369 fineweb_jpn_Jpan_chunk_1.jsonl 1639554 40065129 5370372344 fineweb_jpn_Jpan_chunk_10.jsonl 1575127 42167166 5370298354 fineweb_jpn_Jpan_chunk_11.jsonl 1686375 39225898 5370402506 fineweb_jpn_Jpan_chunk_12.jsonl 1786948 36456352 5370498572… See the full description on the dataset page: https://huggingface.co/datasets/dahara1/FineWeb2-HQ-ja-20B.text10M<n<100M0 likes228 downloads1y agoHugging Face228nuoma /fineweb-4langtext10M<n<100M0 likes193 downloads28d agoHugging Face23JupiterLLM /fineweb_2_500k_both_deduplicatedtabular1M<n<10M0 likes192 downloads1y agoHugging Face24JQL-AI /Fineweb_2_500k_filteredtabular10M<n<100M0 likes175 downloads2y agoHugging Face25yoheikobashi /fineweb-edu-100BT-samples-not-in-10BTtext100K<n<1M0 likes159 downloads1y agoHugging Face26TIGER-Lab /Fineweb-InstructWe convert the pre-training corpus from Fineweb-Edu (https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) to instruction following format. We select a subset with quality filter and then use GPT-4 to extract instruction-following pairs. The dataset contains roughly 16M instruction pairs. The basic concept is similar to MAmmoTH2 (https://arxiv.org/abs/2405.03548). Citation If you use dataset useful, please cite the following paper: @article{yue2024mammoth2… See the full description on the dataset page: https://huggingface.co/datasets/TIGER-Lab/Fineweb-Instruct.textquestion-answering10M<n<100M9 likes137 downloads2y agoHugging Face27model-garden-lms /finewebs-scandeval-results ScandEval Results on English NLU We use ScandEval in revision 8766d2a to conduct experiments with our pretrained FineWeb LMs. Additionally, results for BERT, RoBERTa and ELECTRA were also performed to have a nice comparison. Model ID Avg. Score CoNLL-En SST5 ScaLA-En SQuAD model-garden-lms/bert-base-finewebs-1m 69.03 88.98 ± 0.43 / 88.67 ± 0.36 58.11 ± 1.2 / 59.77 ± 1.49 57.29 ± 3.57 / 77.15 ± 2.17 55.82 ± 1.35 / 66.46 ± 1.51 model-garden-lms/bert-base-finewebs-951k… See the full description on the dataset page: https://huggingface.co/datasets/model-garden-lms/finewebs-scandeval-results.tabularn<1K0 likes134 downloads2y agoHugging Face28ZefanW /fineweb_class5-0tabular1M<n<10M0 likes119 downloads2y agoHugging Face29JQL-AI /Fineweb_2_500k_bothtabular10M<n<100M0 likes112 downloads2y agoHugging Face30agentlans /fineweb2-chinese FineWeb2 - Chinese From cmn_Hani subset Region Rows MAINLAND_CHINA 834356 TAIWAN 83875 HONG_KONG 10411 AMBIGUOUS_TRADITIONAL_TW_OR_HK 67700 OTHER 3658 tabular1M<n<10M0 likes100 downloads4mo agoHugging Face

Listings come live from the Hugging Face Hub API. CoolFace does not host these files.