datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
yeast-no-GTL-pretokenized-NT-v2human-pretokenized-NTmouse-pretokenized-NTyeast-gene-sequence-homology-pretokenized-NTyeast-no-label-pretokenized-NTBEELINE-HepG2-no-GTL-pretokenized-NThuman_and_mouse-pretokenized-NTBEELINE-mDC-no-label-pretokenized-NTBEELINE-mouse-v2-pretokenized-NTBEELINE-human-v2-pretokenized-NTBEELINE-mESC-no-label-pretokenized-NTBEELINE-mHSC-no-label-pretokenized-NTpretraining-pretokenized-smollm3
SmolLM3 Pretokenized Pretraining Sources
Datatrove/Nanotron tokenized-byte versions of three public pretraining sources:
fineweb-edu-10bt: HuggingFaceFW/fineweb-edu, sample/10BT
finemath-4plus: HuggingFaceTB/finemath, finemath-4plus
stack-edu-python: HuggingFaceTB/stack-edu, Python, with content retrieved from the public Software Heritage S3 bucket using the upstream dataset-card procedure
All subsets use HuggingFaceTB/SmolLM3-3B at revision… See the full description on the dataset page: https://huggingface.co/datasets/ZhuofengLi/pretraining-pretokenized-smollm3.BEELINE-hESC-no-GTL-pretokenized-NTBEELINE-mDC-no-GTL-pretokenized-NTfineweb-edu-pretokenized-10b
FineWeb-Edu Pretokenized 10B
Megatron indexed-dataset (.bin/.idx) versions of
HuggingFaceFW/fineweb-edu sample/10BT.
Each Hugging Face subset contains a small metadata.parquet index. The actual
Megatron files are under <subset>/files/; pass each prefix without the
.bin/.idx suffix to Megatron Core or Megatron Bridge.
Documents retain upstream shard and row order. Tokenization disables automatic
special-token insertion and appends exactly one tokenizer EOS/EOD token to each… See the full description on the dataset page: https://huggingface.co/datasets/ZhuofengLi/fineweb-edu-pretokenized-10b.yeast-tf-sequence-homology-pretokenized-NTBEELINE-mESC-no-GTL-pretokenized-NTBEELINE-HepG2-no-label-pretokenized-NTBEELINE-hESC-no-label-pretokenized-NTpretokenized__HuggingFaceFW_fineweb-edu__EleutherAI__gpt-neo-125mBEELINE-mHSC-no-GTL-pretokenized-NTmulti_species-pretokenized-NTBEELINE-human-pretokenized-NTBEELINE-mouse-pretokenized-NTpretokenized_gold_standardyeast-promoters-noGTL-pretokenized-NThuman_and_mouse_2-pretokenized-NTpretokenized__HuggingFaceFW_fineweb-edu__EleutherAI__gpt-neo-125mtokenized_uniprotkb_binary_label_1024_length_pre_split_random_1_percent
