gyung/gdn2-cpt-fineweb-edu-30k

gdn2-370m CPT pack — gdn2-cpt-fineweb-edu-30k Pretraining-corpus pack used for the GDN-2 370M SSKetch+ReMoE 4-way CPT runs (2026-08-23). Source & attribution Origin dataset: HuggingFaceFW/fineweb-edu config='sample-100BT' (public; used under the terms stated on its dataset card) Sampling recipe: streaming skip=800,000, num_samples=30,000 documents Tokenizer: TinyLlama/TinyLlama_v1.1 Total: 33,569,012 tokens across 30,000 documents Contents… See the full description on the dataset page: https://huggingface.co/datasets/gyung/gdn2-cpt-fineweb-edu-30k.