thepowerfuldeez/1226_imu1_base_decay_corpus

IMU-1 Stage 2 Training Corpus (Decay Phase) Pre-tokenized training data for Stage 2 (decay phase) of IMU-1, a sample-efficient 430M parameter language model. Dataset Details Property Value Tokens ~28B Format Memory-mapped NumPy (.npy) Tokenizer SmolLM2-360M Vocab size 49,152 Data Sources Stage 2 uses tighter quality filters compared to Stage 1: DCLM-edu (higher threshold filtering) FineWeb-edu FineMath Curated high-quality… See the full description on the dataset page: https://huggingface.co/datasets/thepowerfuldeez/1226_imu1_base_decay_corpus.