imu-1
Datasets
All datasets matching “imu-1”1226_imu1_base_decay_corpus
IMU-1 Stage 2 Training Corpus (Decay Phase)
Pre-tokenized training data for Stage 2 (decay phase) of IMU-1, a sample-efficient 430M parameter language model.
Dataset Details
Property
Value
Tokens
~28B
Format
Memory-mapped NumPy (.npy)
Tokenizer
SmolLM2-360M
Vocab size
49,152
Data Sources
Stage 2 uses tighter quality filters compared to Stage 1:
DCLM-edu (higher threshold filtering)
FineWeb-edu
FineMath
Curated high-quality sources… See the full description on the dataset page: https://huggingface.co/datasets/thepowerfuldeez/1226_imu1_base_decay_corpus.1218_imu1_base_stable_corpus
IMU-1 Stage 1 Training Corpus (Stable Phase)
Pre-tokenized training data for Stage 1 (stable phase) of IMU-1, a sample-efficient 430M parameter language model.
Dataset Details
Property
Value
Tokens
~29B
Format
Memory-mapped NumPy (.npy)
Tokenizer
SmolLM2-360M
Vocab size
49,152
Data Sources
High-quality filtered web data including:
DCLM-edu (educational content filtered from DCLM)
FineWeb-edu
Curated web sources
Download… See the full description on the dataset page: https://huggingface.co/datasets/thepowerfuldeez/1218_imu1_base_stable_corpus.imu1_base_decay_corpusimu1_base_midtrain_corpusimu1_base_stable_corpus
