hanspeterlyngsoeraaschoujensen/phi-tiny-moe-fineweb-edu-topk-router-traces

Phi-tiny-MoE FineWeb-Edu top-k router traces Native top-2 routing traces from microsoft/Phi-tiny-MoE-instruct over a deterministic 10,000,000-token sample of HuggingFaceFW/fineweb-edu (sample-10BT). Documents are deterministically shuffled, assigned to document-isolated train/eval streams, tokenized without prompt formatting, separated by EOS tokens, and packed into sequences of at most 4,096 tokens. Run:… See the full description on the dataset page: https://huggingface.co/datasets/hanspeterlyngsoeraaschoujensen/phi-tiny-moe-fineweb-edu-topk-router-traces.