tiny-moe
Datasets
All datasets matching “tiny-moe”glm-moe-dsa-tiny-cpu-repro-v1
Tiny GLM MoE DSA: two CPU captures, forced zero-KL replay
Reproducibility evidence for
malaiwah/glm-moe-dsa-tiny-random-bf16,
checkpoint/config/tokenizer revision 45563636ef723acfb826755493447dc40c7a0c37.
This is a synthetic pipeline test, not a quality benchmark, quantization measurement,
qualified production reference, or registry submission. The model is random-init.
No GPU or paid cloud job was used.
Observed result
Two fresh capture processes, two CPU… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/glm-moe-dsa-tiny-cpu-repro-v1.glm-moe-dsa-tiny-fidelity-root-v1
glm_moe_dsa random CPU fixture root
A root fidelity dataset in hidden form, produced by engines/tools/hf_capture.py from malaiwah/glm-moe-dsa-tiny-random-bf16.
The cut
the final hidden state handed to lm_head -- after the text model's final norm and immediately before the head matmul -- captured as the head module's input via torch.nn.Module.register_forward_pre_hook; replay applies the head ONLY (no final norm at replay time: the capture already sits after it).… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/glm-moe-dsa-tiny-fidelity-root-v1.phi-tiny-moe-fineweb-edu-topk-router-traces
Phi-tiny-MoE FineWeb-Edu top-k router traces
Native top-2 routing traces from microsoft/Phi-tiny-MoE-instruct over
a deterministic 10,000,000-token sample of
HuggingFaceFW/fineweb-edu (sample-10BT). Documents are
deterministically shuffled, assigned to document-isolated train/eval streams,
tokenized without prompt formatting, separated by EOS tokens, and packed into
sequences of at most 4,096 tokens.
Run:… See the full description on the dataset page: https://huggingface.co/datasets/hanspeterlyngsoeraaschoujensen/phi-tiny-moe-fineweb-edu-topk-router-traces.