WaveCut/DeepSeek-V4-Flash-0731-REAM-calibration-stats

DeepSeek-V4-Flash-0731 — expert calibration statistics (REAM line)

Layerwise routed-expert statistics of `deepseek-ai/DeepSeek-V4-Flash-0731` (43 MoE layers × 256 experts), collected by running the full model over a ~4.9M-token multi-domain calibration mix (multi-turn dialogs, thinking and direct modes, rendered with the model's own chat encoder). These are the statistics behind the REAM144/96 release line — published so that expert selection, pruning, merging and routing research can start WITHOUT the expensive H100 collection pass.

Files

• —layer_XX.npz (one per MoE layer):
• —saliency [8, 256] — per-domain REAP-style saliency S_i = f_i · E[gate_i · ‖expert_i(x)‖₂ | i ∈ Top-6]; domain order in domains.json;
• —freq [256] — Top-6 selection counts over the mix;
• —coact [256, 256] — joint Top-6 co-activation counts;
• —load [256] — historical load share (sums to 1);
• —gsum [256], token_count — auxiliary.
• —imatrix_raw.npz — squared-input accumulators per expert (gate [43, 256, H], down [43, 256, I], calls [43]) — the raw material for llama.cpp-style importance matrices of ANY expert subset or merge (weighted sums of member rows; see emit_imatrix_merged.py in the release pipeline).
• —domains.json — domain names (canonical order used by the saliency axis) and mix shares.
• —nll_heldout.json — the source model's NLL on the held-out slice of the same mix, collected during the pass (a reference point for compressed variants).

What these enable

• —Reproducing/improving the published selections (each release carries its SELECTION.json);
• —Expert pruning/merging experiments at zero collection cost;
• —Routing analyses: language/domain specialization by depth, co-activation cluster structure, load distributions.

Collection notes

Single pass, batch 6 × 4096 tokens, deterministic packer; the calibration texts themselves are NOT included and are not recoverable from these aggregates. Related models: the REAM144/96 line and its merge variants (see the collection on this profile).

MIT, following the source model.