scottgl/MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10

MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10

A fully-quantized compressed-tensors checkpoint of `catplusplus/MiniMax-M2.7-REAP-172B-A10B-NVFP4`, reshaped to match the layout of `saricles/MiniMax-M2.5-REAP-172B-A10B-NVFP4-GB10` and tuned for the NVIDIA GB10 Grace Blackwell (SM 12.1) Spark node, while remaining loadable on upstream vLLM and SGLang without patches.

What changed vs. the source catplusplus checkpoint

Approximate size: ~92 GiB across 19 shards (≈5 GiB each), down from the source catplusplus checkpoint's ~101 GiB.

Quantization scheme

config.json.quantization_config has two groups:

{
  "quant_method": "compressed-tensors",
  "format": "nvfp4-pack-quantized",
  "version": "0.14.1.dev0",
  "config_groups": {
    "group_0": {
      "targets": ["Linear"],
      "weights":             { "num_bits": 4, "group_size": 16, "type": "float", "symmetric": true, "strategy": "tensor_group", "scale_dtype": "torch.float8_e4m3fn" },
      "input_activations":   { "num_bits": 4, "group_size": 16, "type": "float", "symmetric": true, "strategy": "tensor_group", "dynamic": "local", "scale_dtype": "torch.float8_e4m3fn" }
    },
    "group_1": {
      "targets": ["re:.*lm_head$"],
      "format":  "float-quantized",
      "weights": { "num_bits": 8, "type": "float", "strategy": "tensor", "symmetric": true, "dynamic": false }
    }
  },
  "ignore": [ "... all 62 MoE gate routers ...", "model.embed_tokens" ],
  "kv_cache_scheme": null
}

All tensors ship in the plain (not Marlin-pre-interleaved) compressed-tensors layout. GB10-specific layout repacking is performed at load time by the SGLang SM12.1 path; upstream vLLM / SGLang use their normal FlashInfer / CUTLASS paths.

Serving

Upstream SGLang

python3 -m sglang.launch_server \
    --model-path scottgl/MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10 \
    --port 30000 \
    --trust-remote-code

Upstream vLLM

python3 -m vllm.entrypoints.openai.api_server \
    --model scottgl/MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10 \
    --trust-remote-code \
    --quantization compressed-tensors

GB10-optimized (scottgl SGLang fork)

On an SM 12.1 Spark node, the `scottgl9/sglang-spark-gb10-optimizations` fork auto-detects the capability and repacks weights for Marlin FP4 dense GEMM at load time. You should see one log line per NVFP4 Linear: SM121 (GB10): using Marlin FP4 dense GEMM for NVFP4 layer (N=..., K=...). Target decode throughput ≥ 35 tok/s.

./sglang.sh minimax-m27 --model-path scottgl/MiniMax-M2.7-REAP-172B-A10B-NVFP4-GB10

Architecture (from config.json)

• —Layers: 62 main + 3 MTP
• —Hidden: 3072
• —Heads: 48 (head_dim=128), KV heads: 8 (grouped query)
• —Experts per layer: 192 (REAP-pruned from 256); top-k: 8
• —FFN intermediate: 1536 (per expert)
• —Vocab: 200,064; Max context: 196,608

Provenance and attribution

Original model: `catplusplus/MiniMax-M2.7-REAP-172B-A10B-NVFP4` — this checkpoint is a direct re-formatting / completion-quantization of that model. All weight values for MoE experts (the bulk of the parameters) are numerically identical to catplusplus's checkpoint (rename-only transformation, dequant round-trip delta < 1e-8). Attention Q/K/V/O and lm_head are freshly quantized from their BF16/F32 tensors in the catplusplus checkpoint, using the standard compressed-tensors recipes.

Please credit the catplusplus checkpoint as the original model for this conversion.

• —Base architecture: MiniMax-M2 family
• —REAP expert pruning: inherited from catplusplus (256 → 192 experts per layer)
• —Format reference (on-disk layout template): `saricles/MiniMax-M2.5-REAP-172B-A10B-NVFP4-GB10`
• —Conversion tool: `scottgl9/minimax_reap_reduction/convert.py`

Expected accuracy impact

Compared to the source catplusplus M2.7 (already NVFP4 on experts):

• —Attention F32 → NVFP4 is the dominant term: ~1–2% regression on reasoning / code benchmarks (published NVFP4 W4A4 result class; the saricles M2.5 model went through the same transform on the same architecture and remained usable).
• —lm_head BF16 → FP8 per-tensor: ~0.2% regression on output-head-sensitive benchmarks.
• —Experts are rename-only — zero numeric delta.
• —Net expected: ~1–2.5% aggregate regression on most benchmarks, up to ~3% on rare-token / code tasks.

License

Inherits the upstream MiniMax-M2 license (MIT). See LICENSE in the repo.

Citation

If you use this checkpoint, please cite the underlying MiniMax-M2 release and the REAP pruning methodology by MJPansa.