ji-farthing/MiMo-V2.5-Pro-DFlash-draft-ik-llama-GGUF

MiMo-V2.5-Pro DFlash Draft: BF16 GGUF for ik_llama.cpp

A GGUF conversion of the DFlash block-diffusion draft for MiMo-V2.5-Pro, built for speculative decoding in ik_llama.cpp. This is the validation artifact for ik_llama.cpp PR #2048.

Important: This is not a standalone model. It is a 5-layer draft that shares the target's token embedding and output head. It only produces meaningful output when launched as --model-draft alongside a full MiMo-V2.5-Pro target GGUF.

Files

Matching target

Pair this draft with a MiMo-V2.5-Pro target GGUF. Validated against:

• —`gghfez/MiMo-V2.5-Pro-ik_llama-unfused-GGUF`, file MiMo-V2.5-Pro-IQ3_S-unfused.gguf (IQ3_S, about 349 GiB)

The validation used an unfused target. One fused-MoE conversion tested during the PR produced garbled target-only output, so it was excluded from the DFlash results. That observation does not establish that every fused MiMo conversion is incompatible.

Usage (ik_llama.cpp)

llama-server \
  -m   MiMo-V2.5-Pro-IQ3_S-unfused.gguf \
  --model-draft mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf \
  -ngl 999 -ngld 999 \
  -sm graph --max-gpu 4 \
  -fa on -ctk q4_0 -ctv q4_0 -ctkd q4_0 -ctvd q4_0 \
  --spec-type dflash:n_max=1,p_min=0.0,cross_ctx=16 \
  -c 8192 --host 127.0.0.1 --port 8080

• —--spec-type dflash:... selects the DFlash speculator. n_max=1 was optimal in this test. cross_ctx=16 is a small ring buffer of recent target positions visible to the draft; larger values reduced acceptance sharply.
• —-sm graph --max-gpu 4 was the best multi-GPU profile tested on 4×H200 NVL. Pinning the draft to one GPU with -devd added overhead under graph split.

Conversion metadata

The file contains 63 tensors: five 12-tensor draft blocks plus dflash_fc, dflash_hidden_norm, and output_norm. It includes five blk.N.attn_sinks.weight tensors and does not contain token_embd or output tensors.

Validation

Validation was run remotely on 4×H200 NVL for ikllama.cpp PR #2048. The draft was tested greedily at `nmax=1, crossctx=16`, with q40 KV cache on both target and draft.

Conversion notes

• —The source weights are the BF16 drafter in the dflash/ subdirectory of `XiaomiMiMo/MiMo-V2.5-Pro-FP4-DFlash`.
• —Target metadata and the tokenizer are read from the matching MiMo-V2.5-Pro-FP4-DFlash repository root with --target-model-dir. The draft carries no embeddings of its own.
• —Convert with convert_hf_to_gguf.py from ik_llama.cpp PR #2048, using the DFlashDraftModel path and --outtype bf16.

cd MiMo-V2.5-Pro-FP4-DFlash

python3 ../ik_llama.cpp/convert_hf_to_gguf.py dflash \
  --target-model-dir . \
  --outtype bf16 \
  --outfile mimo-v25-pro-dflash-draft-bf16-rope5m-vscale-rope64.gguf

Use ikllama.cpp at PR #2048 or later. The final PR fix stores the draft's 10,000 RoPE base separately from the 5,000,000 backbone base. Older converter outputs must be reconverted rather than repaired by changing only `rope.freqbase`.

Troubleshooting low acceptance

1. 1.Confirm that the target produces coherent output by itself. A broken target conversion makes DFlash acceptance meaningless.
2. 2.Run an ik_llama.cpp build that includes PR #2048. The change includes runtime handling for attention sinks, value scaling, and MiMo2 target-feature capture in addition to GGUF metadata support.
3. 3.Start with dflash:n_max=1,p_min=0.0,cross_ctx=16 and a greedy code-completion prompt comparable to the validation fixtures. The reported 55–60% rates are fixture-specific, not a universal acceptance baseline.
4. 4.Test the target with this published draft. If this file also gives very low acceptance, inspect the target conversion and runtime setup. If this file works, compare the converted draft against the metadata and 63-tensor contract above.