Baekpica/MiMo-V2.6-Flash-RL-Mixed-Quant-GGUF
MiMo-V2.6-Flash-RL Mixed-Quant GGUF
Mixed weights with original-representation calibration. The four-shard language model passed its 508-tensor structural and quantization audit. Role-aware precision: expert gate/up IQ2_XXS, expert down IQ2_XS, shared dense paths and embedded MTP Q8_0, media BF16, and numerical controls F32. Measured GGUF storage: 93.092 GB / 86.699 GiB, including main model, multimodal projector and separate DFlash weights. Auxiliary files and runtime memory are additional. Text, image, audio, video and audiovisual assets are retained. ds4-dfm-rs on one DGX Spark GB10: the bounded serving checks below cover two 256K text banks, serial media, live partial reuse and restart disk KV with MTP off. Prior 512K serial text and 256K serial media/DFlash checks used different configurations. The checks do not establish throughput parity or general quality retention.
Serving checks, September 25, 2026
The 256K mixed setup uses --max-seqs 2 --prefix-reuse partial --kv-disk-dir ... --mtp-mode off, DS4_MIMO2_PREFILL_CHUNK=2048, and a VMM owner with --no-repack-q8-aligned. Aligned IQ2 expert weights stay enabled; disabling the optional Q8 repack saves 6.26 GiB. The two text banks hold separate KV and serialize forward passes through shared scratch. Images, video and audio use the serial lane.
These are bounded functional checks, not a full-length 256K mixed request, multimodal quality, cache reuse at 1M, or throughput qualification. The serving report records the exact limits. Embedded MTP without DFlash and the external five-layer DFlash with serial media were checked separately before this setup.
Runtime optimization, September 26, 2026
Measured on one DGX Spark / GB10 with MQ-IQ2-XXS-XS-Q8-MM-BF16, CUDA 13.3 (sm_121a), and the user-managed 300–2200 MHz clock range. These results belong to the runtime candidate in PR #65 (tested source).
Image, audio and video latency
Each round starts from its retained predecessor. The table shows three-run medians from fresh workers, each with one warm request before measurement. The rounds are separate paired comparisons; their gains are not cumulative measurements against one common baseline.
Media settings: BF16 projector, context 8192, native prefill chunk 4096, serial lane (continuous width 0), prefix reuse and MTP off, temperature 0, thinking disabled. Output caps are 64 tokens for images and 128 for audio/video. Audio stops naturally; image/video outputs reach their caps. Decode medians remain approximately 26–27 tok/s, with no consistent regression or claimed decode speedup. Busy clocks were 2184–2197 MHz in the final video/image A/B.
Full-vocabulary proofs compare all 152,576 prefill logits, generated tokens and prepared inputs against each round's controls. Image R2 has one differing OFF-control HTTP response out of 18; its model proofs and all original/ON responses match. Before the corrected audio A/B qualification, a test-first repair removed an existing LayerNorm shared-memory reuse race (racecheck: failure → zero hazards). Audio and final video/image comparisons use that repaired baseline; their 36 and 54 warm/measured HTTP responses respectively match exactly. These are bounded parity checks, not general caption or transcription scores. One audio fixture retains a baseline transcription error.
Score caching uses additional shared memory per block but no persistent tensor allocation. Coalesced vision reads are enabled only for 512–3072 attention rows: larger-shape pilots regressed and retain the preceding path. That optimization adds 8448 B shared memory per block and about 199% more executed warp instructions in the isolated 880-row NCU comparison, with unchanged floating-point operation counts; the paired latency gains justify this measured, shape-limited tradeoff. Media results, controls and limits.
Text prefill
Three retained changes improve 8K prefill consistently by less than 1% each. Each uses its preceding retained build as the original control, three fresh samples per arm, a separate warmup process, a resident VMM owner, promessi_sposi.txt (8192 input / 128 greedy output tokens), 4096-token chunks, and MTP/DFlash off.
These text results were qualified at retained text round 3 (a3fd2a5c); the final media build has no separate fresh text A/B. All measured full-vocabulary logits and 128-token continuations match. Decode ranges overlap; no decode improvement is claimed. The fusions reduce traffic without additional arithmetic or allocation. Gate/Up scheduling reduces spill instructions by 52.94% and L2 sectors by 18.77%, at the cost of 2.29% more executed instructions. Three further text candidates were rejected and are not enabled. Text results and receipts.
This campaign does not requalify the earlier long-context, two-bank or cache serving shapes. The 2K–64K curve below remains the September 24 measurement.
Prefill and decode on DGX Spark
The September 24 defaults (acc16308) reached 675.84 tok/s incremental prefill and 17.98 tok/s plain decode at 64K. Across the 2K–64K curve, the median of three run means is 874.46 / 21.47 tok/s.
One DGX Spark / GB10, MQ-IQ2-XXS-XS-Q8-MM-BF16, promessi_sposi.txt. Three fresh processes, one warm session per process, 2,048-token incremental steps and 128 greedy output tokens at every frontier. MTP and DFlash off. Curves show per-frontier medians; recorded bands show observed min–max; legend values are the 64K medians. The historical f09c1862 curve has only recorded medians, so it has no band. Historical curves are not a same-hour A/B. The clock range stays 300–2200 MHz; observed busy SM clocks were 2184–2190 MHz. CSV hashes and measurements · Runtime report.
Fresh paired runtime checks
Same final binary and resident weight owner, three fresh 8K/128 samples per side, ABBAAB order and a fresh warmup before every sample. The controls disable the new SWA tile, warp router and SwiGLU→Q8 fusion; the candidate uses defaults. This 8K prefill is distinct from the incremental curve.
The retained paths share the SWA window, parallelize expert selection, and emit unweighted SwiGLU directly into the down projection's Q8 input. Router and SwiGLU checks preserve logits/tokens exactly. SWA changes FP32 summation: one Italian continuation differs at 94/128 positions after token 31, while three focused math/code/Korean answers match. These are scoped numerical checks, not a general quality benchmark. Dense Q8 CTA grouping and QKV/KV fusion were evaluated and rejected; neither is enabled.
DFlash's separate GPU-attention round improves that mode from 4.41 to 6.97 tok/s on a 256/64 fixture (three samples, +58.05%). A same-binary plain-decode countercheck reaches 23.91 tok/s (one sample). DFlash remains slower on that workload; the main graph above uses plain decode.
Earlier experimental decode rounds on DGX Spark
Historical 8K pairs, 128 greedy tokens, one fresh process per arm. MTP and DFlash stay off. The base stack adds P6 SWA HMMA and the full-attention decode split. Busy clocks were 2190–2197 MHz. KV size and the 8K argmax matched on each pair.
One cold 64K run of that historical enabled stack measured 884.21 tok/s prefill and 17.10 tok/s decode, clocks 2184–2197 MHz. That single process does not replace the incremental median above.
Earlier prefill rounds on DGX Spark
Three same-binary comparisons at one 8,192-token frontier, 128 greedy tokens, promessi_sposi.txt, MTP and DFlash off. Three fresh processes per side after a warmup. Busy SM clocks stayed inside 2177–2197 MHz. That 8,192-token frontier is not the incremental curve above.
Unset, windowless and window-128 prefill of 32 or more rows use tensor cores, and the windowless path stages KV asynchronously. Those prefill changes leave one-row decode unchanged at that revision. Rounds 1 and 3 change summation order: relative RMS was 0.073 and 0.077 under --logit-rel-rms 0.10, and every frontier argmax matched. Round 2 matched logits and greedy tokens.
Support my work
I work on making large language models practical on constrained hardware through mixed quantization, inference optimization, and serving experiments. Contributions help cover calibration, GPU compute, storage, and testing so these results can be published openly.
<a href="https://www.buymeacoffee.com/baekpica" target="blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee" style="height: 60px !important;width: 217px !important;"></a> <a href="https://github.com/sponsors/Baekpica" target="blank"><img src="https://img.shields.io/badge/Sponsor-EA4AAA?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Sponsor Baekpica on GitHub" style="height: 60px !important;width: 217px !important;"></a>
This is a mixed-precision conversion of XiaomiMiMo/MiMo-V2.6-Flash-RL, pinned to revision `3b38d063180c3e4aed9691fdc735f3d10b266ee4`.
Why this model needs an asymmetric layout
The source stores routed expert weights as packed MXFP4. Two logical weights occupy each packed byte, so counting stored tensor elements can produce a roughly 159B figure. The expanded language trunk contains approximately 308.78B logical parameters; the root checkpoint including embedded MTP and media components contains approximately 310.76B, excluding the separate DFlash package. The packed representation does not make this a 159B logical model.
Routed experts account for 302.80B parameters. They dominate the storage budget, so the recipe concentrates compression there and gives their output projections a higher tier. Shared attention and dense paths, the output head, and multimodal components receive substantially more precision.
Variant and availability
Main plus media without the optional separate drafter: 91.526 GB / 85.240 GiB.
Download the complete variant, including template, DFlash configuration and provenance:
hf download Baekpica/MiMo-V2.6-Flash-RL-Mixed-Quant-GGUF \
--include 'MQ-IQ2-XXS-XS-Q8-MM-BF16/*' --local-dir ./mimo-mixed
cd ./mimo-mixed/MQ-IQ2-XXS-XS-Q8-MM-BF16
sha256sum -c SHA256SUMSThese are download and verification commands. See the serving report above for the bounded GB10 runtime configuration.
Quantization targets
The per-tensor inventory and recipe accompany the weights. BF16/F32 in this table describes the output storage format; source tensors already stored in FP8 are expanded from that source precision.
Importance matrix and calibration
Calibration uses the original checkpoint representation: expert MXFP4 values are repacked into GGUF without an additional quantization step, while the source FP8 dense tensors are expanded to BF16. This reference is not an original full-BF16 checkpoint and is not calibrated from the final IQ2 artifact.
The corpus starts from Baekpica/Inkling-Small-Multimodal-Calibration, with media recovered from the source datasets and prompts tokenized using MiMo's own tokenizer. Inkling token IDs and embeddings are not reused. Video clips are added from FineVideo.
The visual-only clips are paired with a separate set of 79 joint audiovisual inputs using restored original audio and production per-frame-pair interleaving. Every prepared audio token is consumed exactly once in each joint input. All 953 native media records completed trunk calibration with finite final logits. Coverage is 12,030 of 12,032 layer/expert pairs; block 7 experts 13 and 184 remain unobserved. The original recipe is unchanged: raw zero counts are retained and the pinned quantizer uses uniform importance weights for those experts. The final imatrix SHA256 is 265cc19bc1470b95a9157d3b2fab893f325a9e2b6ef414cdbdfac4326c9ef8e3.
Text calibration used raw GGUF Qwen2 BPE without the HF tokenizer NFC normalizer. Six decomposed Y-macron occurrences in four corpus lines differ from NFC input; the collected statistics are preserved as observed. Native media prompts used the original HF tokenizer. The ds4 MiMo tokenizer applies NFC.
Multimodal input contract
Input processing is being aligned with the SGLang MiMo implementation, alongside the pinned checkpoint configuration and tokenizer.
Generic video ingestion is insufficient here: using independent image frames or generic timestamps changes the model's input contract. Media imatrix collection must consume the correctly prepared original-model embeddings.
Memory metrics
Exact GGUF file sizes are listed above and in artifact-manifest.json. Main tensor payload is 88,771,782,144 bytes; file sizes additionally include headers, tokenizer metadata and alignment.
Device usage includes KV cache, media activations, allocator overhead, workspaces and the operating system. The qualified serving set is the same one named above. Selective residency beyond that measured process is not claimed.
Conversion and verification
The sampled expert audit is not a whole-file byte comparison. A successful reference smoke is not a quality benchmark for the final mixed model. These conversion checks do not establish accuracy or general quality retention. Scoped throughput measurements are reported above.
The original-representation reference and its audit are available in the separate intermediate GGUF repository.
Reproduction and release contents
The variant includes four mixed language shards, a BF16/F32 multimodal projector (including input audio-codec tensors), the separate Q8_0 DFlash package and mask embedding, original chat template, tensor recipe, final imatrix, calibration provenance and coverage, conversion scripts, checksums and validation reports.
See reproduction instructions. The synthesis decoder and training-only audio-codebook statistics are not part of this input-modality projector. No audio-output serving capability is claimed. Original gated source media is not redistributed.
The separate intermediate repository contains the calibration reference. No full Q8_0 or BF16 language baseline was needed.
Chat template
The upstream `chat_template.jinja` is published alongside this card, copied byte for byte from the pinned source revision. Use it with MiMo’s own tokenizer and special-token mapping. Media preprocessing and embedding insertion remain part of the runtime input contract; the template alone does not implement them.
Runtime support
The current work covers calibration, mixed-quant construction, validation, and the measured ds4-dfm-rs serving set named above. This card does not add a second serving recipe.
License
MIT, inherited from the pinned upstream model. Calibration datasets retain their respective licenses and access conditions.