Jundot/DeepSeek-V4.1-Flash-oQ3e-mtp

DeepSeek-V4.1-Flash-oQ3e-mtp

This model was quantized using oQ mixed-precision quantization.

A mixed-precision MLX checkpoint of DeepSeek-V4.1-Flash, including the language model, Engram tables, vision encoder, multimodal aligner, and DSpark MTP weights.

The effective average is 3.629 bits/weight excluding Engram, or 3.724 bits/weight for the complete checkpoint.

Quantization and Bit Distribution

This checkpoint uses mixed precision rather than a uniform 3-bit format:

• —3-bit affine quantization, group size 64, for language-backbone routed expert weights.
• —4-, 6-, and 8-bit affine quantization, group size 64, for calibrated attention and shared expert projections.
• —MXFP4 for preserved DSpark MTP routed expert weights, and MXFP8 for selected preserved projections.
• —3-bit affine quantization, group size 32, for Engram tables, using a separate quantization budget.
• —BF16 or FP32 for the remaining tensors.
• —BF16 for the vision encoder and multimodal aligner.

The oQ3e allocation uses measured layer sensitivity and importance-matrix calibration. The quantization report records a target of 3.5 bits/weight and a hard cap of 3.7 bits/weight, excluding Engram. The resulting effective average is 3.629 bits/weight. Uncalibrated modules, including DSpark MTP modules, retain their source precision.

Parameter counts below represent logical weight elements, excluding quantization scales and biases. Storage sizes and effective bits per weight include those metadata tensors.

Storage Breakdown

The checkpoint contains 65 safetensors shards and 2,284 tensors. Engram tensors are included in the same model.safetensors.index.json; no separate Engram directory is required.

Including configuration, tokenizer, license, and quantization-report files, the analyzed local model directory contains approximately 355.304 GB / 330.902 GiB of file data, excluding this README.

Memory Requirements with SSD Engram Offload

With Engram SSD offload enabled, the Engram tables are kept on SSD, while the language backbone, vision components, and MTP weights are budgeted as resident weights.

The offload approach reads the required Engram rows from memory-mapped files and reconstructs selected rows for computation. Quantized linear weights remain packed for inference rather than being expanded wholesale to BF16.

Weight Residency

Offloading avoids permanently materializing approximately 91.56 GiB of Engram tensors, equivalent to 27.67% of the checkpoint's tensor payload.

The planning estimate applies 5% headroom and 32 MiB to the resident weight size. It is a calculated estimate, not a measured peak or a memory ceiling. Runtime usage also depends on context length, concurrency, KV cache, MTP settings, temporary buffers, and file-backed memory residency.

Observed Runtime Memory

Runtime memory has not been measured for this checkpoint in this analysis. MLX active memory, peak memory, allocator cache, and process RSS require a separate inference benchmark; the weight-residency figures above are calculated from checkpoint metadata.

Measurement Notes

Storage and parameter counts were calculated from the local safetensors headers and quantization settings in config.json, and checked against quantization_report.json and model.safetensors.index.json. GB uses decimal units; GiB uses binary units. Rounded table entries may not sum exactly to the displayed totals.

The local metadata does not record the oMLX package version used for quantization.