Modelpublic

Thump604/DeepSeek-V4-Flash-MLX-Q3-mixed-gs128-affine

sourceHugging Facemitupdated 5mo agoView on Hugging Face

5likes312downloads

Model Card

DeepSeek V4 Flash MLX Q3 Mixed

This is an MLX conversion of deepseek-ai/DeepSeek-V4-Flash.

Source

—Base model: deepseek-ai/DeepSeek-V4-Flash
—Source revision: 6e763230a9d263eca2023f1d4a5ce1bfe126cf48
—Architecture: DeepseekV4ForCausalLM
—Model type: deepseek_v4

Conversion Recipe

—Tooling branch: Thump604/mlx-lm, branch deepseek-v4-support-fixes
—Minimum tooling commit for generation: 9c990f4
—Output path during conversion: /Volumes/Lexar/mlx_models/DeepSeek-V4-Flash-MLX-Q3-mixed-gs128-affine
—Quantization recipe: mixed_3_6
—Quantization mode: affine
—Group size: 128
—Effective bits per weight reported by MLX: 3.808
—Shards: 28
—Indexed MLX tensor size: 135,346,422,876 bytes

The mixed recipe uses 3-bit affine quantization for lower-risk routed expert paths and 6-bit affine quantization for sensitive paths including embeddings, LM head, attention projections, compressed-attention/indexer components, shared experts, and selected down projections.

Validation

—Conversion completed successfully.
—Lazy MLX load completed successfully on a 128GB Mac Studio.
—One-token generation smoke was attempted and stopped after memory pressure and swap activity exceeded the local safety boundary. Treat this artifact as converted and load-validated, not generation-qualified on 128GB Apple Silicon.

Notes

DeepSeek V4 support in MLX is still under active development. This artifact was produced with local DeepSeek V4 support fixes, including FP4/FP8 checkpoint handling, F8_E8M0 scale metadata reinterpretation as raw uint8 exponent bytes before sanitizer decode, attention sink dtype handling, and quantized grouped output projection support.