CoolFace
Modelpublic

apetersson/DeepSeek-V4-Flash-0731-Abliterated-MLX-Mixed-2bit-3bit-g64

sourceHugging Facemitupdated 2mo agoView on Hugging Face
1likes461downloads
Model Card

DeepSeek-V4-Flash-0731 Abliterated MLX Mixed 2-bit/3-bit g64

Preliminary experimental release. The full paired refusal and capability evaluation is still in progress.

This is a mixed-precision MLX quantization of `apetersson/DeepSeek-V4-Flash-0731-Abliterated-FP8`, designed to run on a 128 GB Apple Silicon system. Weight files occupy about 99.46 GiB.

Quantization profile

Tensor classStorage
Routed expert gate/upaffine 2-bit, group size 64, importance weighted
Routed expert downaffine 3-bit, group size 64, importance weighted
Attention projectionsMXFP8, group size 32
Shared expertsMXFP8, group size 32
Output headMXFP8, group size 32
Embeddings, routers, norms and auxiliary tensorsprotected BF16/FP32

The importance matrix was imported from `jedisct1/DeepSeek-V4-Flash-imatrix-aligned`. The 129 routed modules have architecture-compatible dimensions, but this is an aligned transfer from the earlier DeepSeek-V4-Flash checkpoint, not a fresh 0731-native calibration run.

This variant omits the attached speculative weights and sets num_nextn_predict_layers=0. It is intended for standard trunk generation; it is not a DSpark release. The preferred local runtime is `oMLX` with DeepSeek-V4 support.

Validation status

  • —All 21 shards and 1,931 indexed tensors passed structural validation.
  • —The imported importance matrix was applied to all 129 routed modules.
  • —The full paired refusal and capability benchmark is pending.

Limitations

This is an aggressive mixed 2/3-bit quantization. Quality should be measured against both the abliterated FP8 checkpoint and the clean quantized control. Abliteration can affect behavior beyond refusals and does not guarantee a particular response.

License and attribution

The upstream DeepSeek-V4-Flash-0731 weights are MIT licensed. This repository retains that license and documents both the abliteration and quantization provenance above.