siendsi/DeepSeek-V4-Flash-144t2-UD-IQ1_S

DeepSeek-V4-Flash-144t2 (UD-IQ1_S, per-layer expert pruning)

Hugging Face: `siendsi/DeepSeek-V4-Flash-144t2-UD-IQ1_S`

A 43% expert-pruned build of DeepSeek-V4-Flash-0731 (UD-IQ1_S), reduced from 82.4 GB → 47.0 GB (−43.1%) by keeping the 144 most-activated experts per layer (out of 256) and disabling the remaining 112.

This is a GGUF model for llama.cpp. It is a per-layer MoE expert pruning of the original — not a low-rank distillation and not a weight re-quantization. All non-expert weights and the kept experts are bit-identical to the original UD-IQ1_S checkpoint.

Heads-up: this is the deepest build in the family and sits at the edge of acceptable quality. It is the smallest / cheapest option, but two harder reasoning tasks degrade. If you have the space, the keep-160 build (51 GB, 24/26) is the recommended sweet spot.

What was done

• —Method: per-layer expert pruning. For each of the 43 MoE layers, the 144 experts with the highest activation counts (from a calibration "heat" map) are kept; the 112 least-used experts are disabled.
• —Calibration heat: tuned to a target profile — Russian + English + coding + planning + agentic tasks, plus a math corpus (math is critical for agentic/reasoning tasks). The original calibration corpus (which included Japanese/Chinese) was not used, because it pruned experts that matter for the target profile.
• —Quantization: unchanged — UD-IQ1_S (Unsloth dynamic quant).
• —Architecture: deepseek4 (llama.cpp).

Why per-layer? DeepSeek-V4 expert IDs are per-layer — expert N in layer 0 is a different expert than expert N in layer 3 (0/256 shared). A global top-K expert map is therefore meaningless; pruning must be done independently per layer.

Size

Quality (A/B vs. original)

Evaluated against the original UD-IQ1_S on a 26-task target-profile battery (code / devops / multi-hop / OS / NLP / agent):

• —keep-144: 23/26 parity. Two degradations, both on long, self-correcting reasoning / agentic tasks:
• —multi_hop_math — does not converge on a multi-step train problem (shared with keep-160).
• —multi_tool — does not converge on an agentic multi-tool task (leap-year + sum) that does work on keep-160.

Practical takeaway: for short, deterministic answers (knowledge, math, code snippets, NLP, OS, devops) this model is at parity with the original. For tasks that require long, self-correcting reasoning or multi-step agentic planning, keep-144 is more likely to fail to converge than keep-160 (it does not give wrong answers — it simply does not finish). This is why keep-144 is the "at the edge" option rather than the recommended one.

Speed (llama.cpp, -t 8)

Measured on keep-192 (same build pipeline, 60 GB); keep-144 is expected to be similar or slightly better on prefill (fewer expert weights to load):

Prefill is noticeably faster on the pruned model; generation is roughly at parity (the number of active top-k experts is the same, so the difference in weight loading is minimal).

Usage (llama.cpp)

llama-server \
  -m DeepSeek-V4-Flash-0731-UD-IQ1_S-00001-of-00003.gguf \
  -c 32768 -t 8 -fa on --load-mode mmap \
  --port 8088

Or with the llama CLI:

llama-cli -m DeepSeek-V4-Flash-0731-UD-IQ1_S-00001-of-00003.gguf \
  -c 32768 -t 8 -fa on --load-mode mmap

The model is multi-shard. Point llama.cpp at shard 00001; it will pick up the other shards automatically.

Files

Provenance

• —Base: DeepSeek-V4-Flash-0731 (UD-IQ1_S, Unsloth dynamic quant)
• —License: MIT (inherited from the base model)
• —Pruning: per-layer top-144/256 experts, target-profile + math heat
• —Built with: llama.cpp rebuild_gguf.py (per-layer prune config)
• —Reproduction: see the glm-pruning project — DSV4_PRUNING_FINAL.md, tools/build_dsv4_prune_config_perlayer.py, tools/heat_target.py

Notes

• —This is a pruned model, not a fine-tune. Behavior on the target profile is at parity with the original for short deterministic tasks; out-of-profile (e.g. Japanese/Chinese) is not the goal and may be degraded.
• —Very long contexts (>32k) were not stress-tested.
• —Recommended alternatives in the same family:
• —`siendsi/DeepSeek-V4-Flash-160t2-UD-IQ1_S` (keep-160, 51 GB, recommended optimum, 24/26)
• —`siendsi/DeepSeek-V4-Flash-192t2-UD-IQ1_S` (keep-192, 60 GB, safest margin)