datayoda/DeepSeek-V4-Flash-0731-DS4-Antirez-Coder-Halo-128K-MixQ-GGUF

DeepSeek-V4-Flash-0731 DS4 Antirez Coder Halo 128K MixQ

A DwarfStar-specific mixed-precision GGUF of DeepSeek-V4-Flash-0731, optimized through a measured 12-round quantization search for coding quality on a 128 GB AMD Strix Halo system.

Results at a glance

<p align="center">


</p>

What makes this release different? This is not a fine-tune. It is the best result from a controlled search over which routed-expert layers to preserve at higher precision. Promoting only blk.0 and blk.1 to Q4_K improved the frozen coding-suite result from 21/50 to 26/50. Adding more promoted layers did not reliably help.

Why I built this

I wanted a practical local coding model that combines four things:

1. 1.DeepSeek-V4-Flash-0731.
2. 2.Antirez's focused DwarfStar runtime.
3. 3.A real 128 GB Strix Halo deployment target.
4. 4.Verified 128K context rather than a context-length claim copied from metadata.

The project became an experiment in a narrower question:

Can selective precision allocation improve coding quality while keeping this unusually large model usable on a 128 GB unified-memory machine?

For this evaluation, the answer was yes, but only with a very specific recipe.

The optimization story

The reference Q2 layout already makes a very large routed-MoE model practical on this hardware. Instead of broadly increasing precision, I tested targeted promotions and evaluated every candidate against the same frozen 50-task public coding suite with deterministic decoding.

Best recipe

Only the routed-expert gate, up, and down tensors in the first two blocks were promoted:

blk.0 → Q4_K
blk.1 → Q4_K

Everything else retains the reference layout:

• —Routed-expert down projections: Q2_K, except blk.0 and blk.1.
• —Routed-expert gate/up projections: IQ2_XXS, except blk.0 and blk.1.
• —Promoted tensors in blk.0 and blk.1: Q4_K.
• —Attention projections: Q8_0.
• —Shared experts: Q8_0.
• —Output head: Q8_0.
• —Importance matrix: the published reference imatrix, not a coding-specific imatrix.

What the search found

Two conclusions emerged:

• —Layer position mattered more than the number of promoted layers.
• —More precision was not automatically better. The simplest recipe that reached the best measured score was blk.0 and blk.1 at Q4_K.

128K context was actually tested

The model was launched with a 131,072-token context window and accepted a 130,029-token prompt without silent truncation. It then generated 512 output tokens successfully.

This establishes that the release was loadable and stable near the advertised limit in the tested configuration. It does not by itself prove equal reasoning quality across the full 128K window.

Hardware fit

The final GGUF is 90,343,990,176 bytes, or 84.15 GiB.

During the documented evaluation runs, the model used about 90 GiB of resident model memory, with additional KV/context overhead. The system retained approximately 14–17 GiB of host-memory headroom on the 128 GB Strix Halo machine used for testing.

128 GB unified memory
├── ~90 GiB resident model weights
├── runtime and context overhead
└── ~14–17 GiB observed host-memory headroom

Actual memory use depends on context length, runtime revision, active applications, and launch configuration.

Model details

Run with DwarfStar

This file was built specifically for antirez/ds4. It is not guaranteed to load in mainline llama.cpp.

ds4-server --rocm \
  --model DeepSeek-V4-Flash-0731-DS4-Antirez-Coder-Halo-128K-MixQ.gguf \
  --ctx 131072 \
  --port 8080

For initial validation, consider starting with a shorter context before moving to the full 131,072-token configuration.

What was measured

Public coding suite

• —50 frozen tasks.
• —Deterministic decoding.
• —Temperature 0.
• —This release: 26 passed, 24 failed.
• —Unmodified baseline: 21 passed, 29 failed.

The measured result is specific to this task set and runtime configuration. It should not be interpreted as proof of a general capability improvement across all coding workloads.

Private confirmation suite

A separate five-language, 15-task confirmation suite was run once. Both the selected candidate and baseline scored 0/15, creating a floor effect. It was not useful for differentiating the two builds.

What failed and what I learned

The unsuccessful experiments were as informative as the winning one:

• —Promoting three consecutive early blocks regressed to 16/50.
• —Promoting a third nonconsecutive block tied the best score but added complexity.
• —Promoting four blocks did not beat the two-block recipe.
• —Routed-expert Q8_0 was rejected by the tested ROCm FFN kernel.
• —Additional non-routed F16 promotion attempts were rejected at load time.

The main lesson is simple:

For this model, runtime, quantization family, and coding suite, precision placement mattered more than precision volume.

Known limitations

This release has not established:

• —Generalization beyond the documented coding suites.
• —Improved long-running agent behavior.
• —Equal quality throughout the entire 128K context window.
• —Non-English capability changes.
• —Non-code capability changes.
• —Compatibility with runtimes other than the documented DwarfStar build.
• —The effect of a coding-specific importance matrix.

The release should be treated as a measured quantization result, not a universal claim that two promoted layers improve every workload.

Reproducibility and provenance

• —Base model: deepseek-ai/DeepSeek-V4-Flash-0731.
• —Base revision: 7872f01b1d1fe23eabc4c98b48bffcef5a386062.
• —Quantizer/runtime: antirez/ds4.
• —Runtime commit: 9ab705347c1775e7599ede7eb81a6255ec7dccb5.
• —Selected recipe: blk.0 and blk.1 routed-expert gate/up/down tensors promoted to Q4_K.
• —Importance matrix: published reference imatrix.
• —Final file SHA-256: 2d9c653eacbf4f730d0b2ef0d75e56307b0832659b91b54d7a7d07e901799d4b.

The uploaded file was rebuilt from the same recipe, imatrix content, template, and toolchain commit after the original archived copy was reclaimed for disk space. Its overall hash differs from the earlier archived artifact because an embedded imatrix-file-path metadata string changed. The imatrix content itself was verified as byte-identical, and the tensor recipe is unchanged.

Suggested future work

Potential next experiments include:

• —Repeating the evaluation with a larger public coding suite.
• —Adding repository-scale and repair-loop evaluations.
• —Measuring useful retrieval and reasoning quality at multiple context bands.
• —Building a coding-specific importance matrix.
• —Comparing additional supported mixed-precision layouts.
• —Testing long-running DwarfStar agent workflows.

Acknowledgments

• —The DeepSeek team for the base model.
• —Antirez and contributors for the DwarfStar runtime and GGUF tooling.
• —The llama.cpp and GGML communities for the broader quantization and GGUF ecosystem.

If you test this release on other workloads or hardware, community results are welcome, especially when they include the runtime commit, command line, context length, memory use, and reproducible task details.