robinshao/hauhau_qwen36_35b_a3b_mxfp4_gguf_mtp

hauhauqwen3635ba3bmxfp4ggufmtp

This repo contains a local MXFP4_MOE GGUF export of HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive together with a separate MTP draft GGUF for speculative decoding in a patched local llama.cpp toolchain.

Files

• —hauhau_qwen36_35b_a3b_mxfp4_robinstyle.gguf: main text model in robinshao-style MXFP4 layout
• —hauhau_qwen36_35b_a3b_mxfp4_mtp.gguf: MTP draft GGUF used with --spec-type draft-mtp
• —llama.cpp.src-patched.zip: patched local llama.cpp source snapshot used for conversion and serving
• —build_result.json: build and validation summary for the main model and MTP draft
• —mtp-benchmark.json: short local speed comparison between baseline and MTP serving

Quantization Notes

• —Source FP16 weights: khoi11/Qwen3.6-35B-A3B-Uncensored-HauhauCS-FP16
• —Upstream model lineage: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
• —Main model converted for text inference and quantized into a robinshao-style MXFP4_MOE GGUF
• —MTP draft exported as a separate GGUF for speculative decoding with patched llama.cpp
• —Vision tensors are not included in the final serving package

Local Deployment Notes

This pair was validated locally on Windows with patched llama.cpp using speculative decoding:

llama-server -m hauhau_qwen36_35b_a3b_mxfp4_robinstyle.gguf \
  --spec-type draft-mtp \
  --spec-draft-model hauhau_qwen36_35b_a3b_mxfp4_mtp.gguf \
  --spec-draft-device CUDA0 \
  --spec-draft-ngl all \
  --jinja \
  --device CUDA0 \
  --gpu-layers all \
  --ctx-size 8192 \
  --batch-size 768 \
  --ubatch-size 128 \
  --flash-attn on \
  --cache-type-k f16 \
  --cache-type-v f16 \
  --cache-ram 0

In a short local API benchmark on an RTX 4000 Ada, the MTP setup improved decode speed from about 103.45 tok/s to about 131.65 tok/s, roughly 1.27x.

Caveats

• —This is a text-only export, not the original multimodal package.
• —The draft file is intended for speculative decoding and is not a standalone replacement for the main model.
• —Native training context metadata remains 262144, but the actual no-offload local limit depends on GPU VRAM, KV cache settings, and whether the draft model is loaded.