CoolFace
Modelpublic

itrejomx/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTPLX-6bit

sourceHugging Faceapache-2.0updated 27d agoView on Hugging Face
1likes1.6kdownloads
Model Card

Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTPLX-6bit

MLX 6-bit build of HauhauCS's Qwen3.8-27B Uncensored Aggressive fine-tune with a working multi-token-prediction (MTP) head, forged and verified with MTPLX (contract=true).

As far as I know this is the first MLX conversion of this fine-tune — the author publishes GGUF only.

Measured speed (M-series Mac, 48 GB)

Modedecode tok/svs AR
Autoregressive10.91.00x
MTP depth 123.92.20x
MTP depth 230.92.84x
MTP depth 325.82.38x

MTPLX auto-tunes the depth per machine; on the build machine depth 2 was selected. For comparison, the same fine-tune as Q6 GGUF with the FastMTP sidecar in llama.cpp reached ~12.5 tok/s on the same hardware.

Usage

bash
mtplx pull <this-repo>
mtplx tune --model <this-repo> --retune   # calibrate MTP depth for your machine

Provenance and deliberate changes

This is a community conversion, not an official HauhauCS release (their releases carry Ed25519-signed manifests; this one does not).

  • —Quantization lineage: original bf16 → author's Q8KP GGUF → dequantized f16 → single MLX 6-bit affine quantization (group size 64). The MTP head is kept in bf16 (mtp_policy: keep_bf16) and was verified against the trunk by MTPLX Forge.
  • —Conversion pipeline: llama.cpp's qwen35 GGUF applies a tiled V-head reorder to the linear-attention tensors and stores A_log as -exp(A_log); both transforms were inverted to restore the HF/MLX layout before forging.
  • —Chat template: thinking is disabled by default (the original template defaults to enabled). Pass enable_thinking: true to turn it back on. This avoids greedy-decoding degeneration loops and matches how most people run this model for direct tasks.
  • —Tokenizer: rebuilt from the family reference tokenizer to restore <think>/</think> special-token registration, which the GGUF metadata round-trip loses. Text encoding is byte-identical to the original.

Credits