hotdogs/Qwen3.8-27B-abliterated-code-analysis-preview-mtp-GGUF

Qwen3.8-27B Code Analysis Preview (v2) — MTP GGUF

GGUF quantizations of the code-analysis fine-tune `hotdogs/Qwen3.8-27B-abliterated-code-analysis-preview`, with Multi-Token-Prediction (MTP) tensors preserved for speculative decoding.

Given a code snippet, it returns a structured, multi-paragraph review — real bugs, line-level reasoning, severity, and a concrete fix in a code block. It is a reasoning model: it thinks first, then answers.

v2 fixes the template-collapse of v1. v1 was trained on a synthetic placeholder dataset and answered in one line ("No bugs found. Code is clean."). v2 is retrained on 21,009 real code+bug+answer rows across 5 languages (Python, JS, Go, Rust, C) with detailed 550–880 char answers — the model now actually finds the bugs and generalizes to unseen bug types.

Files

All 3 files: 866 tensors, MTP preserved — 15 blk.64.* tensors (11 transformer-layer + 4 blk.64.nextn.*).

MTP / speculative decoding

The MTP head lives in block 64 (blk.64.nextn.*). With llama.cpp you can use it as a draft model for speculative decoding:

llama-server -m Qwen3.8-27B-code-analysis-preview-v2-mtp-Q6_K.gguf \
  --n-gpu-layers 999 --ctx-size 262144 --parallel 1 \
  --cache-type-k f16 --cache-type-v f16 --flash-attn on \
  --temp 1 --top-k 20 --top-p 0.95 --min-p 0.0 --jinja --tools all

(The blk.64.nextn.* tensors load automatically; enable speculative decoding via the predictor/draft options of your llama.cpp build.)

Smoke test (v2)

Measured on Q6_K (5×3090 / 2 GPUs, flash-attn): ~34 t/s generation, ~141 t/s prompt eval.

Recommended

• —Q6_K — best quality/size balance (21 GB) — recommended
• —Q4_K_M — fastest / smallest (16 GB), great for consumer GPUs
• —F16 — max fidelity (51 GB)

License

MIT (inherits the abliterated base).