BoldingBuilds/Ternary-Bonsai-2-27B-Abliterated-PQ2_0-MTP-GGUF

Ternary Bonsai 2 27B — Abliterated, PQ2_0, with optional MTP

v2 is out: Ternary-Bonsai-2-27B-Abliterated-v2-PQ2_0-MTP-GGUF. It fixes the thinking-mode problem described below: blank answers on harmful prompts drop from 27% to 15% at default settings and to 0% with reasoning_effort=medium. This repo stays up unchanged as v1.

Refusal behaviour removed from prism-ml/Ternary-Bonsai-2-27B-gguf at PQ2_0 (2.13 bpw), by flipping a small number of ternary digits in place. Block scales are never touched and the model is never requantised, so this is the released ternary packing with a few digits changed — not a re-derived one.

A second file adds the Qwen3.8-27B multi-token-prediction head, which makes --spec-type draft-mtp work and decodes ~37% faster at identical weights.

As far as I can tell this is the first abliterated Bonsai 2 that can do speculative decoding.

Also available: [PTQ1_0 (1.75 bpw)](https://huggingface.co/BoldingBuilds/Ternary-Bonsai-2-27B-Abliterated-PTQ1_0-GGUF) — smaller pack, same abliteration method, 1,680 downloads and the original release. Take it if size matters more than speed; MTP does not help that pack, for the reason given further down.

Files

Both contain identical language-model weights. The MTP file adds 15 tensors (blk.64.*). With speculation off, the two files generate byte-identical text — verified, 5/5 prompts over 400 tokens each. If you don't want to patch anything, take the first file and lose nothing but the speed.

What was changed

The 98 are the residual-writing tensors — ffn_down, ssm_out, attn_output — across blocks 15–63.

Refusal

Measured here, both suites, one judge model, same settings for every row. Harmful-prompt refusal is meaningless without the over-refusal number beside it, because a broken model also scores 0% refusal — so both are given.

Thinking off, greedy decoding, 2,048-token answers. Judged by Qwen3.8-27B-OBLITERATED-Q8_0 with the StrongReject rubric, not prefix matching. Prefix graders overstate baseline refusal and inflate everyone's apparent effect size.

Capability

Full MMLU test split, all 14,042 questions, 0-shot, next-token letter logit. Arms answer the same questions in the same order, so they're paired.

Our edit costs nothing measurable on MMLU: +0.15 pp on PQ20 (exact McNemar p = 0.40) and +0.12 pp on PTQ10 (p = 0.51, 577 discordant pairs).

HumanEval-164 (greedy pass@1, paired per problem): stock PQ2_0 0.890, this model 0.872 (−1.8 pp, 3 gained, 6 lost, p = 0.51). Not significant.

Ternary compression itself costs about 5.2 pp against the non-ternary parent (p = 2.6e-71) — worth knowing, and not something the abliteration causes.

Thinking mode: leave it off for now

This build has a known problem with thinking on. On safe prompts it reasons about as long as stock. On harmful prompts it often drafts its answer inside the reasoning and never finishes: with a 4,096-token budget and the default sampler settings (temperature 1.0, top-k 20, top-p 0.95), 27% of harmful prompts got no answer at all (stock: 0%), and 20% of safe prompts got no answer (stock: 6%). Greedy decoding makes it worse: the reasoning falls into repeated sentences and 65% of harmful prompts get no answer. Every number above was measured with thinking off. A fix is being tested.

The MTP file: ~37% faster decode

It needs a patched llama.cpp

--spec-type draft-mtp fails at startup on stock binaries:

Hadamard-latent table 'token_embd.weight' is read without the inverse transform

This is not a defect in the graft. PrismML's ternary models store token_embd.weight rotated, and every read must undo that first. The main model path does; the MTP draft graph builds its own embedding lookup and skips it. 0001-qwen35-mtp-hadamard-inverse.patch in this repo fixes it in 14 lines against tag prism-b10687-5d80cff:

git clone https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
git checkout prism-b10687-5d80cff
git apply ../0001-qwen35-mtp-hadamard-inverse.patch
cmake -B build -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=<your arch> -DLLAMA_CURL=OFF
cmake --build build --target llama-server -j

The same bug was found and fixed independently by decent-jawfish and ProCreations — credit to them; see below. Their patch targets an earlier tag.

Run

llama-server -m Ternary-Bonsai-2-27B-Abliterated-PQ2_0-MTP.gguf \
  -ngl 99 -fa on -c 32768 --jinja \
  --spec-type draft-mtp --spec-draft-n-max 2

Measured

One RTX 3090, 5 prompts, 400 tokens each, --parallel 1, greedy, median.

2 was best; sweep it on your own card.

The gain depends entirely on what you generate

Baseline is 68.8. On free prose, speculation was slower than no speculation — you pay for drafts that get rejected. If your workload is creative writing, this file will not help you and may cost you a few percent.

Don't bother doing this to PTQ1_0

I grafted the same head onto the 1.75 bpw PTQ10 build and measured **+1.6%** — nothing. Acceptance was fine (0.648, essentially the same as PQ20), so the head drafts just as well; the verification is what doesn't pay. PTQ10 also decodes *slower* than PQ20 despite being 1.26 GB smaller (57.8 vs 68.8 tok/s), which suggests its unpacking is compute-bound, so verifying a batch of 3 tokens costs about 3x verifying 1. Published here so nobody else spends a day finding out.

On losslessness

Speculative decoding verifies every draft against the target model, so the output distribution is unchanged in principle. In practice, batched verification changes the floating-point reduction order, which can flip the argmax at a near-tie. Measured:

• —the same configuration run twice is 5/5 byte-identical, acceptance equal to 4 decimals
• —spec vs no-spec diverges on 2 of 5 prompts, at the same character offsets across independent runs, always at near-synonyms (seeped/crept, ascending/non-decreasing)

A verification bug could not be reproducible to the character. So: deterministic within a configuration, and differing from the non-speculative path only at close token decisions.

What this does not claim

• —Not that quality is unchanged in general. The claim is narrower: the base tensors are byte-identical to PrismML's release except in the 98 listed, and MTP is target-verified.
• —No vision evaluation. The mmproj is not included; download PrismML's matching projector.
• —Tested on CUDA sm_86 only. Metal and CPU paths are untested.

Safety

This model has had its refusal behaviour removed and will comply with requests a stock model declines. It is published for research on alignment robustness and for users who need an unfiltered local model. You are responsible for what you do with it. Do not deploy it in a user-facing product without your own safety layer.

Credits

• —Base model: `prism-ml/Ternary-Bonsai-2-27B-gguf` (Apache-2.0)
• —MTP head: copied verbatim and unrotated from `unsloth/Qwen3.8-27B-GGUF`, quantised from `Qwen/Qwen3.8-27B` (Apache-2.0)
• —Runtime: PrismML-Eng/llama.cpp (MIT)
• —The MTP recipe is not mine. That the head ships inside the parent GGUF, that it can be grafted unrotated, and that the draft graph needs the Hadamard inverse were worked out by decent-jawfish and ProCreations, building on sudoingX/qwen38-mtp. I re-derived the patch against a newer tag and applied the graft to an abliterated trunk; the idea is theirs. ProCreations additionally fine-tuned the head against Bonsai's own hidden states, which is worth more than the raw graft and is not done here.

Not affiliated with or endorsed by PrismML, Qwen, or Unsloth.

Apache-2.0, as are all components.