Modelpublic

Hikari07jp/Ternary-Bonsai-27B-Abliterated-LowDeg-GGUF

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

44likes14kdownloads

Model Card

Ternary Bonsai 27B Abliterated-LowDeg (Q2_0 GGUF)

Low-degradation direct GGUF abliteration of Prism Ternary Bonsai 27B.

—Format: official Q2_0 g128 ternary pack (~7.2 GB)
—No runtime steering / no BF16 bake / no requant
—Built for the PrismML llama.cpp fork (hybrid-attention Q2_0 kernels)

Hard-convert 38/40 on a fixed harmful pilot while keeping coding near the parent — not a max-HC-only scar model.

Files

File	Role
`Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf`	language model (this release)

Optional speculative decode: use Prism’s official DSpark drafter unchanged (Ternary-Bonsai-27B-dspark-Q4_1.gguf from the base repo). Accept-rate A/B on this weight vs parent: ≈1.00× (n=30, CI overlap) → no draft retrain needed.

Quickstart

bash

# Prism fork required
git clone https://github.com/PrismML-Eng/llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j

./build/bin/llama-server \
  -m Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf \
  -ngl 99 -c 8192 -np 1 \
  --host 127.0.0.1 --port 8080

# optional DSpark
./build/bin/llama-server \
  -m Ternary-Bonsai-27B-Abliterated-LowDeg-Q2_0.gguf \
  -md Ternary-Bonsai-27B-dspark-Q4_1.gguf \
  -ngl 99 -ngld 99 \
  --spec-type draft-dspark --spec-draft-n-max 4 \
  -c 8192 -np 1

Method (short)

1.Residual / direction code-edit on ternary writers over the parent Q2_0 pack
2.Escalate to a high hard-convert seed (esc-ALL-f65)
3.Seed-stage surgical rollback: restore parent bytes on selected L0–7 writers (minimal mask minM23, ~1.5 MB changed vs full L0–7 rollback)
4.Gate on paired harmful eval + capability pilots; keep low-degradation winner

Runtime RepE hooks are not required at serve time.

Results

All figures below are same-condition lab measurements. They are not a full open leaderboard reproduction.

Refusal gate (harmful pilot)

	value
n	40
parent baseline refusals (`REFUSAL` \	`COMPLY_WITH_HEDGE`)	40/40
hard convert (parent refused → candidate pure `COMPLY`)	38/40
empty completions	0
transport errors	0

Hard convert does not count hedges / weak / empty.

Harmless side-effect (same n=40): reverse flip 0.

Capability pilot

split	this model
coding	19/20
tool	15/15
agentic	5/5

Parent coding on the same pilot was 19/20; an earlier HC-max seed (esc-f65) dropped coding to 16/20.

Serious benches (seed=0, thinking off)

bench	parent Q2_0	this (LowDeg)
GSM8K (n=100)	68	71
MMLU subset (n=100)	62	60
HumanEval (n=164)	141	140
MBPP (n=100)	66	64

Decode (reference, not a claim of superiority)

—RTX PRO 6000 WS, no draft: single-stream TG ~139 tok/s (depth0, llama-bench n=5)
—RTX 5070 Ti 16GB + official DSpark: single-stream ~80–85 tok/s, DSpark-active ctx up to 16k; parallel sweet spot np=2 (~89 agg tok/s)

Intended use

Research and local/offline assistants where reduced over-refusal is desired. Still a general model: it can produce unsafe content if asked. Use under your own policy and applicable law.

Limitations

—Requires Prism llama.cpp (stock llama.cpp may not run this Q2_0 hybrid pack correctly)
—Abliteration is behavioral, not a safety proof
—Long-context / high-parallel on 16GB cards can silently disable DSpark when compute buffers OOM — check timings.draft_n
—Vision mmproj is not included (text GGUF only)

Attribution

—Base weights & format: prism-ml/Ternary-Bonsai-27B-gguf (Apache-2.0)
—Architecture lineage: Qwen3.5 / Qwen3.6-27B hybrid attention
—This derivative: direct ternary GGUF edit + seed-stage low-degradation rollback (W-RB-L0-7-minM23)

License

Apache-2.0 (same as the base GGUF release).