Modelpublic

jakeatx/Qwen3.8-27B-Ternary-PTQ1_0-MTP-GGUF

sourceHugging Faceapache-2.0updated 9d agoView on Hugging Face

0likes416downloads

Model Card

Qwen3.8-27B Ternary PTQ1_0 (1.75 bpw) with Native MTP

Ternary (1.75 bpw) quantization of Qwen3.8-27B (Prism ML Ternary Bonsai 2 architecture) with native 1-layer MTP speculative drafting.

—Format: PTQ1_0 (group 128, 28 bytes/block, ~1.75 bpw)
—Model Size: 5.81 GiB
—MTP Drafter: Native 1-layer MTP head
—Target Hardware: Single 24 GB GPU (RTX 3090 / 3090 Ti) with 262K context capacity under full offload
—Runtime: Optimized for llamAmpere (branch feature/v0.3.1-bonsai2)

Usage with llamAmpere

bash

./build-sm86/bin/llama-server \
  -m Qwen3.8-27B-Ternary-PTQ1_0-MTP.gguf \
  -c 32768 -b 4096 -ub 1024 -t 8 -ngl 99 -fa on -ctk q8_0 -ctv turbo3 \
  --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-p-min 0