Modelpublic

protogeni/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4

sourceHugging Faceapache-2.0updated 11d agoView on Hugging Face

7likes1.3kdownloads

Model Card

Qwen3.8-27B Uncensored · W4A16 + MTP4

A complete uncensored checkpoint optimized for speculative decoding on a single 24 GB GPU. 100K text context with BF16 KV cache and MTP4 is tested on an RTX 3090 Ti.

The W4A16 transformer body is paired with INT8 token embeddings, calibrated GPTQ INT4 output/MTP heads, and a 40,960-token draft vocabulary selected from this model's own outputs. Group size is 128. The vision tower is retained; the 100K serving profile is text-only.

100K serving setup

Use the patched syv-ai runtime, tested with vLLM 0.27.1. 100K profile details.

Download into the runtime repository:

bash

hf download protogeni/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4 \
  --local-dir models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4

Merge these settings into its .env:

dotenv

MODEL=/app/models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4
CTX=fast
SPEC=mtp
DRAFT_TOKENS=4
PREFIX_CACHE=1
VISION=0
GPU_UTIL=0.95
MAX_LEN=100000
MAX_SEQS=4
CG=8
EXTRA_ARGS=--kv-cache-memory=7730941132 --max-num-batched-tokens 512

bash

docker compose --profile single up -d --no-deps --force-recreate single

Set the client's context limit to 100,000 tokens, including the reply.

Measured performance

RTX 3090 Ti, BF16 KV + MTP4. Generation rates exclude prompt processing and include reasoning tokens.

Workload	Generation speed
Three short coding prompts, pooled	128.4 tokens/s
64,230 input + 1,024 output tokens	92.1 tokens/s
96,238 input + 3,072 output tokens	82.7 tokens/s
98,233 input + 1,536 output tokens	84.6 tokens/s

Quality results

Benchmark	Result
GSM8K	94.77% — 1,250/1,319
MMLU-Pro, 5-shot	76.14% — 1,066/1,400
HumanEval	94.51% pass@1 — 155/164
HumanEval+	90.85% pass@1 — 149/164
IFBench	39.33% strict / 42.00% loose
MMMU validation	24.56% — 221/900; 891 output-cap hits

MMLU-Pro used thinking; the other listed tests used thinking off. MMMU used an eager, MTP-off profile. Protocols, raw results, and earlier speed tests.

Lineage and license

Qwen/Qwen3.8-27B → OrcaRouter uncensored → noon-at-cgn AutoRound W4A16 → this serving-optimized checkpoint.

Build and calibration details. Apache-2.0 license.