RentedNoodle/Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-Uncensored

Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-Uncensored

A 3-bit GGUF quant of the uncensored OrcaRouter Qwen3.8-27B, carrying an S1-trained MTP draft head and a custom OrcaRouter-native imatrix. Built to run on a single 16 GB GPU.

A 9.75 GiB, mixed-precision GSQ/RCO quant of orcarouter/Qwen3.8-27B-Uncensored, preserving the MTP head and validated for 16 GB GPUs.

This is a quantization of `orcarouter/Qwen3.8-27B-Uncensored`, not a new fine-tune. The underlying model is Qwen3.8-27B; the surgical uncensoring comes from OrcaRouter; the quantization work here follows the GSQ/RCO methodology developed by ISTA-DASLab. The checkpoint is designed to substantially reduce learned refusal behavior. This is not a guarantee of universal compliance — the deployer is responsible for compliant use.

Quick specs

What I changed

Starting from the OrcaRouter uncensored checkpoint, I produced this GGUF using a GSQ/RCO-based non-uniform allocation and IQ3_XXS packing, then trained the MTP draft head (S1) and built a custom OrcaRouter-native importance matrix.

The important honesty clause: I reproduced ISTA-DASLab's published per-tensor RCO allocation verbatim from their stock-base artifacts. I did not independently re-run the multi-GPU budget search — same map, applied to the uncensored base.

The imatrix is custom and OrcaRouter-native; it differs from ISTA's on the edit-mask tensors. ISTA's published imatrix was also validated against this file. If a change makes the benchmark prettier but the model worse to actually use, it does not ship.

Tested performance (RTX 5070 Ti 16 GB, build2 llama.cpp, this exact file)

MTP head (S1-trained)

The offline k1 gain is large and generalizes (measured on a distribution never trained on). Serve-time throughput at n-max 2 is bounded by verification cost, so the head's drafting advantage shows as a modest serve-time gain — not a one-to-one map of the offline gain. Needle 6/6 re-verified on this file.

Usage

Froggeric template is required for the intended tool/reasoning behavior.

llama-server \
  -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf \
  --alias Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-Uncensored \
  --jinja --chat-template-file froggeric-qwen3.8-tool-use.jinja \
  --ctx-size 32768 -b 2048 -ub 2048 -fa on -ngl 99 \
  --reasoning-budget 256 --spec-type draft-mtp --spec-draft-n-max 2

`-b 2048` is required for depth-0.5 retrieval fidelity. This is a documented chunking artifact: a smaller batch splits the retrieval span and the depth-0.5 needles are lost. Keep -b 2048 -ub 2048 for retrieval-sensitive workloads.

Vision: projector files live in this repo under mmproj/ (BF16 or Q8_0). Start the server with one, then send a multimodal chat request (skip both flags for text-only):

llama-server -m Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf --mmproj mmproj/mmproj-Qwen3.8-27B-BF16.gguf [same flags as above]
# then POST /v1/chat/completions with content blocks including {"type":"image_url",...}

Ollama (text-only; tool/reasoning behavior is uncertified — use llama-server + Froggeric for the certified path):

FROM ./Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0.gguf
PARAMETER num_ctx 32768
PARAMETER num_gpu 99
PARAMETER temperature 0.6

• —Dual-engine routing: for ≤128K context use standard build2 (fast path; measured ~82 vs ~72 t/s at short context). For >128K up to 256K use the BrunoPPassini KV-ring fork — it holds 200K context and passes 2/2 needle retrieval where build2 cannot fit the KV cache.

Honest limitations

Aggressive 3-bit quant — do not expect BF16 behavior. Knowledge recall is weaker than higher-bit variants; the model leans on retrieval/tools instead. Vision + long context is tight on 16 GB (split text/vision profiles or offload the projector).

1. 1.RCO allocation is reproduced verbatim from ISTA-DASLab's stock-base artifacts (not re-searched) and applied to the uncensored base.
2. 2.The imatrix is custom OrcaRouter-native; it differs on the edit-mask tensors. ISTA's published imatrix was also validated against this file.
3. 3.Abliteration Tax — GPQA-Diamond 75.25% (harness-conditional). Measured with thinking ON, a generous thinking budget, and temp 0 over 198 questions; not a matched comparison to ISTA's stock-aligned 88.89%. The ~13-point gap is the documented cost of uncensoring — surgical rotation of the residual stream combined with 3-bit noise over long reasoning chains. It is a limitation of the abliterated base, not of the GSQ-RCO map, which is proven clean by WikiText-2 perplexity 6.17.
4. 4.The v1.1 Huihui line is frozen/deprecated in favor of this OrcaRouter line. v1.1 is retained for reproducibility only.
5. 5.Quant layout: IQ3XXS trunk (~3.06 bpw), MTP head Q6K, F32 norms preserved.

MTP output may differ from serial generation on quantized targets; fix the seed and runtime config for reproducibility, and disable spec when exact serial behavior matters.

Files in this repo

GGUF embeds tokenizer metadata — no separate tokenizer needed for llama.cpp/Ollama/LM Studio/Pi. The vision projector is included under mmproj/ (BF16 default, Q8_0 alternative); it is optional and text-only use works without it.

Credits

• —Qwen — architecture + pretrained weights: Qwen3.8 repo, Qwen3.8-27B
• —OrcaRouter — uncensored checkpoint: orcarouter/Qwen3.8-27B-Uncensored
• —GSQ — Dadgarnia, Tabesh, Nikdan, Helcig, Kurtic, Kleinegger, Alistarh (2026): paper, code
• —RCO — Helcig & Alistarh (2026): paper, code
• —ISTA-DASLab quant release — Qwen3.8-27B-GSQ-RCO-GGUF (allocation source; imatrix also validated)
• —Froggeric — Qwen-Fixed-Chat-Templates
• —llama.cpp/GGML — runtime + format; IQ3_XXS is their standard type, nothing custom here

Reproducibility

• —Base: orcarouter/Qwen3.8-27B-Uncensored
• —Allocation: REF-IQ3_XXS-mtp.rco-allocation.txt (in this repo) — ISTA-DASLab's published per-tensor RCO map, reproduced verbatim, applied via llama-quantize --tensor-type-file
• —Imatrix: imatrix.dat (in this repo) — custom OrcaRouter-native; ISTA's published matrix also validated
• —MTP head: S1-trained draft head (Q6_K), F32 norms preserved
• —Runtime: den_llama.cpp (build2)
• —GPU: RTX 5070 Ti 16 GB, Windows 11

Qwen3.8-27B-OrcaRouter-GSQ-RCO-IQ3_XXS-v2.0-qatfa.gguf (new file — FA-aligned MTP head)

A second GGUF in this repo, byte-identical to v2.0 except the MTP draft head block, calibrated for the FlashAttention KV path (-fa on -ctk q8_0 -ctv q4_0).

File

What changed vs v2.0

1. 1.FA-aligned MTP head @3300 — the draft head is fine-tuned against FA-trunk logits, restoring a large share of the acceptance lost under the FA KV path.
2. 2.ssm_alpha BF16 fix — ssm_alpha tensors corrected to BF16 (×48).

Lineage: same v2.0 trunk, same quant config, same template — the only delta is the head block.

Validation — fresh A/B (build2+ship control vs build-fa+qatfa treatment)

Same flags both legs; battery 2026-09-11 → 2026-09-13.

Caveats (stated plainly):

• —CUM noise ≈ 0.009 run-to-run. The clean-clean Δ (0.0196) sits only 0.0004 inside the 0.020 bound; the canonical Δ (0.0105) is the headline. Both are GREEN; no threshold was moved.
• —Needle is 5/6, not 6/6. needle-05 (depth 0.5) misses on all measurements — a consistent property of the file lineage, not a leg difference. The v2.0 card line "Needle 6/6" is a different measurement (15K-word haystack, build2 profile); on this A/B battery the result is 5/6 both legs.

Regime guidance

v2.0.gguf (build2 + ship) is retained unchanged. build-fa + v2.0.gguf is superseded by build-fa + v2.0-qatfa.gguf for >128K. No binary or file is deleted.

Versions

Changelog

• —v2.0 (current): OrcaRouter uncensored base; S1-trained MTP head; custom OrcaRouter-native imatrix; ISTA-verbatim RCO allocation. Needle 6/6, GPQA-Diamond 75.25% (harness-conditional), WikiText-2 ppl 6.17, toolcall 9/14, IFEval 70.24/76.26, TruthfulQA 77.60/81.98, 0% over-refusal. MTP offline k1 +0.278 over native; serve +13.6% t/s / +12.4% accept vs ISTA-imatrix build.
• —v2.0-qatfa: same trunk/quant as v2.0; FA-aligned MTP head @3300 + ssm_alpha BF16 fix. Fresh A/B: CUM accept Δ −0.0105 canonical / −0.0196 clean-clean (both ≤0.020 GREEN), t/s +18.4%, PPL −0.045%, needle 5/6 both legs. Designated >128K golden; v2.0 (build2) retained as ≤128K fallback.
• —v1.x (Huihui line) — frozen/deprecated. Superseded on every axis by this OrcaRouter line, except the native-head toolcall 10/14 vs 9/14 on this file. Retained for reproducibility only.

Roadmap

• —Bilingual (EN/ZH) ModelScope card: deferred to upload time if requested. This staging card is English-only.

Community quant, not affiliated with Qwen, OrcaRouter, Huihui, or ISTA-DASLab. Bug reports: include model revision, llama.cpp build, ctx/KV/spec settings, sampler. "Feels worse" is fine; reproducible info makes it actionable.