pcmaker/Huihui-Qwen3.8-27B-Uncensored-NInfer

Huihui‑Qwen3.8‑27B‑Uncensored — ready‑to‑run NInfer artifact (RTX 5090)

A prebuilt .ninfer artifact of huihui‑ai/Huihui‑Qwen3.8‑27B‑abliterated (uncensored), so you can run it on a single RTX 5090 without doing the quantize/convert yourself.

• —Full native context: 262,144 tokens
• —4‑bit KV cache (K8V4) + optional host‑RAM KV offload
• —DFlash2 speculative decoding (built into the artifact)
• —~120–180 tok/s decode on one 5090, ~3,470 tok/s prefill on long prompts
• —OpenAI‑ and Anthropic‑compatible server

Authorized use only. Uncensored/abliterated weights remove the model's refusals. Use this for red‑teaming your own apps, security research and offline experiments. You are responsible for what you generate.

Requirements

• —NVIDIA RTX 5090 (Blackwell, sm_120), 32 GB
• —Ubuntu 24.04, CUDA 13.3
• —NInfer built from source — github.com/Neroued/ninfer

Run

# 1) build NInfer (see its README), then:
export LD_LIBRARY_PATH=/usr/local/cuda-13.3/lib64:$LD_LIBRARY_PATH
hf download pcmaker/Huihui-Qwen3.8-27B-Uncensored-NInfer --local-dir ./art

./ninfer/build/apps/ninfer-serve ./art/huihui_uncensored_nvfp4.ninfer \
  --host 0.0.0.0 --port 8188 \
  --max-context 262144 \
  --kv-dtype k8v4 \
  --spec dflash2 --draft-tokens 3 \
  --default-thinking-budget 600

Then call the OpenAI‑compatible endpoint at http://localhost:8188/v1/chat/completions. Disable "thinking" per request with {"reasoning_effort": "none"}.

How it was built

Full step‑by‑step guide (build → quantize → convert → serve), scripts and benchmarks: [pcmaker/Qwen3.8‑27B‑Uncensored‑NInfer‑RTX5090‑Guide](https://huggingface.co/pcmaker/Qwen3.8-27B-Uncensored-NInfer-RTX5090-Guide)

In short: the uncensored BF16 base was quantized to the mixed FP8+NVFP4 layout that NInfer's converter expects, then combined with unsloth/Qwen3.8-27B-NVFP4 (reference layout) and z-lab/Qwen3.8-27B-DFlash2 (draft head) into this single .ninfer file.

Credits

huihui‑ai (uncensored base), the Qwen team (base model + tokenizer), unsloth (NVFP4 reference), z‑lab (DFlash2), and NInfer (engine). All weights and tools belong to their respective authors and licenses.