Modelpublic

orcarouter/Qwen3.8-27B-Uncensored-MLX

sourceHugging Faceapache-2.0updated 29d agoView on Hugging Face

1.5klikes121kdownloads

Model Card

<h1>Qwen3.8-27B-Uncensored-MLX</h1>

<p><em>An abliterated (refusal-removed) MLX build of Qwen's Qwen3.8-27B — 2 / 4 / 6 / 8-bit for Apple Silicon</em></p>

<p><strong>One Gateway. Every Model.</strong> — Route Smarter · Ship Safer · Spend Less.</p>

<p> <a href="https://www.orcarouter.ai">Website</a> · <a href="https://www.orcarouter.ai/models">Model Catalog</a> · <a href="https://www.orcarouter.ai/models/qwen/qwen3.8-27b">Model API</a> · <a href="https://github.com/Continuum-AI-Corp">GitHub</a> · <a href="https://discord.gg/yAh6Tex6kx">Discord</a> · <a href="https://x.com/OrcaRouter">X</a> </p>

</div>

An abliterated (refusal-removed) build of `Qwen/Qwen3.8-27B` — a 27B-parameter dense, hybrid-attention (Gated DeltaNet linear + full attention) native vision-language model with thinking control, tool-calling and an MTP head — quantized to MLX format for Apple Silicon. Four precisions are provided — 2 / 4 / 6 / 8-bit (affine, group size 64) — each as a subfolder, with the 4-bit build also mirrored at the repo root so that orcarouter/Qwen3.8-27B-Uncensored-MLX loads directly in LM Studio and other tools that treat a repo as a single model. The vision tower, norms and conv layers are kept in BF16; only the language-model linear weights (including embed_tokens / lm_head) are quantized. Browse all models in the OrcaRouter Model Catalog. This model is deployed as API here.

⚠️ Disclaimer & risks — read before use

This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:

—It will comply with harmful, unethical, offensive, or illegal requests that the original Qwen3.8-27B would refuse. It has no meaningful built-in guardrails.
—It is released strictly for legitimate research — interpretability, AI-safety and refusal-mechanism study, red-teaming, robustness evaluation, and controlled experiments.
—You assume full responsibility and liability for how you use it and for everything it generates. Add your own safety, moderation and abuse-prevention layers before any deployment.
—Use must comply with the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0) inherited from the base model, and all laws and regulations that apply to you.
—The authors and uploaders accept no liability for any misuse or harm. Outputs do not reflect the views of the uploaders or of Qwen / Alibaba.

Specific risks

—Harmful content on demand — it will produce instructions for malware, exploits, weapons, fraud and other illegal or dangerous activity when asked.
—No refusals — jailbreak / safety probes "succeed" trivially; do not mistake this for a passing safety evaluation.
—Confident falsehoods & bias — it can generate false, defamatory, biased or offensive text and present it authoritatively.
—Expanded attack surface — preserved vision, tool-calling and 262K context mean these risks extend to image understanding and autonomous / agentic use.
—Quantization noise — lower-bit builds (esp. 2-bit) add instability on top of the above; outputs can be degraded or nonsensical.

Intended use vs out of scope

—Intended: AI-safety and interpretability research, refusal-mechanism study, red-teaming, guardrail and robustness evaluation, controlled academic experiments.
—Out of scope: any deployment to end users, minors, or production without your own moderation / safety layer; any unlawful, harmful, or rights-infringing use.

By downloading or using this model you acknowledge and accept the above.

Available quantizations

Folder	Bits/weight	Size	Shards	Min Mac RAM	Quality vs BF16 source
`8-bit/`	8.627	~27.5 GB	6	32 GB	Near-lossless — recommended for quality
`6-bit/`	6.661	~22 GB	5	24–32 GB	Excellent — strong quality/size balance
`4-bit/`	4.695	~15 GB	3	24 GB	Very good — recommended default
`2-bit/`	2.729	~8.7 GB	2	16 GB	⚠️ Severely degraded — archival only

2-bit warning: at 27B, 2-bit quantization collapses generation quality (repetition loops, garbled output). It is included only as an extreme-compression archive; do not use it for real work — prefer 4-bit or higher.

Repo root = `4-bit/`. The root of this repo holds a copy of the 4-bit build, so --model orcarouter/Qwen3.8-27B-Uncensored-MLX (no subfolder) resolves to 4-bit. Use the subfolder paths to pick any other precision.

Verification & test results

All builds were quantized from the same abliterated BF16 source and verified numerically (dequantized weights vs. source) plus tested by generation on GPU.

Precision	Numerical fidelity (cosine)	Text / Chinese / Code	Refusal probes	Vision
8-bit	cos 0.9997	✅	✅ 0 refusals	✅
6-bit	cos 0.9996	✅	✅ 0 refusals	✅
4-bit	cos 0.996	✅	✅ 0 refusals	✅
2-bit	cos 0.92	⚠️ breaks down	⚠️ garbled (not refusal)	partial

—Uncensored preserved: red-team probes (exploit walkthrough, controversial argument) return substantive content with zero refusals on 4 / 6 / 8-bit.
—Multimodal preserved: shapes, colors, position, background and text in a probe image are described correctly on 4 / 6 / 8-bit.
—Speed: ~32–37 tok/s steady-state on a single H200 (MLX CUDA backend). MLX's native target is Apple Silicon (Metal).

Note: on 6-bit, mlx's offline mx.dequantize mis-unpacks these weights (a library edge case), so correctness is verified by clean generation — inference is unaffected.

Usage (mlx-vlm, Apple Silicon)

bash

pip install -U mlx-vlm    # needs mlx-vlm >= 0.6.13, mlx >= 0.32

# download one precision (e.g. 4-bit) from the subfolder
hf download orcarouter/Qwen3.8-27B-Uncensored-MLX --include "4-bit/*" \
    --local-dir ./Qwen3.8-27B-Uncensored-MLX

# text
python -m mlx_vlm generate \
    --model ./Qwen3.8-27B-Uncensored-MLX/4-bit \
    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

# vision (image + text)
python -m mlx_vlm generate \
    --model ./Qwen3.8-27B-Uncensored-MLX/4-bit \
    --image path/to/image.png \
    --prompt "Describe this image." --max-tokens 256

# OpenAI-compatible server
python -m mlx_vlm server --model ./Qwen3.8-27B-Uncensored-MLX/4-bit --port 8080

On Apple Silicon the Metal backend is used automatically — no CUDA setup needed. (On a Linux CUDA backend, vision requires MLX_CUDA_USE_CUDNN_SDPA=0; this does not apply on macOS.)

Multi-Token Prediction (MTP) — speculative decoding

This model has a native MTP head. In MLX, MTP is loaded as a separate drafter for speculative decoding: the main model is loaded with the MTP weights stripped, and the drafter is passed explicitly. The drafter lives in the `mtp/` subfolder of this repo (model_type: qwen3_5_mtp) and works with any main-model precision (4 / 6 / 8-bit).

Setting an mtp_enabled flag on the main model alone does nothing — MLX needs the separate drafter passed via --draft-model … --draft-kind mtp.

bash

# fetch a main-model precision (e.g. 6-bit) plus the MTP drafter
hf download orcarouter/Qwen3.8-27B-Uncensored-MLX --include "6-bit/*" "mtp/*" \
    --local-dir ./Qwen3.8-27B-Uncensored-MLX

# generate with MTP speculative decoding
python -m mlx_vlm generate \
    --model       ./Qwen3.8-27B-Uncensored-MLX/6-bit \
    --draft-model ./Qwen3.8-27B-Uncensored-MLX/mtp \
    --draft-kind mtp --draft-block-size 4 \
    --prompt "Explain quantum entanglement in one sentence." --max-tokens 256

# OpenAI-compatible server with MTP
python -m mlx_vlm server \
    --model       ./Qwen3.8-27B-Uncensored-MLX/6-bit \
    --draft-model ./Qwen3.8-27B-Uncensored-MLX/mtp \
    --draft-kind mtp --draft-block-size 4 --port 8080

Requirements: an mlx-vlm build with the qwen3_5_mtp drafter and --draft-kind mtp (available on mlx-vlm main). MTP acceptance is lossless — with greedy decoding the output is identical to running without the drafter, just fewer forward passes on accepted tokens. The speedup is realized on Apple Silicon (Metal); one drafter serves all precisions.

Usage (LM Studio)

Search for orcarouter/Qwen3.8-27B-Uncensored-MLX in LM Studio and download it — the repo root is the 4-bit build, and the other precisions appear as separate download options.

Three things to get right:

1.This repo is gated. LM Studio downloads anonymously by default and will get an HTTP
2.Accept the terms on the model page once, then paste a Hugging Face read token into LM Studio under Settings → Integrations → Hugging Face.
3.Turn off KV cache quantization. MLX vision models do not support it on this architecture, and loading fails during initialization if it is enabled (mlx-engine#286).
4.Pick a quant that fits. 8-bit is ~29.5 GB on disk and wants a 64 GB Mac; 6-bit suits 48 GB; 4-bit (~16 GB) is the right choice on a 32 GB Mac. LM Studio's "Likely too large" badge is a RAM warning, not an error.

If you are on an older LM Studio MLX runtime, update it (Settings → Runtime): qwen3_5 support landed in mlx-vlm 0.6.x, and older runtimes cannot load this architecture at all.

Model details


Base model	`Qwen/Qwen3.8-27B`
Architecture	`Qwen3_5ForConditionalGeneration` — 64 layers, hidden 5120, hybrid Gated DeltaNet (48 linear + 16 full attention, interval 4), native VL tower
Modification	Abliteration (refusal-direction removal), then MLX affine quantization
Quantization	MLX affine, group size 64, per-precision 2 / 4 / 6 / 8-bit
Kept in BF16	vision tower, all norms, linear-attention `conv1d`
Quantized	language-model linear layers incl. `embed_tokens` and `lm_head`
Context	262,144 tokens