afkaf/Qwen3.8-27B-uncensored-w4a8-convrot-ComfyUI

Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 — W4A8 ConvRot for ComfyUI

A asym_w4a8_int8 quantization of AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16, packaged as a single-file ComfyUI text encoder.

20.2 GB on disk, ~19.4 GB resident. Fits entirely in 24 GB of VRAM with room for context, so it runs without offload on an RTX 3090 or 4090.

Built for single-image captioning and prompt generation through ComfyUI's Generate Text node. It is a full vision-language model: load it with CLIPLoader, wire an image in, and it describes what it sees.

Why this exists

An INT8 ConvRot build of this model lands around 32 GB. On a 24 GB card that means streaming weights over PCIe on every forward pass, and generation is memory-bandwidth-bound — every token re-reads the weights. Dropping to 4-bit weights is not about disk space; it is about staying resident.

W4A8 gives no compute advantage over INT8. It executes on the same int8 GEMM path and adds a codebook dequantization step on top. The entire win is that 16.4 GiB of weight traffic per token at ~936 GB/s beats streaming 10 GB over a ~25 GB/s PCIe link, by roughly an order of magnitude.

Format

asym_w4a8_int8 — ConvRot-rotated int4 weights, a per-tensor Lloyd-Max codebook, and fp8 group scales, executed on int8 GEMM. Calibration-free.

Each quantized layer carries four tensors:

Three tiers of scaling — codebook shape, group magnitude, row magnitude — which is why 4-bit weights hold up here. The codebook is fitted to each tensor's actual distribution rather than spacing levels uniformly, so the packed values are indices into a learned table, not magnitudes.

ConvRot applies a group-wise Hadamard rotation before quantization, spreading outliers across the group so no single large weight dominates its scale. Group size 256 for the rotation, 16 for the codebook.

Precision plan

Not everything is quantized. 387 tensors across layers 1–62 go to 4-bit; everything below stays BF16 for a specific reason.

On the MTP head

The multi-token-prediction head is retained at BF16 and costs zero VRAM — ComfyUI has no speculative decoding path, so its Qwen3.5 implementation never constructs those modules and the keys are dropped at load. It is kept because Comfy-Org's reference Qwen3.5 files keep it, and because MTP is BF16-or-nothing: quantized MTP weights collapse draft acceptance from the 79–85% range to 5–11%. Should ComfyUI ever gain speculative decoding, the weights are here and usable. Until then it is 0.79 GB of disk and nothing else.

Requirements

• —ComfyUI ≥ 0.31.0 (w4a8 loader support)
• —comfy-kitchen ≥ 0.2.31 with AsymW4A8Int8Layout
• —PyTorch built against CUDA 13.0+
• —Compute capability ≥ 8.0 (Ampere or newer). INT8 tensor cores are the execution path, so 30-series and up.

Usage

Drop the .safetensors into ComfyUI/models/text_encoders/.

1. 1.CLIPLoader → select this file. The type dropdown is ignored; detection is shape-based and resolves to QWEN35_27B automatically.
2. 2.Wire CLIP into Generate Text.
3. 3.Wire a Load Image into its image input.
4. 4.Write your prompt and run.

Sampling

From the base model card, non-thinking mode:

temperature 0.7   top_p 0.80   top_k 20   min_p 0.0   presence_penalty 1.5   repetition_penalty=1.0

Thinking mode: temperature 1.0, top_p 0.95, top_k 20, presence_penalty=0.0.

Verification

Every quantized layer was checked for serialization fidelity against comfy-kitchen's own output, not merely for "does it load."

tensors                 2747
parameters              17.472 B
size                    20.20 GB
quantized layers        387
dtypes                  BF16=812  F32=774  F8_E4M3=387  I8=387  U8=387
detect_te_model      -> QWEN35_27B
serializer           -> state_dict_tensors
serialization fidelity  ~0.000000
VERIFY PASSED

serialization fidelity compares this file's dequantized weights against comfy-kitchen re-quantizing the same source tensor. Near-zero means the file faithfully reproduces what the library itself produces — no scale or codebook tensor silently dropped, which is a failure mode that loads without error and computes wrong.

Note that weight_correction is absent by design: comfy-kitchen 0.2.31 does not emit a correction tensor in codebook mode, so correction=None at load is correct rather than a missing field.

Reproducing

Converted with a purpose-built script that reconciles the packed weight by tensor identity rather than by naming convention, holds the DeltaNet gates and edge layers out of quantization, and audits the result:

python convert_qwen38_w4a8.py \
    -i ./Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16 \
    -o ./qwen3.8_27b_uncensored_w4a8_convrot.safetensors \
    --comfyui-path ./ComfyUI --seed 42 --verify

--seed 42 pins the codebook fitting, which samples, so rebuilds are comparable.

Safety and user responsibility

This model has had its safety alignment removed. The upstream AEON-7 release applied directional ablation (abliteration) to suppress refusal behaviour. This repository contributes quantization only — no alignment, guardrails, or filtering were added, and none were removed here either.

Consequences you are accepting by using it:

• —It will not refuse. It will attempt to answer requests that the original Qwen3.8-27B declines, including harmful, illegal, or dangerous ones. There is no residual safety layer to catch anything.
• —Ablation degrades more than refusals. Suppressing refusal directions perturbs the model's weights generally. Expect some loss of judgment, calibration, and factual reliability relative to the original, in ways that are not confined to safety-adjacent topics.
• —4-bit quantization compounds this. The precision plan above minimizes it, but this is a lossy artifact of a lossy artifact.
• —Not suitable for unsupervised or public-facing deployment. If you expose this to users who are not you, you need your own moderation layer. It has none.
• —You are responsible for what you generate. Output is your responsibility, not the model's, not this repository's, and not any upstream author's. You are responsible for compliance with applicable law and with the licenses of all upstream artifacts.

No warranty of any kind. Provided as-is.

Attribution and non-endorsement

• —Base model: Qwen/Qwen3.8-27B by the Qwen team, Alibaba Cloud.
• —Ablation: AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16.
• —Quantization format: asym_w4a8_int8 from comfy-kitchen, by Comfy-Org.

This is an unofficial, community-produced derivative. It is not endorsed by, affiliated with, or supported by the Qwen team, Alibaba Cloud, AEON-7, or Comfy-Org. The behaviour of this model does not reflect the intentions, standards, or positions of any of them. Do not report issues with this model to those projects — its modifications are not theirs.

Please respect the license terms of all upstream artifacts. Verify the license field above against both parent repositories before redistributing.