Vtuber-plan/Qwen3.8-27B-Uncensored-NVFP4

Qwen3.8-27B-Uncensored-NVFP4

NVFP4 (4-bit per-block) quantized version of the Qwen3.8-27B Uncensored model, produced with NVIDIA TensorRT Model Optimizer.

Quality Comparison (Original vs BF16 vs NVFP4)

Preliminary evaluation with lm_eval + sglang (greedy, temperature 0). MMLU / CMMLU / C-Eval use a 20-question-per-subtask sample; GSM8K uses the full test set. We compare: the original Qwen3.8-27B, our Uncensored BF16 (before quantization), and the Uncensored NVFP4 (4-bit) checkpoint.

[image]

Key takeaways:

• —Quantization preserves quality. Unlike the Original vs Uncensored gap, NVFP4 tracks BF16 almost exactly — all NVFP4−BF16 deltas are within ±1 standard error, i.e. no measurable degradation from 4-bit quantization (~4× weight compression).
• —Uncensored is generally stronger on these benchmarks than the stock original. The uncensored checkpoint scores higher on CMMLU / C-Eval / GSM8K and comparable on MMLU. This is not caused by quantization — the same difference already exists between the uncensored BF16 model and the stock original, so it reflects the uncensoring/finetuning itself.
• —The small positive NVFP4−BF16 deltas are within noise and should not be read as "NVFP4 is better than BF16"; the practical takeaway is that 4-bit quantization is effectively lossless on these tasks.

Quantization

• —Format: NVFP4 weights, FP8 KV cache, group size 16
• —Tool: NVIDIA ModelOpt 0.45.0
• —Calibration: ultrachat_200k + nvidia/Nemotron-SFT-Multilingual-v2 (code/math/stem across Japanese, Korean, Portuguese, Hindi)
• —Excluded modules: lm_head, embeddings, linear-attention conv1d/in_proj_a/in_proj_b, and MTP layers

Shards

All safetensors shards are ≤ 5 GB (5 shards), so the repository can be cloned and uploaded without Hugging Face large-file (>5 GB) restrictions.

Usage

Load with Hugging Face transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Vtuber-plan/Qwen3.8-27B-Uncensored-NVFP4"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)

Note: this NVFP4 checkpoint is intended for deployment with frameworks that support the ModelOpt NVFP4 format (e.g. TensorRT-LLM). Plain transformers/BF16 inference will not dequantize it natively and requires the corresponding quantization backend.