Modelpublic

ajgazin/Swift-Qwen3.8-27B-Uncensored-NVFP4

sourceHugging Faceotherupdated 2d agoView on Hugging Face

7likes749downloads

Model Card

Swift-Qwen3.8-27B-Uncensored-NVFP4

NVFP4 checkpoint of ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP, an abliterated Swift-Qwen3.8-27B (UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B). For vLLM and SGLang. GGUFs for llama.cpp: Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF. The Swift 1.5 version is Swift-1.5-Qwen3.8-27B-Uncensored-NVFP4.

—Uncensored: 15/100 refusals against Swift's 98/100, KL divergence 0.0634 (measured on the BF16 source).
—Swift's own NVFP4 recipe, unmodified, from ukisai/Swift-Qwen3.8-27B-NVFP4, calibrated with NVIDIA ModelOpt.
—W4A4: runs on native FP4 kernels on Blackwell.
—MTP head and vision tower in BF16, bit-identical to the source.

Precision

Component	Precision
192 MLP projections and `lm_head`	NVFP4 (block size 16, weights and activations)
144 Gated DeltaNet projections (`in_proj_qkv`, `in_proj_z`, `out_proj`)	FP8 E4M3
64 attention projections (`q_proj`, `k_proj`, `v_proj`, `o_proj`)	FP8 E4M3
Embeddings, norms, small DeltaNet tensors, vision tower, MTP head	BF16
KV cache	not quantized

21.9 GB, NVIDIA ModelOpt mixed-precision format.

Usage

bash

vllm serve ajgazin/Swift-Qwen3.8-27B-Uncensored-NVFP4 \
  --max-model-len 32768 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder

# self-speculative decoding with the built-in MTP head: add
  --speculative-config '{"method":"mtp","num_speculative_tokens":2}'

Needs a vLLM with ModelOpt mixed-precision support (tested on 0.29.0). No --quantization flag.

Sampling, as for Swift and Qwen: temperature 1.0, topp 0.95, topk 20, min_p 0. The model thinks before answering by default.

Tested on an RTX 5090 (32 GB) with vLLM 0.29.0: NVFP4 layers on FlashInfer's CUTLASS FP4 kernel (W4A4), FP8 layers on FlashInfer's FP8 kernel, text, MTP decoding and image input all working.

`--gpu-memory-utilization 0.88`	Weights	KV cache
Without MTP	18.9 GiB	91,629 tokens
With MTP	19.7 GiB	43,194 tokens

The model

The source applies the refusal direction of orcarouter/Qwen3.8-27B-Uncensored (Arditi et al. 2024, one direction) to Swift's weights. Method, direction and scripts are on the source model's card.

Model	Refusals	KL divergence
BF16 source (against Swift)	15/100	0.0634
Swift-Qwen3.8-27B	98/100	0

Measured with Heretic on the BF16 source. Not re-measured on this checkpoint: Heretic cannot load a ModelOpt export.

Quantization

—ModelOpt 0.47.0rc0 at commit `022767c`, the one Swift used. One RTX PRO 6000 Blackwell.
—Recipe: `modelopt-recipe.json`, Swift's. Local-Hessian calibration with the FP8 scale sweep for NVFP4, max calibration for FP8.
—Calibration: 2,048 samples (6,450,474 tokens, up to 4,096 each) from NVIDIA's seven Nemotron post-training v3 datasets, rendered with the chat template, seed 42. Swift's sample selection script is not published, so the samples differ from theirs; `calib-manifest.json` records ours.

Checks

Held-out NLL (64 samples, disjoint from calibration)
BF16	2.93182
Quantized (ModelOpt simulation, weights and activations)	2.93752 (+0.19%)

—193 NVFP4 and 208 FP8 modules, exactly the recipe's; tensor and scale counts equal Swift's checkpoint.
—All 798 BF16 tensors, the 15 MTP tensors included, are bit-identical to the source (`precision-inventory.csv`).

Reproduce

`quantization/` has the recipe, manifests, scripts, package list and run log.

bash

git clone https://github.com/NVIDIA/Model-Optimizer && cd Model-Optimizer
git checkout 022767c7ab3d7d36211affd85e5c496770cde768
pip install torch==2.13.0 --index-url https://download.pytorch.org/whl/cu130
pip install ".[hf]" transformers==5.14.1 datasets==5.0.1 accelerate==1.14.0 safetensors==0.8.0 \
  flash-linear-attention==0.5.2
cd ..

# from a download of quantization/; <source dir> is the source repo at revision f39e20d
python scripts/check_recipe.py --recipe modelopt-recipe.json \
  --index <source dir>/model.safetensors.index.json
python scripts/build_calib.py --manifest calibration-manifest.json --tokenizer <source dir> --out calib
python scripts/quantize_nvfp4.py --model <source dir> --recipe modelopt-recipe.json \
  --calib calib/calib.jsonl --heldout calib/heldout.jsonl --out out \
  --modelopt-examples Model-Optimizer/examples/hf_ptq
python scripts/check_export.py --out out --source <source dir> --recipe modelopt-recipe.json

About 60 GB of GPU memory, about an hour.

Not evaluated

Refusals and KL divergence of the quantized model, general benchmarks, and FP8 KV cache (the checkpoint has no KV scales).

License

Derivative of Swift-Qwen3.8-27B, under the Swift Open License v1.0 (Swift model card): free for individuals and organizations up to US$1,000,000 annual recurring revenue, above that commercial use needs a Swift Enterprise License from UkisAI. Qwen3.8-27B and orcarouter/Qwen3.8-27B-Uncensored are Apache 2.0.

Intended use

The model answers requests the original declines. You are responsible for how you use it and for complying with applicable law and the license.

Credits

—Qwen for Qwen3.8-27B.
—UkisAI for Swift-Qwen3.8-27B and its NVFP4 recipe.
—OrcaRouter for Qwen3.8-27B-Uncensored and its refusal direction.
—NVIDIA for ModelOpt and the Nemotron datasets.
—Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).
—Heretic, used to evaluate the source model.