ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF

Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF

GGUF quants of ajgazin/Swift-Qwen3.8-27B-Uncensored-MTP, an abliterated Swift-Qwen3.8-27B (UkisAI's reasoning-efficient fine-tune of Qwen3.8-27B). For vLLM and SGLang: NVFP4.

• —Uncensored: 15/100 refusals against Swift's 98/100, KL divergence 0.0634.
• —Unsloth Dynamic 3.0 layout: every tensor has the type Unsloth chose for its quant of the same size, with Unsloth's importance matrix.
• —MTP head included in every main GGUF, for self-speculative decoding in llama.cpp.
• —Vision: image and video input through the included projector.
• —Measured: KL divergence against the BF16 (see Quality).

Quants

Each quant is one file, Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-<quant>.gguf; the BF16 is Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf.

Other files:

Quality

Each quant compared with the BF16 it was made from, token by token: wikitext-2 test set, 36 × 8192 tokens, f16 KV cache, llama.cpp 94659b076. Lower KLD and higher top-1 are better.

[image]

• —UD-IQ3_S and UD-IQ4_XS were added after this measurement run, so they have no numbers yet and are absent from the chart for the same reason. Both come with the next batch.
• —Mean KLD is the average divergence of the quant's next-token distribution from the BF16's. 99 % KLD is the level only the worst 1 % of tokens exceed. Same top-1 is how often both pick the same most likely token.
• —Quality follows file size with no exceptions. The knee is at UD-Q5KM: above it a GiB buys little, below it each GiB saved costs about twice as much.
• —The typical token is barely touched (median KLD is 0.0002 at UD-Q8KXL and 0.003 at UD-Q4KXL); the mean is carried by a small share of tokens that change a lot. From UD-Q3KXL down the error also turns one-sided, taking 0.4–1.3 % of probability from the BF16's token on average.

This measures quantization only, against this repository's own BF16. It is a different quantity from the 0.0634 under The model, which is the abliteration measured against Swift. Prose only: code and chat-formatted text were not measured.

Every statistic is in kld_results.csv. It also includes perplexity, which does not rank these files: several quants score below the BF16.

Usage (llama.cpp)

# text only
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768

# with vision (image and video input)
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768 \
  --mmproj mmproj-BF16.gguf

# self-speculative decoding with the built-in MTP head
llama-server -m Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q6_K_XL.gguf -ngl 99 -c 32768 \
  --spec-type draft-mtp

--spec-type draft-mtp needs a llama.cpp build with MTP support for qwen35. The MTP head loads from the main GGUF; there is no separate draft file.

Sampling, as for Swift and Qwen: temperature 1.0, topp 0.95, topk 20, min_p 0. The model thinks before answering by default.

The model

The source applies the refusal direction of orcarouter/Qwen3.8-27B-Uncensored (Arditi et al. 2024, one direction) to Swift's weights: 131 tensors edited (attention outputs, mlp.down_proj, embed_tokens, MTP layer included), everything else Swift's. Method, direction and scripts are on the source model's card.

Measured with Heretic on the BF16 weights (100 prompts from mlabonne/harmful_behaviors, keyword refusal detector; first-token KL on mlabonne/harmless_alpaca; thinking skipped). Not re-measured on the quants.

Quantization

• —Conversion: the source model's BF16 safetensors (all 1199 tensors, MTP included) were converted with llama.cpp's convert_hf_to_gguf.py, once for the language model and once with --mmproj for the vision projector.
• —Quantization: llama-quantize with Unsloth's imatrix_unsloth.gguf from unsloth/Qwen3.8-27B-GGUF, plus a --tensor-type-file that sets every tensor's type to the one in Unsloth's GGUF of the same size.

The imatrix was calibrated on the original Qwen3.8-27B; Swift's low-rank fine-tune and the rank-one abliteration change the weights only slightly.

Checks

• —Layout: each quant's 866 tensors have exactly the types and shapes of Unsloth's file of the same size.
• —MTP: every main GGUF has 65 blocks, with the MTP layer's 15 tensors in blk.64 (nextn_predict_layers = 1). llama.cpp's --spec-type draft-mtp loads it and drafts with it.
• —Tokenizer and chat template:
• —The vocabulary, merges and special tokens are the same as in Unsloth's Qwen3.8-27B GGUF.
• —add_bos_token = false and the padding token <|endoftext|> come from Swift's own tokenizer config.
• —The embedded chat template is byte-identical to Swift's, which is Qwen3.8-27B's.
• —Vision projector: all 334 tensors are identical to Qwen3.8-27B's vision projector. Neither Swift's fine-tune nor the abliteration touches the vision tower.
• —Load test: every quant loads and runs on a 32 GB GPU: fully offloaded up to UD-Q6KXL, with 56 of 65 layers offloaded for UD-Q8KXL. Each produced sane logits on a short perplexity run before upload; the measurement that ranks them is under Quality.

Reproduce

# 1. convert (Swift-Qwen3.8-27B-Uncensored-MTP/ is a download of the source repository)
python convert_hf_to_gguf.py Swift-Qwen3.8-27B-Uncensored-MTP --outtype bf16 \
  --model-name Swift-Qwen3.8-27B-Uncensored-MTP --outfile Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf
python convert_hf_to_gguf.py Swift-Qwen3.8-27B-Uncensored-MTP --outtype bf16 --mmproj \
  --model-name Swift-Qwen3.8-27B-Uncensored-MTP --outfile mmproj-BF16.gguf

# 2. per-size --tensor-type-file from tensor_types.tsv (here UD-Q5_K_M), one anchored regex per tensor
awk -F'\t' 'NR==1 { for (i=2; i<=NF; i++) if ($i=="UD-Q5_K_M") c=i; next }
            { n=$1; gsub(/\./,"\\\\.",n); print "^" n "$=" $c }' \
  tensor_types.tsv > tensor_types_UD-Q5_K_M.txt

# 3. quantize. The last argument is the fallback type for tensors the file doesn't list:
#    UD-Q2_K_XL Q2_K, UD-IQ3_XXS IQ3_XXS, UD-IQ3_S IQ3_S, UD-Q3_K_XL Q3_K_L,
#    UD-IQ4_XS IQ4_XS, UD-Q4_K_S Q4_K_S, UD-Q4_K_XL Q4_K_M, UD-Q5_K_S Q5_K_S,
#    UD-Q5_K_M Q5_K_M, UD-Q6_K_XL Q6_K, UD-Q8_K_XL Q8_0
llama-quantize --imatrix imatrix_unsloth.gguf \
  --tensor-type-file tensor_types_UD-Q5_K_M.txt \
  Swift-Qwen3.8-27B-Uncensored-MTP-BF16.gguf \
  Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q5_K_M.gguf Q5_K_M

tensor_types.tsv holds the type of every non-F32 tensor in each of Unsloth's sizes, one row per tensor and one column per size. The regexes must be anchored (^…$). llama-quantize matches them unanchored, so a bare output\.weight line would also match every attn_output.weight.

Not evaluated

General benchmarks, KL divergence of the quants on code or chat-formatted text, refusal behaviour in thinking mode and on the quants, whether Swift's shorter reasoning traces survive, and MTP acceptance against Swift.

License

Derivative of Swift-Qwen3.8-27B, under the Swift Open License v1.0 (Swift model card): free for individuals and organizations up to US$1,000,000 annual recurring revenue, above that commercial use needs a Swift Enterprise License from UkisAI. Qwen3.8-27B and orcarouter/Qwen3.8-27B-Uncensored are Apache 2.0.

Intended use

The model answers requests the original declines. You are responsible for how you use it and for complying with applicable law and the license.

Credits

• —Qwen for Qwen3.8-27B.
• —UkisAI for Swift-Qwen3.8-27B.
• —OrcaRouter for Qwen3.8-27B-Uncensored and its refusal direction.
• —Unsloth for the Dynamic 3.0 quantization layout and imatrix.
• —Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024).
• —Heretic, used for evaluation.