d0xin/Swift-Qwen3.8-27B-Uncensored-NVFP4-LocalHessian-ActivationHeadroom-NInfer

Swift-Qwen3.8-27B-Uncensored-NVFP4-LocalHessian-ActivationHeadroom-NInfer

Native NInfer build of d0xin/Swift-Qwen3.8-27B-Uncensored-BF16, re-quantized from BF16 using NVIDIA ModelOpt Local-Hessian calibration + activation headroom and a conservative mixed NVFP4/FP8 topology.

Release goals

The goal of this release is straightforward: retain Swift-Qwen3.8's capabilities while removing refusal behavior.

It is designed to preserve Swift-Qwen3.8's reasoning, agentic, tool-calling, multimodal and long-context capabilities while removing refusal behavior.

For this quantized build specifically, the goal is also to increase inference throughput and reduce VRAM usage while preserving FP8-level measured quality and avoiding longer reasoning traces under the recommended B2048 serving profile.

In our fixed validation, this build delivered +20–27% decode throughput, +34–38% prefill throughput, and 15.1% lower VRAM usage versus FP8, while scoring 225/280 vs 224/280 on the B2048 MMLU-Pro subset. Mean reasoning usage was also lower: 741.9 vs 793.0 tokens.

What makes this build different

This is not a repack of an existing NVFP4 checkpoint.

Calibration used NVIDIA ModelOpt Local-Hessian, activation headroom, FP8 scale sweep, layer-wise checkpointing, upper_percentile=99.99, and rho=16384.

Quality validation

Fixed MMLU-Pro subset: 280 questions, 14 categories, 20 questions per category, seed 20260918, temperature=0, reasoning_effort=xhigh, max_tokens=4096.

For FP8+B2048 vs this build+B2048: both correct 216, both wrong 47, FP8-only correct 8, NVFP4-only correct 9, exact McNemar p=1.0.

No statistically significant accuracy difference was observed on this subset.

Recommended thinking budget

Recommended NInfer setting: --default-thinking-budget 2048.

Without a thinking budget, 36/280 NVFP4 evaluation generations reached the 4096-token output ceiling. With B2048, reasoning median was 379.5 tokens, p90 was 2072, mean was 741.87, 0/280 reached the 4096-token limit, all 280 requests finished normally, and accuracy was 225/280.

The thinking budget is a serving recommendation and is not baked into the artifact.

Performance vs FP8

Test hardware: NVIDIA RTX PRO 6000 Blackwell Workstation Edition.

Performance is hardware- and workload-dependent.

Refusal evaluation

Fixed Swift ORCA-K1 100-prompt evaluation:

Configuration: temperature=0, seed=42, max_tokens=1536, thinking disabled.

Historical FP8 responses re-judged with the same current judge produced 87 DIRECT, 12 SAFETYDEFLECT, 1 REFUSE, and 0 OTHERFAILURE.

The 0/100 refusals result applies only to this fixed automated evaluation. It is not a universal non-refusal guarantee.

Safety and responsible use

This model has deliberately reduced refusal behavior.

Reduced refusal behavior also reduces model-level safety filtering. Users are responsible for evaluating the model for their own deployment, safety, legal, and compliance requirements.

The 0/100 figure describes the documented fixed evaluation set and inference configuration. It should not be interpreted as a guarantee that no conceivable prompt, sampling configuration, system prompt, or inference engine can ever produce a refusal.

Artifact

• —File: Swift-Qwen3.8-27B-Uncensored-NVFP4-LH-ActHeadroom.ninfer
• —Size: 23,719,715,844 bytes
• —SHA-256: a298e28529a31f5447574ba2ad5ba5812d952053292ca017115f9d735dfdadcd
• —Stored objects: 1,246

The artifact contains Text, Vision, MTP, DFlash2, the optimized proposal head, and tokenizer/frontend resources.

Runtime requirements

Validated NInfer revision: f76e19c0fbd026c86f46005acf2c80c54084bade.

Current NInfer targets NVIDIA Blackwell sm_120a.

This build was directly validated on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition.

RTX 5090

The artifact is expected to fit on a 32 GB RTX 5090, but the exact RTX PRO 6000 serving profile should not be copied unchanged.

Our RTX PRO 6000 production profile uses about 35.1 GiB VRAM, so the full --kv-capacity 262144 profile with Vision enabled does not fit within 32 GB.

Upstream NInfer runs Qwen3.8-27B NVFP4 on RTX 5090 with approximately:

• —252,928 tokens maximum text-only context
• —81,920 tokens when Vision is enabled

This exact quantized build has not yet been directly benchmarked by the author on RTX 5090, so RTX 5090 support should be considered expected rather than independently validated.

The current .ninfer runtime should not be considered portable to Ampere, Ada, or Hopper.

Recommended RTX 5090 profile

For a 32 GB RTX 5090, start with a text-only profile like this:

ninfer-serve \ ./Swift-Qwen3.8-27B-Uncensored-NVFP4-LH-ActHeadroom.ninfer \ --host 0.0.0.0 \ --port 8080 \ --model-id Swift-Qwen3.8-27B-Uncensored-NVFP4 \ --max-context 250000 \ --kv-capacity auto \ --max-concurrency 2 \ --kv-dtype fp8 \ --prefill-chunk 1024 \ --spec dflash2 \ --draft-tokens 5 \ --lm-head-draft \ --preserve-thinking \ --default-thinking-budget 2048

For Vision workloads, use a substantially smaller context budget; 81920 is a reasonable starting point based on upstream NInfer's RTX 5090 Qwen3.8 NVFP4 profile.

--kv-capacity auto is recommended on 32 GB cards so NInfer can size the shared Main Text KV pool from the VRAM remaining after model and runtime allocations.

Recommended NInfer server

ninfer-serve \ ./Swift-Qwen3.8-27B-Uncensored-NVFP4-LH-ActHeadroom.ninfer \ --host 0.0.0.0 \ --port 8080 \ --model-id Swift-Qwen3.8-27B-Uncensored-NVFP4 \ --max-context 262144 \ --kv-capacity 262144 \ --max-concurrency 8 \ --kv-dtype fp8 \ --prefill-chunk 1024 \ --spec dflash2 \ --draft-tokens 5 \ --lm-head-draft \ --preserve-thinking \ --default-thinking-budget 2048 \ --vision

--max-context 262144 is the logical ceiling for one request.

--kv-capacity 262144 is a shared Main Text KV pool across concurrently admitted requests. It does not allocate 262144 KV tokens independently to each concurrent request.

Toolchain

• —NInfer: f76e19c0fbd026c86f46005acf2c80c54084bade
• —ModelOpt: 9e3d555aa16038f7b46a631ca603351146c09123
• —llmcompressor: 6693441a7db6984415d524f51ed16a1605bf2fec
• —compressed-tensors: 0.18.1.a20260914

Model lineage

Qwen/Qwen3.8-27B ↓ ukisai/Swift-Qwen3.8-27b ↓ d0xin/Swift-Qwen3.8-27B-Uncensored-BF16 ↓ Local-Hessian + Activation Headroom mixed NVFP4/FP8 ↓ this NInfer artifact

Limitations

Quantization can change individual generation trajectories even when aggregate benchmark accuracy is similar.

The MMLU-Pro result reported here uses a fixed 280-question subset, not the full benchmark. The refusal evaluation uses a fixed automated 100-prompt set. Performance results were measured on one RTX PRO 6000 Blackwell system and should not be interpreted as universal performance values.

License

This model is a derivative of ukisai/Swift-Qwen3.8-27b.

Swift-specific contributions are distributed under the Swift Open License v1.0. The underlying Qwen base model is distributed under Apache License 2.0.

See the included LICENSE, LICENSE-APACHE-2.0, and NOTICE files for applicable terms and attribution requirements.

Attribution

• —Base model: Qwen/Qwen3.8-27B
• —Swift fine-tune: ukisai/Swift-Qwen3.8-27b
• —Uncensored derivative, quantization, NInfer conversion, and validation: d0xin