Modelpublic

ukisai/Swift-Qwen3.8-27B-NVFP4

sourceHugging Faceotherupdated 6d agoView on Hugging Face

39likes6.1kdownloads

Model Card

<div align="center"> <a href="https://ukisai.com"><img src="ukisai-banner.png" alt="UkisAI" style="width:100%;max-width:100%;height:auto;display:block;margin-bottom:0.6em;" /></a> <div style="display:flex;justify-content:center;gap:0.6em;margin-bottom:1em;"> <a href="https://ukisai.com"><strong>Website</strong></a>  •  <a href="https://ukisai.com/products/swift"><strong>Learn more</strong></a>  •  <a href="https://huggingface.co/ukisai/Swift-Qwen3.8-27b"><strong>BF16 model</strong></a>  •  <a href="https://huggingface.co/ukisai/Swift-Qwen3.8-27B-GGUF"><strong>GGUF</strong></a>  •  <a href="#license-and-access"><strong>Enterprise licensing</strong></a> </div> </div>

Swift-Qwen3.8-27B-NVFP4

NVFP4/FP8 mixed-precision quantization of Swift-Qwen3.8-27B, UkisAI's reasoning-efficient derivative of Qwen3.8-27B. This update was created directly from the original BF16 source weights using NVIDIA Model Optimizer, following the verified precision mapping of NVIDIA's Qwen3.8-27B-NVFP4. The model architecture, vision components, tokenizer, and BF16 MTP weights are preserved.

@media (max-width: 640px) { .swift-table { display:block !important; width:100% !important; max-width:100%; overflow-x:auto !important; -webkit-overflow-scrolling:touch; table-layout:auto !important; } .swift-table th, .swift-table td { min-width:100px; } .swift-table th:first-child, .swift-table td:first-child { min-width:160px; } } </style>

Weight-memory fix

The previous checkpoint retained the 144 large projections across 48 Gated DeltaNet layers and lm_head in BF16. This update reduces model-weight memory, leaving more memory for the existing KV cache. KV-cache precision is unchanged and the KV cache was not quantized.

What is in this checkpoint

Component	Precision
192 MLP projections (`gate_proj`, `up_proj`, `down_proj`) and `lm_head`	NVFP4, block size 16
144 large DeltaNet projections (`in_proj_qkv`, `in_proj_z`, `out_proj`)	FP8 E4M3
64 full-attention projections (`q_proj`, `k_proj`, `v_proj`, `o_proj`)	FP8 E4M3
Embeddings, norms, small DeltaNet projections/controls, convolutions, vision, MTP	BF16
Format	NVIDIA ModelOpt mixed precision
Checkpoint size	Approximately 21.945 GB

All 798 retained BF16 tensors, including all 15 MTP tensors, were checked byte for byte against the original source. The precision inventory lists every exported tensor.

Calibration

Source: ukisai/Swift-Qwen3.8-27b at revision 1b30aaaf753fe5c1cb51ada2ea0367a53445359c. ModelOpt commit: 022767c7ab3d7d36211affd85e5c496770cde768 (installed version 0.47.0rc0). Calibration used 2,048 examples from NVIDIA's seven-source Nemotron post-training v3 collection, a 4,096-token maximum, seed 42, and 6,459,330 actual tokens. The original source model was split across both H100s in one process. Both full calibration passes completed and all 193 NVFP4 modules had Local-Hessian coverage.

NVFP4 uses ModelOpt Local-Hessian calibration with block size 16 and the full FP8 scale sweep. FP8 projections use max calibration. Quantizers are enabled only for the explicit projection list; KV-cache quantizers remain disabled.

See the exact recipe, calibration manifest, and package versions. This reproduces NVIDIA's published approach with documented representative data; NVIDIA's exact calibration sample IDs and sampling parameters are not public.

Requirements and hardware validation

Validated with vLLM 0.28.0, using a loader that supports ModelOpt mixed-precision checkpoints. This update changes the checkpoint serialization from the previous compressed-tensors format to ModelOpt.

H100 inference uses Marlin weight-only NVFP4 kernels. Native Blackwell W4A4 execution has not been tested for this new checkpoint. The reported memory and quality results are H100 measurements. FP4/FP8 activation quantization was also exercised separately through ModelOpt fake-quantized likelihood evaluation.

Evaluation scope

These results compare the previous NVFP4 checkpoint with this release on H100.

Benchmarks

Fresh paired tests used both H100s, with one H100 per inference measurement and identical settings for old/new models. Full budget means gpu_memory_utilization=0.90; constrained budget means 0.40. Both use BF16 model dtype and kv_cache_dtype=auto.

<table class="swift-table"> <thead><tr> <th class="benchmark-heading" style="width:50%;text-align:left;padding-left:18px;">Metric</th> <th class="score-heading" style="width:25%;">Old NVFP4</th> <th class="tokens-heading" style="width:25%;">New NVFP4</th> </tr></thead> <tbody> <tr><td class="benchmark">Checkpoint size</td><td>28.572 GB</td><td class="swift"><strong>21.945 GB</strong></td></tr> <tr><td class="benchmark">Model weight VRAM</td><td>26.15 GiB</td><td class="swift"><strong>20.00 GiB</strong></td></tr> <tr><td class="benchmark">Free VRAM after loading</td><td>52.06 GiB</td><td class="swift"><strong>58.27 GiB</strong></td></tr> <tr><td class="benchmark">Cache memory — 40% budget</td><td>3.08 GiB</td><td class="swift"><strong>9.29 GiB</strong></td></tr> <tr><td class="benchmark">Available cache tokens — 40% budget</td><td>47,824</td><td class="swift"><strong>149,744</strong></td></tr> <tr><td class="benchmark">Tested context — 40% budget</td><td>47,040</td><td class="swift"><strong>148,960</strong></td></tr> <tr><td class="benchmark">Available cache tokens — 90% budget</td><td>691,036</td><td class="swift"><strong>791,861</strong></td></tr> <tr><td class="benchmark">Tested context — 90% budget</td><td>262,144</td><td class="swift"><strong>262,144</strong></td></tr> <tr><td class="benchmark">GSM8K quality sanity test</td><td>124/128</td><td class="swift"><strong>124/128</strong></td></tr> <tr><td class="benchmark">MTP arithmetic</td><td>32/32</td><td class="swift"><strong>32/32</strong></td></tr> <tr><td class="benchmark">KV cache quantized?</td><td>No</td><td class="swift"><strong>No</strong></td></tr> </tbody> </table>

The 3.17x constrained-context increase comes from lower weight memory. Cache-entry specifications are identical. Full-budget tests reach the unchanged 262,144-token architecture ceiling. Long-context checks retrieved three independent values placed at 10%, 50%, and 90% of the prompt and completed a real 256-token generation at the capacity boundary. Strict loading, numeric precision checks, MTP and synthetic image tests passed. These are sanity checks, not a full production benchmark suite or a claim of improved inference speed.

See validation details. The previous checkpoint and its original Blackwell benchmark remain available at their pinned revision; those Blackwell results describe the old weights.

How to use

vLLM

bash

vllm serve ukisai/Swift-Qwen3.8-27B-NVFP4 \
  --tensor-parallel-size 1 \
  --dtype bfloat16 \
  --kv-cache-dtype auto \
  --max-model-len 262144 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --port 8000

Adjust tensor parallelism and context length to your GPU memory. See the base model's vLLM recipe for installation and hardware-specific settings.

Optional MTP decoding

The checkpoint includes the base model's MTP head in BF16. To enable self-speculative decoding, append to the server command above:

bash

--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

UkisAI API

Swift is also served through an OpenAI-compatible API at https://ukisai.com/api/swift/v1. It is free for research purposes and needs no API key. The model id is swift.

License and access

Swift weights are distributed through gated access under the Swift Open License v1.0. Personal, research, educational, evaluation, and commercial use are free for individuals and organizations with annual recurring revenue, including affiliates, of up to US$1,000,000. Above that threshold, commercial use requires a separate Swift Enterprise License. Contact UkisAI for terms.

Citation

bibtex

@misc{swift-qwen3.8-27b,
  title  = {Swift-Qwen3.8-27B},
  author = {UkisAI},
  year   = {2026},
  url    = {https://huggingface.co/ukisai/Swift-Qwen3.8-27b}
}

Acknowledgements

We acknowledge the NVIDIA Innovation Lab for providing access to 8× NVIDIA H100 GPUs to train Swift.