Modelpublic

windowsxp811203/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

2likes215downloads

Model Card

Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 — NVFP4

NVFP4 quantization of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 — all credit for the tune itself (the "Cold Fusion" / GAIN training) goes to DavidAU. This repo only changes the numerics: 28.55 GB, built for Blackwell (sm120) + vLLM.

Made on request (discussion). Same pipeline as Qwen3.8-27B-Abliterated-NVFP4.

What is and isn't quantized

Correction (2026-08-23, numbers revised 08-24). An earlier version of this card said that most NVFP4 conversions of this architecture ship without a usable draft head, and presented the bf16 vision tower as something these conversions usually lose. Both were wrong and I withdraw them. I surveyed every NVFP4 build of Qwen3.8-27B and its finetunes on the Hub (136 repos; 91 independent auditable artifacts after collapsing true re-uploads): 78 of 91 ship the complete 15-tensor head with 5 quantizing it, and 84 of 87 full multimodal artifacts leave the vision tower unquantized. I had generalized from my own first broken build without checking the population. Full method, revision history and raw data: nvfp4-mtp-survey.

—The MTP draft head is intact and verified working. The 15 mtp.* tensors are kept in bf16 and listed in quantization_config.ignore. Both halves are needed: without the ignore entry vLLM treats the bf16 head as a quantization target and rejects every draft (0% acceptance, with clean-looking logs). That failure mode is real but uncommon in published repos. This build uses the tune's own MTP tensors — DavidAU's checkpoint ships them, which not every finetune does.
—Vision tower kept in bf16 (all 333 visual.* tensors) — image input works. This is standard practice for this model, not a distinguishing feature of this build.
—*Gated DeltaNet path (`linear_attn.) and lmhead` kept in bf16.** This is where these builds genuinely differ from most: 15 of the 91 surveyed artifacts leave every `linearattn` projection unquantized, 20 quantize all 240 projections, and the partial builds span 20–97 % coverage. Keeping all of it costs file size and buys leaving a path several vendors treat as quantization-sensitive untouched.

Recipe: llm-compressor 0.13.0, NVFP4A16, targets Linear, ignore [lm_head, mtp.*, visual.*, linear_attn.*].

Measured on this exact checkpoint (vLLM 0.27.1, 1× RTX PRO 6000 Blackwell)

check	result
MMLU (400-sample)	79.5%
MTP draft acceptance	97.6%, mean acceptance length 1.98 (temp 0, short-form; expect lower under high-temperature long-form)
Vision probe	correct scene description
Thinking behavior	inherits the tune's heavily shortened thinking — reasoning is compact and may appear inline; `chat_template_kwargs: {"enable_thinking": false}` works as usual

Run it

bash

vllm serve windowsxp811203/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4 \
  --served-model-name cf-nvfp4 \
  --max-model-len 32768 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}'

Notes:

—NVFP4 needs Blackwell (compute capability 10.0+ — B100/B200 included; this build was tested on sm120). On pre-Blackwell cards use DavidAU's GGUF builds instead.
—Weights are ~28.6 GB; a 32 GB card runs it with moderate context, larger cards let you raise --max-model-len (native 262K).
—Image input: standard OpenAI-style multimodal messages (content array with image_url parts).

License

apache-2.0, inherited from the base tune. Please also see DavidAU's original card for the tune's own documentation and benches.