esatapedico/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-GGUF

Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-GGUF

A family of six GGUF files of DavidAU/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1, DavidAU's "Cold Fusion" (GAIN + Unsloth) tune of Qwen3.8-27B: a 27B dense hybrid model (Gated DeltaNet + Gated Attention every fourth layer, 262K native context, embedded MTP speculative head, native vision tower). The conversion source is windowsxp811203/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4, their NVFP4 quantization of DavidAU's original. The MTP head is baked into every file; no separate drafter is needed (--spec-type draft-mtp).

My part here is only the numerics: I converted the NVFP4 checkpoint to GGUF and built a size/precision ladder for the tensors that most affect output quality and decode speed. All credit for the model itself belongs upstream (full chain below).

Follow along & support

I post updates on new conversions, benchmarks, and what I'm working on over on Ko-fi. Follow along there to keep up with new releases and the work in progress. If you'd like to support more of it, a coffee is always welcome. I do this on consumer hardware and like seeing how far it goes. More is on the way.

☕ ko-fi.com/esatapedico. Updates, work-in-progress, and an optional coffee.

The six files

Every tier carries the same byte-identical 448-tensor native NVFP4 backbone (per-tensor SHA-256 verified) and differs only in the precision of the LM head, token embedding, and MTP draft head:

Picking a tier: MID-HIGH is the highest-precision compact option (all three head groups at Q8_0) and our fastest compact decode on dual-GPU split; LOW/VERY-LOW trade some head precision for ~2 GB less VRAM; HIGH/VERY-HIGH restore BF16 heads where VRAM allows.

Tensor layout

The source NVFP4 checkpoint (windowsxp811203/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4) quantizes all Linear layers to true NVFP4 (group size 16, fp8-e4m3 scales) and deliberately keeps the vision tower, the Gated DeltaNet linear-attention path, lm_head, embeddings, and the MTP head in BF16.

ORIG parent (kept locally, 27.65 GB, not published): 1,122 tensors = 256 NVFP4 (MLP everywhere + attention projections on the 16 full-attention layers) + 240 BF16 (DeltaNet path) + BF16 heads/MTP/vision-side metadata + F32 norms/scales.

Compact tiers (published, 6 files): built from ORIG with llama-quantize --tensor-type-file, normalizing the whole language-model backbone to a uniform 448-tensor NVFP4 block (the tiny ssm_alpha vectors stay F32, matching our other Qwen3.8 families). The 256 source-NVFP4 tensors pass through untouched (byte-identical, SHA-256 verified against the converted parent); the DeltaNet path is re-typed from BF16 to NVFP4 once during tier building and is then frozen byte-identical across all tiers.

Vision

The Cold Fusion tune left the original Qwen3.8 vision tower untouched: we compared DavidAU's published mmproj-BF16.gguf tensor-by-tensor against the base model's projector and all 334 tensors are byte-identical. Pair any tier with his `mmproj-BF16.gguf` via --mmproj.

How this was made

1. 1.Converted windowsxp811203/Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4 (compressed-tensors NVFP4A16) to an ORIG parent GGUF with convert_hf_to_gguf.py --outtype auto.
2. 2.Generated per-tier tensor-type override maps (included below) from the parent inventory.
3. 3.Built each tier with llama-quantize --tensor-type-file <overrides> over the shared backbone.
4. 4.Verified: per-tier check_nvfp4 gate (448 NVFP4 tensors each), per-tensor SHA-256 backbone identity across all tiers, and passthrough integrity of the 256 source-NVFP4 tensors.

Rough sanity observations, not a benchmark (single runs, dual RTX 5070 Ti / 5060 Ti 16 GB, tensor split, llama-bench pp512/tg128):

Perplexity stays flat within ~2.6% across the whole ladder, so the quantization costs almost nothing even at the smallest tier; decode speed tracks head precision (smaller heads decode faster).

Serving check through LocalAI/llama.cpp on the same dual-16 GB box (fresh process per tier, 180k-token payload, single run each; decode samples are short because every tier stopped naturally, so treat decode as indicative only). This is a shared inference box, so runs can collide with other tenants:

Prefill is flat across the ladder (~640-650 tok/s at 180k context) and the embedded MTP head speculates at 0.80-0.93 acceptance everywhere.

SHA-256

bc67b5c698547ea38beaca6847b40b3952c8bc0891c7945fabdf30102b5cef60  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-VERY-LOW.gguf
c1aca6c2310d477e1cc053620dd1f1432833675a9a4785fac187ed26254244c1  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-LOW.gguf
2346b7639d6d66f61009e6b715b80e84d1088bbb8393b4e12a507a6fb4c9f0d4  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MEDIUM.gguf
77add63c9c551a00f7b74b1d6b347c34b122be420b38e61a73243c5e88e5254b  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-MID-HIGH.gguf
a32a2d161ebdfeb2178bcd5e610ec5db548aba420b6b4ace3c854633a3600262  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-HIGH.gguf
af68ca2711681076ddcd3087d0117f5cecb6a30192ff614f10cc353456f0a555  Qwen3.8-27B-Cold-Fusion-GAIN-V1.1-NVFP4-VERY-HIGH.gguf

Attribution & provenance

This is a derivative work built entirely from existing Apache-2.0 artifacts. Nothing here was trained or fine-tuned. Credit belongs to:

1. 1.Alibaba / Qwen team for the base model, Qwen/Qwen3.8-27B (Apache-2.0): 27B dense, 64 blocks, Gated DeltaNet + Gated Attention hybrid, native vision-language, 262,144-token context, MTP head.
2. 2.DavidAU for the tune itself, Qwen3.8-27B-Cold-Fusion-GAIN-V1.1 (Apache-2.0): the "Cold Fusion" (GAIN + Unsloth) training that raises general intelligence while cutting thinking tokens to a fraction of stock, and for publishing the GGUFs and discussions this work builds on.
3. 3.Unsloth, whose trainers/systems power the Cold Fusion method.
4. 4.windowsxp811203 for the NVFP4 checkpoint this family converts (...-NVFP4, Apache-2.0).
5. 5.This repo's author for the GGUF conversion and the tier ladder only.

Repository contents

• —Six tier GGUFs (table above)
• —overrides-{very-low,low,medium,mid-high,high,very-high}.txt: per-tensor quantization maps (1,122 entries each) for reproduction

Card written with AI assistance.