lribeiro/Qwen3.8-27B-nvfp4-v6

Qwen3.8-27B-NVFP4-GPTQ-v6

NVFP4 (W4A4) + FP8 (W8A8) mixed-precision, GPTQ-quantized variant of Qwen/Qwen3.8-27B, produced with llm-compressor and saved in compressed-tensors format for direct use with vLLM.

This is experiment v6 in a 38-run quantization sweep. The first 16 MLP layers use NVFP4 4-bit weights + 4-bit activations with GPTQ Hessian error correction; the remaining 48 MLP layers and all attention layers use FP8 E4M3 (W8A8). v6 is the Pareto-optimal config in the NVFP4 W4A4 line — the best fidelity achievable with native FP4 tensor cores before KLD degrades below the Acceptable tier.

The best NVFP4 W4A4 config, but not the best model overall. v17+ (all-FP8 W8A8) achieves lower KLD at comparable speed and strictly dominates the entire NVFP4 W4A4 line. v6 is the recommended choice only when 4-bit NVFP4 tensor-core prefill throughput is required and KLD must stay within the Acceptable band.

Model Details

Qwen3.8-27B (qwen3_5_text):

• —Type: Causal language model (hybrid linear + full attention)
• —Parameters: 27B
• —Layers: 64 — 48 linear-attention (GatedDeltaNet) + 16 full-attention (every 4th layer: 0, 4, 8, …, 60)
• —Hidden size: 5,120
• —Intermediate size: 17,408
• —Attention heads: 24 × 256 dim
• —Vocabulary: 248,320
• —Activation: SiLU
• —Context: 8,192 tokens (benchmark configuration)

Each linear-attention layer has in_proj_qkv, in_proj_z, out_proj plus recurrent state paths in_proj_a/in_proj_b. Each full-attention layer has standard q/k/v/o_proj. All layers share gate_proj/up_proj/down_proj MLP blocks.

Quantization Configuration

Precision Assignment

GPTQ: Hessian-based weight correction, sequential targets Qwen3_5DecoderLayer, actorder=static, default dampening_frac (no override).

Modules Kept in BF16

Calibration

• —Source: malaiwah/qwen38-27b-fidelity-suite-v3 tokens (181 contexts × 2048 tokens)
• —Samples: 181
• —Max sequence length: 2048
• —Format: Decoded token IDs from suite, not raw text

Usage

# vLLM serve (requires Blackwell SM120 + FlashInfer)
vllm serve /models/Qwen3.8-27B-nvfp4-gptq-v6 \
  --max-model-len 8192 \
  --max-num-seqs 512 \
  --quantization compressed-tensors

`--max-num-seqs` matters on this architecture. 48 of 64 layers use linear attention, and vLLM allocates one Mamba-style cache block per decode sequence. The default max_num_seqs=1024 can exceed available blocks and fail during CUDA graph capture with max_num_seqs (1024) exceeds available Mamba cache blocks. Lower to 512 or raise --gpu-memory-utilization. This is a property of the base model, not of quantization.

from vllm import LLM, SamplingParams

llm = LLM(model="/models/Qwen3.8-27B-nvfp4-gptq-v6",
          quantization="compressed-tensors",
          max_model_len=8192)
out = llm.generate(
    ["Explain 4-bit quantization in two sentences."],
    SamplingParams(temperature=0.7, max_tokens=256),
)
print(out[0].outputs[0].text)

Evaluation

Distribution Fidelity (KLD)

Hidden-state capture after final RMSNorm → shared BF16 LM-head replay → full-vocabulary KL(BF16_ref ‖ candidate), two-pass log-sum-exp normalization. Body-only comparison (candidate LM head unused). Suite: 136 contexts × 2048 tokens, 278,392 scored positions over 248,320-token vocabulary.

KLD by Domain

Literary text is the dominant KLD contributor — 4.9× the scientific stratum. 4-bit quantization disproportionately affects creative/prose generation.

Throughput

Single NVIDIA RTX PRO 6000 Blackwell (SM120, 96 GB), vLLM 0.26.1rc0, FlashInfer backend. Duration-based sustained decode, 30s cells, temperature 0.0, ignore_eos=true.

Composite Score

$$\text{score} = \frac{\text{prefill}{2k} \times \text{decode}{c1}}{\text{KLD} \times 10^6} = \frac{7756 \times 52.3}{0.01597 \times 10^6} = \mathbf{25.42}$$

Context Within Experiment Series

v6 is the second data point in the NVFP4 W4A4 mixed-precision sweep (Phase 1), testing how many early MLP layers can use 4-bit NVFP4 before KLD degrades unacceptably. KLD scales roughly linearly with NVFP4 layer count — each additional 8-layer block adds ~0.004–0.005 KLD.

v6 is the Pareto-optimal NVFP4 W4A4 config — the only point in the sweep that stays within the Acceptable KLD tier (< 0.031) while still using native FP4 tensor cores for prefill. Later phases (v17+: all-FP8 W8A8) superseded this entire line — FP8 achieves KLD 0.0123 at comparable speed, strictly dominating NVFP4 W4A4 for this model.

Provenance & Integrity

14 model shards individually hashed in reports/report-nvfp4-gptq-v6.json → candidate_identity.shard_sha256. Full fidelity report, benchmark JSON, and hidden-state captures archived in the experiment workspace.

Hardware & Runtime

Reproduction

# Inside Docker (vLLM infernal-invocation image, FlashInfer, SM120)
CUDA_VISIBLE_DEVICES=0 python /data/scripts/quantize_nvfp4_gptq_v6.py \
  --model /data/models/Qwen3.8-27B-bf16 \
  --output /data/models/Qwen3.8-27B-nvfp4-gptq-v6 \
  --calib-samples 181 \
  --calib-max-len 2048

Calibration tokens from $SUITE_DIR/tokens (default /data/suite-v3/tokens). Script: scripts/quantize_nvfp4_gptq_v6.py. Log: quant_nvfp4_gptq_v6.log.

Limitations

• —Not the best config. v17 (all-FP8 W8A8) achieves 23% lower KLD (0.0123 vs 0.0160) at comparable speed and strictly dominates. Use v6 only when native FP4 tensor-core prefill is required and KLD must stay within the Acceptable band.
• —Blackwell-only. NVFP4 W4A4 requires SM120 FP4 tensor cores via FlashInfer CUTLASS. No fallback path for non-Blackwell GPUs.
• —NVFP4 prefill only for MLP. FlashInfer native FP4 MMA is used for MLP GEMMs only; attention layers run FP8 MMA. No native FP4 path for attention.
• —Body-only KLD. Fidelity measured against a shared BF16 LM head; end-to-end generation quality may differ slightly.
• —Literary domain degradation. 4.9× higher KLD on literary text vs scientific — creative/prose generation is disproportionately affected by 4-bit quantization.
• —Quantization is lossy. Validate on your own workload before production use.