rockerBOO/qwen3-vl-32b-h3-tail-nvfp4

Qwen3-VL-32B H3 Generation Tail — NVFP4

Decoder layers 50–63, the final norm, and the LM head of Qwen/Qwen3-VL-32B-Instruct, quantized to NVFP4 for use as a generation tail on top of MiniMax H3's conditioning text encoder.

This is not a standalone model. It has no token embeddings, no vision tower, and no layers 0–49. It is useless on its own.

Why this exists

MiniMax H3 ships its Qwen3-VL-32B text encoder truncated to the first 50 of 64 layers, with no final norm and no LM head — enough to condition the video model, not enough to generate text. If you want to use that same encoder as a prompt enhancer, you need the missing tail.

Loading a second complete 32B model just to generate a prompt is wasteful. This file is only the missing piece, so it can be loaded temporarily alongside the encoder you already have resident and unloaded afterwards.

Contents

156 tensors, ComfyUI naming:

The LM head is left at full precision on purpose. It is the projection to 151936 logits, so quantization error there lands directly in sampling. It is also the single largest tensor, which is why the file is bigger than a naive "4-bit" estimate.

Requirements

• —Base layers 0–49 from a MiniMax H3 Qwen3-VL-32B conditioning encoder. This tail supplies nothing else.
• —Blackwell GPU (SM ≥ 10.0/12.0) for NVFP4 inference.

Usage (ComfyUI)

Place in models/text_encoders/. With boo-textgen, connect BOO MiniMax H3 Generation Tail Loader → the clip_tail input of BOO Generate MiniMax H3 Prompt, with your normal 50-layer H3 CLIP on clip.

The tail is loaded for the duration of generation and unloaded immediately after.

Provenance

Extracted directly from Qwen/Qwen3-VL-32B-Instruct (shards 11–14 of 14) and remapped to ComfyUI key naming:

model.language_model.layers.{50..63}.*  ->  model.layers.{50..63}.*
model.language_model.norm.weight        ->  model.norm.weight
lm_head.weight                          ->  model.lm_head.weight

On matching MiniMax H3: MiniMax did not fine-tune the tail. model.norm.weight (complete tensor) and the leading 64 KiB of model.lm_head.weight are byte-identical between MiniMaxAI/MiniMax-H3 and stock Qwen3-VL-32B-Instruct. Note this was verified on the unquantized tensors only — the 14 decoder layers could not be compared directly, as the available MiniMax-derived tail was already quantized.

How it was quantized

Using convert_to_quant (branch feature/layer-config-convrot-plus-int4):

ctq -i tail_quant_src.safetensors -o tail_nvfp4.safetensors \
    --nvfp4 --comfy_quant --save-quant-metadata --full_precision_matrix_mult

The LM head was held out of this step and merged back afterwards at bf16, because ctq quantizes every 2D *.weight it is given.

Default learned-rounding (AdaRound/Prodigy) was used rather than --simple. No AWQ smoothing was applied — see below.

File

Roughly 3.6 GiB of NVFP4 layers plus a 1.5 GiB bf16 LM head.

Verification

• —All 156 source tensors confirmed finite before quantization.
• —All 98 FP8 E4M3 block-scale tensors in the output confirmed finite — zero NaN, and zero non-finite values anywhere in the file.
• —Reconstruction error, dequantizing through ComfyUI's own TensorCoreNVFP4Layout and comparing against the bf16 source:

~9.5% relative error is the expected band for 4-bit E2M1 with per-16 block scales (about 20 dB SNR). The tight spread across projection types and across layers 50, 56 and 63 indicates no tensor degenerated.

Why no AWQ

An existing NVFP4 AWQ build of this same tail is defective: 46 of its `float8_e4m3` block scales are NaN, spread across 13 tensors and exclusively in o_proj and down_proj — precisely the two projections carrying AWQ pre_quant_scale. Its finite scales saturate at 448.0, the maximum finite value of E4M3. AWQ smoothing inflates weight magnitudes; when a block's required scale exceeded 448 the exporter emitted NaN instead of clamping. Because model.layers.50 is affected, NaN enters on the very first token and every logit becomes NaN, tripping a CUDA device-side assert in multinomial that aborts the whole process.

ComfyUI treats pre_quant_scale as optional (weight_scale and weight_scale_2 are the only mandatory NVFP4 tensors), so AWQ was skipped here entirely. That gives up some accuracy relative to a correct AWQ build, in exchange for removing the mechanism that produced the failure.

If you are evaluating any NVFP4 quant, it is worth checking:

import torch
from safetensors import safe_open

with safe_open("model.safetensors", framework="pt", device="cpu") as f:
    for k in f.keys():
        t = f.get_tensor(k)
        if t.dtype == torch.float8_e4m3fn and torch.isnan(t.float()).any():
            print("NaN block scales in", k)

License

Apache 2.0, inherited from Qwen3-VL-32B-Instruct. See LICENSE and NOTICE.