Modelpublic

InsecureErasure/Krea2-Turbo-mixed-NVFP4

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

1likes255downloads

Model Card

Krea2-Turbo-mixed-NVFP4

Mixed 8-bit microscaling quantization of Krea-2-Turbo, generated with `convert_to_quant`.

—Format: Mixed NFVP4 and MXFP8 with some layers preserved as BF16.
—Size: 8.8G GB (−66% vs BF16).
—Inference: ComfyUI + `comfy-kitchen`, Blackwell GPU (RTX 50xx / B100 / B200).

Quantization

The model weights have been partially quantized to NVFP4 (NVIDIA Floating Point 4-bit) and MXFP8, quantization formats supported on NVIDIA Blackwell architecture GPUs.

The following convert_to_quant parameters where used. This conversion takes about 1 hour on an RTX 5060 TI.

bash

$ convert_to_quant -i krea2_turbo_bf16.safetensors \
    --nvfp4 \
    --krea2 \
    --comfy_quant \
    --save-quant-metadata \
    --custom-type mxfp8 \
    --custom-layers \
      "blocks\.(0|1|2|24|25|26)\.attn\.(wq|wk|wv|wo)\.weight|blocks\.(0|1|2|25|26|27)\.attn\.gate\.weight|blocks\.(0|1|2|3|25|26|27)\.mlp\.gate\.weight|txtfusion\.layerwise_blocks\.(0|1)\.attn\.(wq|wk|wv|wo|gate)\.weight|txtfusion\.layerwise_blocks\.(0|1)\.mlp\.gate\.weight|txtfusion\.refiner_blocks\.(0|1)\.attn\.(wq|wk|wv|gate)\.weight|txtfusion\.refiner_blocks\.(0|1)\.mlp\.(gate|up)\.weight|txtfusion\.refiner_blocks\.0\.mlp\.down\.weight" \
    --exclude-layers \
      "blocks\.27\.attn\.(wq|wk|wv)\.weight|txtfusion\.refiner_blocks\.(0|1)\.attn\.wo\.weight|txtfusion\.refiner_blocks\.1\.mlp\.down\.weight" \
    --num-iter 4000 \
    --top-p 0.35 \
    --calib-samples 8192 \
    --scale-optimization iterative \
    --scale-refinement 2 \
    --extract-lora \
    --lora-rank 64 \
    --lora-target "attn\.(wo|gate)\.weight|mlp\.(gate|down)\.weight" \
    -o krea2_turbo_mixed_nvfp4.safetensors