lite-infer/krea-2-turbo-nunchaku-lite-nvfp4_r32-bnb4-text-encoder

Krea 2 Turbo Nunchaku Lite NVFP4 r32

Diffusers-loadable NVFP4 conversion of krea/Krea-2-Turbo, quantized with `diffuse-compressor`. It loads with a plain from_pretrained call — no runtime graph patches and no extra runtime package.

The transformer uses quant_method: nunchaku_lite, NVFP4 SVDQ with group size 16, rank 32, 256 SVDQ targets and no AWQ W4A16 targets. Eight outer linears (img_in, time/text embedders, time_mod_proj, text_fusion.projector, final_layer.linear) stay in bf16, and the text_encoder component is BitsAndBytes 4-bit NF4 with bf16 compute. QKV projections are not fused, so this trades some speed for loading through the stock Diffusers graph. Calibrated on 32 prompts at 8 steps, 1024x1024.

Benchmark

NVIDIA RTX PRO 4000 Blackwell, settings as in the Run snippet below, one warmup and three measured runs, everything resident on the GPU with no offload. Latency covers the full pipeline call; VRAM is peak device usage.

There is no dense bf16 row because it does not fit on a 24 GiB card — the transformer alone is 26.3 GiB — while both quantized builds run resident. It is 1.78x faster than the INT4 build here.

Output Comparison

[image]

Dense reference (left) and this build (right). Same prompt, seed, scheduler, resolution, and step count. Pixel MAE 8.18 / RMSE 19.92, versus 16.01 / 34.05 for the INT4 build.

Run

Requires the Hugging Face kernels package and DIFFUSERS_TRUST_REMOTE_KERNELS=true, plus a Diffusers build with the nunchaku_lite quantizer. NVFP4 needs a Blackwell or newer NVIDIA GPU; Hopper is unsupported. On Turing through Ada use the INT4 build instead.

import torch
from diffusers import Krea2Pipeline

pipe = Krea2Pipeline.from_pretrained(
    "lite-infer/krea-2-turbo-nunchaku-lite-nvfp4_r32-bnb4-text-encoder",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt="a fox in the snow",
    generator=torch.Generator("cuda").manual_seed(12345),
    width=1024,
    height=1024,
    num_inference_steps=8,
    guidance_scale=0.0,
).images[0]
image.save("output.png")