lite-infer/krea-2-turbo-nunchaku-lite-int4_r32-bnb4-text-encoder

Krea 2 Turbo Nunchaku Lite INT4 r32

Diffusers-loadable INT4 conversion of krea/Krea-2-Turbo, quantized with `diffuse-compressor`. It loads with a plain from_pretrained call — no runtime graph patches and no extra runtime package.

The transformer uses quant_method: nunchaku_lite, INT4 SVDQ with group size 64, rank 32, 256 SVDQ targets and no AWQ W4A16 targets. Eight outer linears (img_in, time/text embedders, time_mod_proj, text_fusion.projector, final_layer.linear) stay in bf16, and the text_encoder component is BitsAndBytes 4-bit NF4 with bf16 compute. QKV projections are not fused, so this trades some speed for loading through the stock Diffusers graph. Calibrated on 32 prompts at 8 steps, 1024x1024.

Benchmark

NVIDIA RTX PRO 4000 Blackwell, settings as in the Run snippet below, one warmup and three measured runs, everything resident on the GPU with no offload. Latency covers the full pipeline call; VRAM is peak device usage.

There is no dense bf16 row because it does not fit on a 24 GiB card — the transformer alone is 26.3 GiB — while both quantized builds run resident. The NVFP4 build is 1.78x faster on this GPU, because NVIDIA RTX PRO 4000 Blackwell has native FP4 tensor cores that only NVFP4 can use. INT4 is the build for Turing through Ada, where NVFP4 does not run at all, and has not been benchmarked there.

Output Comparison

[image]

Dense reference (left) and this build (right). Same prompt, seed, scheduler, resolution, and step count. Pixel MAE 16.01 / RMSE 34.05, versus 8.18 / 19.92 for the NVFP4 build.

Run

Requires the Hugging Face kernels package and DIFFUSERS_TRUST_REMOTE_KERNELS=true, plus a Diffusers build with the nunchaku_lite quantizer. INT4 needs a Turing or newer NVIDIA GPU; Hopper is unsupported.

import torch
from diffusers import Krea2Pipeline

pipe = Krea2Pipeline.from_pretrained(
    "lite-infer/krea-2-turbo-nunchaku-lite-int4_r32-bnb4-text-encoder",
    torch_dtype=torch.bfloat16,
).to("cuda")

image = pipe(
    prompt="a fox in the snow",
    generator=torch.Generator("cuda").manual_seed(12345),
    width=1024,
    height=1024,
    num_inference_steps=8,
    guidance_scale=0.0,
).images[0]
image.save("output.png")