bahadirakdemir/Z-Image-Turbo-FP8-attn-qkv-only

Z-Image-Turbo FP8 — attn-qkv-only

Only the attention QKV projection is FP8; everything else stays in BF16.

★ The overall best practical choice in our benchmark is `online_fp8` (5.937 s/image, -24.2% vs BF16, peak GPU 13.34 GB — 38.4% less than BF16). It runs straight from the upstream `Tongyi-MAI/Z-Image-Turbo` checkpoint with --quantization-config '{"method":"fp8"}' — no separate download. The pre-quantized variants in this family are useful when you want deterministic static scales (calibrated offline, frozen on disk) or when online quantization is not desired.

This variant (attn-qkv-only) → mean latency 7.344 s/image (-6.2% vs BF16), peak GPU 20.21 GB (6.7% less than BF16's 21.66 GB), image quality PSNR 24.41 dB / SSIM 0.8423 vs the BF16 reference at identical (prompt, seed).

This is one of 7 FP8 quantization policies of `Tongyi-MAI/Z-Image-Turbo` prepared for vLLM-Omni serving. Each variant differs only in which transformer submodules are quantized to FP8 (E4M3 with per-tensor static scales); everything else stays in BF16.

In the tables below, ★ marks the overall best choice (online_fp8) and ← marks the row that corresponds to this repo.

Quick start (vLLM-Omni)

docker run --rm --gpus all --ipc=host \
  -p 18002:8002 \
  -e HF_HOME=/hf_cache \
  -e DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA \
  -e VLLM_ATTENTION_BACKEND=FLASH_ATTN \
  -v /your/hf/cache:/hf_cache \
  your-vllm-omni:latest \
    vllm-omni serve bahadirakdemir/Z-Image-Turbo-FP8-attn-qkv-only \
    --omni --host 0.0.0.0 --port 8002 \
    --gpu-memory-utilization 0.90 \
    --tensor-parallel-size 1 \
    --quantization-config '{"method":"modelopt","quant_method":"FP8","is_checkpoint_fp8_serialized":true,"kv_cache_quant_method":null,"exclude_modules":["model*","lm_head*","*feed_forward*","*to_out*"]}'

Then call the OpenAI-style image API:

curl -s http://127.0.0.1:18002/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "model": "bahadirakdemir/Z-Image-Turbo-FP8-attn-qkv-only",
    "prompt": "a cinematic photograph of an old fisherman standing on a foggy pier at sunrise",
    "size": "1024x512",
    "num_inference_steps": 8,
    "guidance_scale": 0.0,
    "n": 1,
    "response_format": "b64_json"
  }'

Benchmark — all 7 FP8 policies at 8 steps, 1024×512 on NVIDIA GB10

Same hardware, same (prompt, seed) tuples, same Docker image, fresh container per variant.

Pass A — DIFFUSION_ATTENTION_BACKEND=SAGE_ATTN

PSNR / SSIM are computed against the BF16 reference image at the same prompt and seed. Higher is better; BF16 is ref (it's the reference).

Pass B — top variants at DIFFUSION_ATTENTION_BACKEND=TORCH_SDPA

On NVIDIA GB10 (compute capability SM 12.1), FLASH_ATTN is platform-gated and silently falls back to TORCH_SDPA. SageAttention has a small but consistent overhead vs TORCH_SDPA on Blackwell, so the production recommendation is TORCH_SDPA.

Image comparisons

Same 3 prompts rendered across every variant with identical seeds. Click for full resolution.

Prompt 0 — "a cinematic photograph of an old fisherman standing on a foggy pier at sunrise"

[image]

Prompt 1 — "a detailed watercolor painting of a mountain village beside a clear blue lake"

[image]

Prompt 2 — "a futuristic city street at night with neon reflections on wet pavement"

[image]

Benchmark parameters

All 7 repos in this family

• —bahadirakdemir/Z-Image-Turbo-FP8-full
• —bahadirakdemir/Z-Image-Turbo-FP8-ffn-only
• —bahadirakdemir/Z-Image-Turbo-FP8-ffn-attn-out
• —bahadirakdemir/Z-Image-Turbo-FP8-ffn-attn-qkv
• —bahadirakdemir/Z-Image-Turbo-FP8-attn-only
• —bahadirakdemir/Z-Image-Turbo-FP8-attn-out-only
• —bahadirakdemir/Z-Image-Turbo-FP8-attn-qkv-only ← you are here

License & attribution

• —Apache 2.0, inherited from upstream Tongyi-MAI/Z-Image-Turbo.
• —Quantization: NVIDIA ModelOpt FP8 PTQ with 8 calibration prompts × 9 denoise steps each.
• —Serving: vLLM-Omni.
• —Benchmarked on NVIDIA GB10 (Grace Blackwell, SM 12.1), 2026-05-17.