WaveCut/HiDream-O1-Image-Dev-SDNQ-uint4-svd-r32-downproj-bf16

HiDream-O1-Image-Dev SDNQ - Static UINT4 + SVD r32, down_proj BF16

Smallest root-cause fix. Only MLP down projections are kept in BF16 beyond the standard output/embed skips; this isolates down_proj as the main grid culprit.

This repository is part of the fixed SDNQ 4-bit HiDream O1 quantization set. The previous broad 4-bit recipes produced a visible tiled/grid artifact. The fix keeps the sensitive decoder projection path in higher precision, especially model.language_model.layers.*.mlp.down_proj.weight.

Comparison

Benchmarks were run on an NVIDIA RTX PRO 6000 Blackwell Workstation Edition with the HiDream O1 repository inference path, BF16 runtime, 28 flash-scheduler steps, and flash attention disabled for parity. The requested 1024x1024 size is snapped by O1 to 2048x2048.

Variant Strengths

• —[Dynamic UINT4 threshold 1e-2, fixed](https://huggingface.co/WaveCut/HiDream-O1-Image-Dev-SDNQ-4bit-dynamic-uint4-th1e-2): Fastest balanced fixed variant. Dynamic quantization keeps the known artifact-producing down/output projections unquantized and raises difficult layers automatically.
• —[Static UINT4 + SVD r32, o/down BF16 guard](https://huggingface.co/WaveCut/HiDream-O1-Image-Dev-SDNQ-uint4-svd-r32): Conservative default. Keeps both attention output and MLP down projections in BF16, the visually accepted fix for the tiled-grid artifact.
• —Static UINT4 + SVD r32, down_proj BF16: Smallest root-cause fix. Only MLP down projections are kept in BF16 beyond the standard output/embed skips; this isolates down_proj as the main grid culprit.
• —[Static UINT4 + SVD r32, last 8 o/down BF16](https://huggingface.co/WaveCut/HiDream-O1-Image-Dev-SDNQ-uint4-svd-r32-last8-odown-bf16): Most memory-efficient clean-looking compromise from the matrix. It protects only the last 8 decoder layers' o/down projections.
• —[Static UINT4 + SVD r32, last 16 o/down BF16](https://huggingface.co/WaveCut/HiDream-O1-Image-Dev-SDNQ-uint4-svd-r32-last16-odown-bf16): Safer memory-efficient compromise. It protects the last 16 decoder layers' o/down projections and keeps much lower storage than the full o/down guard.

This Variant

• —Source model: HiDream-ai/HiDream-O1-Image-Dev
• —Source snapshot: 833d408a57a7c1e399757c7f2f174670726fd43c
• —Recipe: r32_skip_down_proj_bf16
• —SDNQ layer counts: {"uint4": 332}
• —Quantized parameter counts: {"uint4": 5705330688}
• —Benchmark average generation time: 9.24s
• —Benchmark peak allocated VRAM: 9.66 GiB
• —Saved parameter storage: 8.92 GiB
• —10-demo average generation time: 9.15s
• —10-demo peak allocated VRAM: 19.44 GiB

Demo Comparisons

Each image in comparison/ is a pairwise original BF16 output next to this quantized variant with the same prompt, seed, and sampler settings.


Usage

pip install sdnq torch transformers diffusers accelerate einops pillow scipy torchvision
git clone https://github.com/HiDream-ai/HiDream-O1-Image
cd HiDream-O1-Image

import torch
import sdnq
from transformers import AutoProcessor
from models.qwen3_vl_transformers import Qwen3VLForConditionalGeneration

model_id = "WaveCut/HiDream-O1-Image-Dev-SDNQ-uint4-svd-r32-downproj-bf16"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    device_map="cuda",
).eval()

Files

• —quantization_config.json - saved SDNQ config.
• —quantization_summary.json - quantized layer/parameter/storage summary.
• —benchmark_summary.json - matrix metrics plus 10-demo generation metrics.
• —comparison/00.jpg ... comparison/09.jpg - pairwise original vs quantized comparisons.
• —comparison/contact_sheet.jpg - compact overview of all 10 comparisons.