Modelpublic

yunfengwang/Qwen-Image-2.1-MNN-int4

sourceHugging Faceotherupdated 2d agoView on Hugging Face

0likes

Model Card

Qwen-Image-2.1-MNN-int4

Qwen-Image-2.1 converted to MNN with weight-only int4 quantization — 14.4 GB, less than half the fp16 set — for on-device text-to-image inference with no torch, no diffusers and no 33 GB checkpoint. The text encoder and the DiT are int4 (block 32, HQQ), the VAE stays fp32.

Run it with the qwen-image-mnn CLI:

bash

uvx qwen-image-mnn "a red panda riding a bicycle" --quant int4 --steps 40 --out panda.png

Files

file	size	precision
`dit.mnn` + `dit.mnn.weight`	5.34 GB	int4
`text_encoder.mnn` + `text_encoder.mnn.weight`	7.71 GB	int4
`vae_decoder.mnn`	1.0 GB	fp32
`vae_encoder.mnn`	0.31 GB	fp32
`tokenizer.json`, `vae_config.json`	11 MB	—
`manifest.json`	—	per-file sha256

Verification

check	int4	int8	fp16
DiT, one forward at T=1120, max rel	6.3e-2	2.5e-2	baseline
text encoder, mean abs drift / max abs value	8.1e-3	4.8e-4	8.5e-5
pipeline, 8 steps @512 px, latent drift step 1 → step 8	1.3e-2 → 4.4e-1	1.4e-3 → 2.4e-2	4.9e-3 → 2.9e-2
pipeline, image PSNR vs the reference render	28.5 dB	52.0 dB	43.6 dB

The int4 end-to-end drift (4.4e-1) is twice the released model's own fp16-vs-bf16 noise (2.0e-1), so this is the first variant whose difference is measurable rather than theoretical. Isolating the two graphs (int8 text encoder + int4 DiT, and int4 text encoder + fp16 DiT) shows both contribute, with the DiT the larger share. Single-object prompts still come out clean; expect visible differences on detailed or text-heavy prompts.

Prefer int8 unless the ~7 GB saved is decisive: it lands inside the model's own noise at two-thirds of the fp16 download.

Speed

As with int8, the saving is disk and download, not step time: the stock CPU backend dequantizes weights as it pages them in, so both precisions run the same multiply-accumulates. Measured warm on the same machine (minimum of repeated calls in one process), one DiT step at 512 px (T=1039) takes 10.4 s at int4 against 10.1 s at fp16, and the text encoder is flat too (0.38 s against 0.39 s for a 28-token prompt). Size is what this set is for — 14.4 GB instead of 30.7 GB, at the quality cost shown above. First-call figures, the 40-step render at 1024 px and the per-process shape-planning overhead are in the package README.

Backend support

—CPU, `precision=high`: the verified path.
—Metal: broken in the current MNN release; fixed in master, see the fp16 model card.

License

A format conversion of Qwen-Image-2.1 and carries its license: Qwen Research License — research use.