yunfengwang/Qwen-Image-2.1-MNN-fp16

Qwen-Image-2.1-MNN-fp16

Qwen-Image-2.1 converted to MNN for on-device text-to-image inference — no torch, no diffusers, no 33 GB checkpoint. The text encoder and the DiT are fp16 weights, the VAE is fp32. Quantized alternatives: Qwen-Image-2.1-MNN-int8 (21.4 GB) and Qwen-Image-2.1-MNN-int4 (14.4 GB).

Run it with the qwen-image-mnn CLI (downloads ~31 GB into ~/.cache/qwen-image-mnn/fp16 on first use, then renders offline):

uvx qwen-image-mnn "a red panda riding a bicycle" --steps 40 --out panda.png

Files

The tokenizer, the flow-matching scheduler and the joint-sequence bookkeeping (3-axis RoPE, block-causal mask, modulation rows) are plain numpy in the CLI; only the transformer, the text encoder and the VAE run in MNN.

Verification

Each graph was checked against its source before publishing:

End to end (8 steps, 512 px, same latents): the MNN pipeline's latents drift from 4.9e-3 (step 1) to 2.9e-2 (step 8) relative to the released pipeline's own fp16 run, image PSNR 43.6 dB. The released model compared against itself across dtypes (fp16 vs bf16, same seed and latents) drifts up to 2.0e-1 — the conversion is closer to the model than the model is to itself once the dtype moves.

Speed

CPU, precision=high, 10 threads, warm session (minimum of repeated calls in one process — what a multi-step render actually pays): the text encoder takes 0.39 s for a 28-token prompt and one DiT step takes 10.1 s at 512 px (T=1039) or 58 s at 1024 px (T=4111). A full 40-step render at the official 1024 px has been clocked at 4817 s (~80 min) on a machine also carrying other work, but the same step measured 58 s warm later on the same busy machine, which puts that render nearer 40 min — per-step time tracks the machine's other load, so treat these as ballparks. A cold process pays MNN's dynamic-shape planning once on top: ~23 s to resize the DiT session, so the first step costs ~35 s instead of ~13 s of work, and ~20–30 s for the text encoder's first prompt instead of 0.4 s. The int8 and int4 sets are within noise of these numbers — quantization here is a size win, not a speed one.

Backend support

• —CPU, `precision=high`: the verified path. MNN's low precision swaps the CPU kernels to fp16 arithmetic, which overflows this model's residual stream.
• —Metal: broken in MNN 3.6.1 — its tensor-API probe fails to compile against the macOS 26 SDK (static_assert … At least one of M or N must be a multiple of 16) and the fallback kernels return garbage, exactly as seen with other MNN models on this OS. MNN master fixed the probe; a local master build renders the VAE correctly on Metal (max rel 1.5e-3) and 3.1x faster than CPU (2.3 s against 7.2 s for a 512 px decode) and the DiT forward is numerically correct too (max rel 2.2e-3 against CPU) — but there it is ~4x slower than CPU in the same probe, so CPU stays the fastest full path even once the fix ships. The pip wheel also cannot load a locally built libMNN.dylib (the process is killed on load), so this needs an upstream MNN release either way.

License

A format conversion of Qwen-Image-2.1 and carries its license: Qwen Research License — research use.