sakesan12/LLaDA-Image-Turbo-q6_K-GGUF

LLaDA-Image-Turbo — q6_K GGUF

A q6_K quantization of the official LLaDA-Image-Turbo transformer, for the pig diffusion engine in ggk.

It exists because the 4-bit build that was already circulating is visibly coarse, while the BF16 original is 12.2 GB and does not fit a 6 GB card even with offloading. q6_K lands between them: 5.0 GB, near-BF16 fidelity, and it streams through a 6 GB GPU in graph-cut segments.

How it was made

The four official BF16 shards were merged into one safetensors and quantized with the engine's own quantizer:

ggk editor quantize -m merged-bf16.safetensors \
  -o LLaDA-image-turbo-q6_k.gguf --type q6_k

12,474 MB → 5,118 MB. Only tensors of rank 2 and above are quantized; the one-dimensional ones — norms, biases and the cap_pad_token / sigvq_pad_token / x_pad_token entries — keep their original precision, following the llama.cpp and stable-diffusion.cpp convention. That matches how the existing nvfp4 build is laid out.

The tensor set is unchanged from the original: 335 tensors, the same names, 36 separate to_q projections and no fused qkv. The engine identifies the model from the model.diffusion_model.sigvq_pad_token tensor name, so this file loads without a general.architecture stamp.

Usage

This is the transformer only. It needs the text encoder, adapter and VAE from gguf-org/llada-image-gguf and gguf-org/pig-clip.

ggk diffuser engine -- \
  --diffusion-model LLaDA-image-turbo-q6_k.gguf \
  --vae pig_flux2_vae_fp32-f16.gguf \
  --llm pig_clip-q8_0.gguf \
  --llm-adapter pig_llada_adapter-f16.gguf \
  -p "A cinematic photograph of a red fox standing in fresh snow, soft winter light" \
  --cfg-scale 1.00 --steps 4 -W 1024 -H 1024 --sampling-method euler \
  --diffusion-fa --vae-tiling --vae-tile-size 16x16 \
  --backend te=cpu --offload-to-cpu --max-vram 4 \
  -o out.png

The turbo variant is distilled for 4 steps at CFG 1.0 — raising either makes the picture worse, not better. Width and height must divide by 16, or by 32 when editing with --ref-image.

Fitting it on 6 GB

The three flags that matter on a small card, measured on an RTX 4050 Laptop (6 GB) at 1024x1024:

• —--max-vram 4 cuts the graph into segments so the 5 GB of weights stream through the card instead of having to sit on it. Without it the run dies trying to allocate 5,054 MB at once.
• —--vae-tiling --vae-tile-size 16x16 keeps VAE decode to a 416 MB buffer. The default 32x32 tile asks for 1.7 GB and runs out of memory.
• —--backend te=cpu keeps the text encoder in system RAM.

Sampling then takes about 32 s and decoding about 6 s, at roughly 3.6 GB of VRAM.

Prompting

One subject, its setting, then the lighting and the shot:

A cinematic photograph of a red fox standing in fresh snow, soft winter light, detailed fur, shallow depth of field

For editing, give an instruction rather than a description — "Turn it into a watercolor painting" — which preserves the source composition.

The model is strong on a single subject and its style. It is weak at binding colours across two subjects, at spatial arrangement, and at rendering readable text; a longer prompt does not fix those.

Credits

Base model © inclusionAI, Apache-2.0. This is an unofficial quantized derivative, not affiliated with inclusionAI. The pig engine and the companion text-encoder weights are by gguf-org.