veran666/zoria-realistic-vision-5.1-6bit-coreml

Realistic Vision 5.1 — Core ML, 6-bit, SPLITEINSUMV2

A Core ML conversion of SG161222/Realistic_Vision_V5.1_noVAE, built for on-device generation on iPhone.

Produced with Apple's ml-stable-diffusion (torch2coreml), MIT licensed, Copyright (c) 2024 Apple Inc.

What is in here

995 MB in total, against 2.1 GB for the same model at float16.

The U-Net is palettized to 6 bits and split into two chunks, which is what iOS needs — a single Core ML model that size will not load on a phone. The text encoder and the VAE decoder are left at float16 on purpose: palettizing them costs prompt fidelity and colour for about a seventh of the download.

Attention is SPLIT_EINSUM_V2, so the models are meant for the Neural Engine (MLComputeUnits.cpuAndNeuralEngine). Running them on the GPU instead is markedly slower.

Output resolution is 512×512.

Conversion

python -m python_coreml_stable_diffusion.torch2coreml \
  --model-version SG161222/Realistic_Vision_V5.1_noVAE \
  --convert-unet --convert-text-encoder --convert-vae-decoder \
  --attention-implementation SPLIT_EINSUM_V2 \
  --quantize-nbits 6 --chunk-unet \
  --bundle-resources-for-swift-cli \
  --compute-unit CPU_AND_NE --latent-h 64 --latent-w 64 \
  -o out

Two fixes to the upstream script were needed:

1. 1.quantize_weights quantizes unet, but with --chunk-unet it is unet_chunk1/unet_chunk2 that get bundled — and chunking runs before quantization. Left alone, the shipped chunks stay float16. The U-Net is quantized before it is cut.
2. 2.chunk_mlprogram.py calls .index() and integer subscripts on Block.operations, which is a CacheDoublyLinkedList in coremltools 8 rather than a list.

The text encoder is float16 rather than 6-bit for a second reason as well: CLIP stores its causal attention mask as a constant full of -inf, and k-means palettization fails on it.

Usage

Point Apple's StableDiffusionPipeline(resourcesAt:) at a directory holding these files, flat, and set computeUnits to .cpuAndNeuralEngine.

Licence

CreativeML Open RAIL-M, inherited from the source model. Conversion and quantization produce a derivative work, so the original licence governs this build unchanged — including the use-based restrictions in Attachment A, which travel with the model and with anything derived from it.

This licence is royalty-free and permits commercial use. It is not, however, unconditional: read Attachment A before shipping anything built on it.