Xiejiehang/ERNIE-Image-Turbo-MLX-Q4

ERNIE-Image-Turbo-MLX-Q4

ERNIE-Image-Turbo-MLX-Q4 is a local inference distribution of ERNIE-Image-Turbo optimized for Apple Silicon. This directory contains ready-to-run MLX inference assets and minimal inference scripts, aiming to help developers perform high-quality text-to-image verification, prototyping, and engineering integration on macOS with lower VRAM consumption.

Compared to the original BF16 weights, this version quantizes the most memory-intensive modules—Transformer, Text Encoder, and Prompt Enhancement—to 4-bit, while keeping the VAE decoder at high precision to achieve a stable balance between deployment efficiency and generation quality.

<p align="center"> <img src="https://ai-studio-static-online.cdn.bcebos.com/c089f215fd764a10a38084dc416bba16ba177272cc364f38a089bc966eb1b3b0" /> </p>

Key Features

• —Apple Silicon Friendly: Built on MLX, specifically optimized for the Unified Memory Architecture of Apple M-series chips.
• —Massive Weight Compression: Core weights are compressed from ~30.56 GB to ~8.21 GB, making it ideal for local experiments and integration.
• —Preserved Strengths: Retains the core capabilities of ERNIE-Image-Turbo, including complex instruction following, long-form text rendering, and structured image generation.
• —Out-of-the-Box: Includes demo.py, a local scripts/ inference implementation, and all necessary configuration files. Developers can verify the model immediately.

Model Components

This distribution includes the following core components:

Use Cases

• —Rapidly verify ERNIE-Image-Turbo's local inference capabilities on Apple Silicon.
• —Provide a lightweight model base for integrating text-to-image features into applications.
• —Serve as a baseline for further Core ML conversion, service deployment, or on-device experiments.

Requirements

• —macOS
• —Apple Silicon Chip
• —Python 3.11+

Install dependencies:

pip install -r requirements.txt

The requirements.txt includes core dependencies for local execution:

• —mlx
• —mlx-lm
• —transformers
• —Pillow
• —numpy
• —safetensors

Quick Start

1. Run the Built-in Demo

Execute the following command in the current directory:

python demo.py

By default, this will:

• —Load the Q4 model assets from the current directory.
• —Use the prompt: "A cinematic poster of a futuristic city at sunrise, ultra detailed, realistic lighting".
• —Run 5 denoising steps.
• —Save the output to output.png.

2. Python Integration

To integrate this model into your own scripts, you can directly use the provided scripts.pipeline:

from pathlib import Path
from scripts.pipeline import generate, load_pipeline

# Load the pipeline from the current directory
model_dir = Path(".")
pipeline = load_pipeline(model_dir)

# Generate an image
image = generate(
    pipeline,
    prompt="A cinematic poster of a futuristic city at sunrise, ultra detailed, realistic lighting",
    steps=5,
    seed=42,
    height=1024,
    width=1024,
)

image.save("output.png")
print("Saved to output.png")

3. Recommended Parameters

For the best balance between quality and speed, we recommend:

• —Resolution: 1024x1024
• —Inference Steps: 8
• —Seed: Specify as needed for reproducibility.

For portrait or landscape orientations, you can explore:

• —848x1264 / 1264x848
• —768x1376 / 1376x768
• —896x1200 / 1200x896

Directory Structure

Experimental Results

We conducted local inference verification on an Apple M3 device. The core results are as follows:

Notes:

• —Results were measured on an Apple M3 device with 24 GB Unified Memory.
• —Testing configuration: 1024x1024 resolution, 5 denoising steps.
• —Actual performance may vary based on device model, MLX version, prompt length, and step count.

Background

The base model, ERNIE-Image-Turbo, was released by the Baidu ERNIE-Image team and excels in:

• —Complex instruction following and multi-object relationships.
• —Long-form text rendering and layout-sensitive content.
• —Structured image generation (posters, comics, multi-panel layouts).
• —Diverse style coverage (realistic, design-oriented, stylized aesthetics).

This MLX Q4 version is intended as a lightweight deployment option for developers rather than a redefinition of the model's capabilities.

References

This project references and builds upon the following open-source works:

• —Baidu ERNIE-Image: For providing the original ERNIE-Image and ERNIE-Image-Turbo models and methods.
• —mlx-ernie-image: For providing critical references for weight organization and inference implementation on MLX.
• —mlx-lm: For providing MLX quantization and language model loading capabilities.
• —MLX: For providing an efficient array framework and neural network runtime on Apple Silicon.