CoolFace
Modelpublic

danhtran2mind/Ghibli-Stable-Diffusion-2.1-Base-finetuning-INT8

sourceHugging Facemitupdated 1y agoView on Hugging Face
0likes
Model Card

<div align="center"> <h1> Ghibli Stable-Diffusion-2.1 Base finetuning Quantization INT8 </h1> <a href="https://github.com/your-repo/releases/tag/v1.0.0"> <img src="https://img.shields.io/badge/version-1.0.0-blue.svg" alt="Version 1.0.0"> </a> <a href="https://opensource.org/licenses/MIT"> <img src="https://img.shields.io/badge/license-MIT-green.svg" alt="License MIT"> </a> <a href="https://www.python.org"> <img src="https://img.shields.io/badge/python-3.8%2B-blue.svg?logo=python" alt="Python 3.8+"> </a> <a href="https://pytorch.org"> <img src="https://img.shields.io/badge/PyTorch-2.0%2B-orange.svg?logo=pytorch" alt="PyTorch 2.0+"> </a> <a href="https://huggingface.co/docs/diffusers"> <img src="https://img.shields.io/badge/diffusers-0.20%2B-red.svg?logo=huggingface" alt="Diffusers 0.20+"> </a> <a href="https://www.intel.com/content/www/us/en/developer/tools/openvino-toolkit/overview.html"> <img src="https://img.shields.io/badge/OpenVINO-2023.0%2B-blue.svg?logo=intel" alt="OpenVINO 2023.0+"> </a> </div>

Quantizate from Base Model

Install Dependencies

bash
pip install -q "optimum-intel[openvino,diffusers]" torch transformers diffusers openvino nncf optimum-quanto

Import Libraries

python
from diffusers import StableDiffusionPipeline, AutoencoderKL, UNet2DConditionModel, PNDMScheduler
from transformers import AutoTokenizer, CLIPTextModel, CLIPTokenizer
from optimum.intel import OVStableDiffusionPipeline
from optimum.intel import OVQuantizer, OVConfig, OVWeightQuantizationConfig
import torch
from nncf import CompressWeightsMode
import os

Load Base Model

python
model_id = "danhtran2mind/Ghibli-Stable-Diffusion-2.1-Base-finetuning"
device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.float16 if torch.cuda.is_available() else torch.float32

# Load and export the model to OpenVINO format
pipeline = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=dtype)

Export to OpenVINO format without quantization

python
# Export to OpenVINO format without quantization
ov_pipeline = OVStableDiffusionPipeline.from_pretrained(
    model_id,
    export=True,
    compile=False,
    load_in_8bit=False,  # Explicitly disable 8-bit quantization
    load_in_4bit=False,  # Explicitly disable 4-bit quantization
    torch_dtype=dtype
)

Define INT8 quantization configuration

python
# Define INT8 quantization configuration
ov_weight_config_int8 = OVWeightQuantizationConfig(
    weight_only=True,
    bits=8,
    mode=CompressWeightsMode.INT8_SYM  # Use enum instead of string
)
ov_config_int8 = OVConfig(quantization_config=ov_weight_config_int8)

Processing and Save Quantization Model

python
# Create Quantization Directory
save_dir_int8 = "ghibli_sd_int8"
os.makedirs(save_dir_int8, exist_ok=True)

# Initialize quantizer
quantizer = OVQuantizer.from_pretrained(ov_pipeline, task="stable-diffusion")

# Quantize the model
quantizer.quantize(ov_config=ov_config_int8, save_directory=save_dir_int8)

# Save scheduler and tokenizer
pipeline.scheduler.save_pretrained(save_dir_int8)
pipeline.tokenizer.save_pretrained(save_dir_int8)

Usage

Install Dependencies

bash
pip install -q "optimum-intel[openvino,diffusers]" openvino

Import Libraries

python
import torch
from optimum.intel import OVStableDiffusionPipeline

###

python
device = "cuda" if torch.cuda.is_available() else "cpu"

pipe = OVStableDiffusionPipeline.from_pretrained("danhtran2mind/Ghibli-Stable-Diffusion-2.1-Base-finetuning-INT8")
pipe.to(device)