Echo9Zulu/gemma-3-12b-it-OpenVINO

My Project OpenArc, an inference engine for OpenVINO, now supports this model and serves inference over OpenAI compatible endpoints for text to text and text with vision!

We have a growing Discord community of others interested in using Intel for AI/ML.


• —Find documentation on the Optimum-CLI export process here
• —Use my HF space Echo9Zulu/Optimum-CLI-Tool_tool to build commands and execute locally

This repo contains OpenVINO quantizied versions of Phi-4-mini-instruct.

I reccomend starting with Echo9Zulu/gemma-3-12b-it-int8_asym-ov

To download individual models from this repo use the provided snippet:

from huggingface_hub import snapshot_download

repo_id = "Echo9Zulu/gemma-3-12b-it-OpenVINO"     

# Choose the weights you want
repo_directory = "gemma-3-12b-it-int4_asym-ov"

# Where you want to save it
local_dir = "./Echo9Zulu_gemma-3-12b-it-OpenVINO"

snapshot_download(
    repo_id=repo_id,
    allow_patterns=[f"{repo_directory}/*"], 
    local_dir=local_dir,
    local_dir_use_symlinks=True
) 

print("Download complete!")