beleata74/BgGPT-Gemma-3-27B-IT-GGUF-Q4_K_M-mmproj

BgGPT-Gemma-3-27B-IT Q4KM + mmproj

This repository packages a working llama.cpp multimodal setup for BgGPT-Gemma-3-27B-IT:

• —BgGPT-Gemma-3-27B-IT-Q4_K_M.gguf
• —mmproj-bggpt-gemma3-27b-it-BF16.gguf

The GGUF model file comes from the original quantized release by INSAIT. The mmproj file was generated from the base Hugging Face model with llama.cpp/convert_hf_to_gguf.py --mmproj, so image input works in llama-server and llama-mtmd-cli.

What is included

• —Recommended Q4KM quantization for the 27B model
• —Generated Gemma 3 mmproj file required for image understanding in llama.cpp
• —Ready-to-run example commands

Example: llama-server

llama-server \
  -m BgGPT-Gemma-3-27B-IT-Q4_K_M.gguf \
  --mmproj mmproj-bggpt-gemma3-27b-it-BF16.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -c 4096 \
  --gpu-layers all

Example: OpenAI-compatible image request

{
  "model": "BgGPT-Gemma-3-27B-IT-Q4_K_M.gguf",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Опиши какво има на изображението."},
        {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}}
      ]
    }
  ]
}

Notes

• —This setup was tested successfully with llama.cpp CUDA build and the OpenAI-compatible server endpoint.
• —The base model and quantized weights belong to the original INSAIT BgGPT Gemma 3 release.
• —Please follow the Gemma terms of use when redistributing or serving the model.