devoffeed/gemma-cvantic

<div align="center">

gemma-cvantic

multimodal SFT fine-tune of Google's `gemma-4-E4B-it` · GGUF

    ![Quant: Q8/Q5/iQ4]()

</div>

What is this?

An experimental instruction fine-tune of google/gemma-4-E4B-it — the 4.5B-effective-parameter omni-modal Gemma 4 model (128K context, text + image + audio input).

Trained with SFT (13,000 examples) to strengthen math, coding, physics/astronomy reasoning and Russian-language instructions. Quantized to GGUF with llama.cpp for fast local inference.

Quantization lineup

mmproj is the multimodal (vision) projector — pass it with --mmproj to enable image understanding. All quants are BF16/FP16 conversions of the same merged weights, so any main-file + mmproj combo works.

Training recipe

Dataset mix (MIT / Apache-2.0 only):

Quick start

llama.cpp (text)

llama-cli \
  -m gemma-cvantic.Q5_K_S.gguf \
  -p "Реши задачу: если цена товара выросла на 20% и составила 480 руб., какой была исходная цена?"

Multimodal (vision)

llama-cli \
  -m gemma-cvantic.Q5_K_S.gguf \
  --mmproj gemma-cvantic.BF16-mmproj.gguf \
  -i

> what's in this photo?

llama-server (OpenAI-compatible API)

llama-server \
  -m gemma-cvantic.Q8_0.gguf \
  --mmproj gemma-cvantic.BF16-mmproj.gguf \
  --port 8080

import openai

client = openai.OpenAI(base_url="http://localhost:8080/v1", api_key="local")
resp = client.chat.completions.create(
    model="gemma-cvantic",
    messages=[{"role": "user", "content": "Расскажи про эффект Доплера на пальцах"}],
)
print(resp.choices[0].message.content)

Picking a file

• —CPU-only, want quality → Q5_K_S (fits ~8 GB RAM/VRAM, sweet spot)
• —16 GB+ / strong GPU → Q8_0
• —Tiny footprint / speed first → iQ4_XS (fits ~6 GB)
• —Vision → always add the BF16-mmproj

Sizes are approximate; VRAM usage depends on context length.

Notes

• —Working-name "cvantic" — experimental build, quality varies per domain. Think of it as a field test of Gemma 4 E4B fine-tuning.
• —Text-only SFT; vision/audio behavior inherited from the base model and not specifically tuned.
• —Base model: Google DeepMind · License: Apache 2.0 · Gemma 4 docs

Quantized with `llama.cpp` (BF16 base + imatrix for iQ4_XS).