devoffeed/gemma-cvantic

1---2license: apache-2.03base_model:4- google/gemma-4-E4B-it5tags:6- gemma7- gemma-48- gguf9- multimodal10- vision11- finetune12- sft13- russian14- physics15- math16- coding17library_name: gguf18pipeline_tag: text-generation19---20 21<div align="center">22 23# gemma-cvantic24 25**multimodal SFT fine-tune of Google's `gemma-4-E4B-it` · GGUF**26 27[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20HuggingFace-gemma--cvantic-FF9D00)](https://huggingface.co/devoffeed/gemma-cvantic)28[![Base](https://img.shields.io/badge/base-google%2Fgemma--4--E4B--it-blue)](https://huggingface.co/google/gemma-4-E4B-it)29[![License](https://img.shields.io/badge/license-Apache--2.0-green)](https://www.apache.org/licenses/LICENSE-2.0)30[![GGUF](https://img.shields.io/badge/format-GGUF%20%2F%20llama.cpp-yellow)](https://github.com/ggml-org/llama.cpp)31[![Quant: Q8/Q5/iQ4](https://img.shields.io/badge/quants-Q8_0%20%E2%80%A2%20Q5_K_S%20%E2%80%A2%20iQ4_XS-orange)]()32 33</div>34 35---36 37## What is this?38 39An experimental **instruction fine-tune** of [google/gemma-4-E4B-it](https://huggingface.co/google/gemma-4-E4B-it) — the 4.5B-effective-parameter omni-modal Gemma 4 model (128K context, text + image + audio input).40 41Trained with **SFT (13,000 examples)** to strengthen math, coding, physics/astronomy reasoning and Russian-language instructions. Quantized to GGUF with `llama.cpp` for fast local inference.42 43---44 45## Quantization lineup46 47| File | Quant | Size | Quality / speed |48|---|---|---|---|49| `gemma-cvantic.Q8_0.gguf` | **Q8_0** | ~7.6 GB | near-lossless, best quality |50| `gemma-cvantic.Q5_K_S.gguf` | **Q5_K_S** | ~5.4 GB | great balance ★ recommended |51| `gemma-cvantic.IQ4_XS.gguf` | **iQ4_XS** (imatrix) | ~4.8 GB | smallest, fast, slightly less accurate |52| `gemma-cvantic.BF16-mmproj.gguf` | vision projector | ~0.9 GB | required for image input |53 54> `mmproj` is the **multimodal (vision) projector** — pass it with `--mmproj` to enable image understanding. All quants are BF16/FP16 conversions of the same merged weights, so any main-file + `mmproj` combo works.55 56---57 58## Training recipe59 60| Parameter | Value |61|---|---|62| Base model | `google/gemma-4-E4B-it` |63| Method | SFT (DoRA / LoRA-style adapter, then merged) |64| Trainable params | ~36.7M (0.61%) |65| Examples | 13,000 |66| Context | 128K (inherited) |67 68**Dataset mix** (MIT / Apache-2.0 only):69 70| Dataset | Split | Rows | Domain |71|---|---|---|---|72| `HuggingFaceH4/ultrachat_200k` | train_sft | 5,000 | general chat |73| `theblackcat102/evol-codealpaca-v1` | train | 3,000 | coding |74| `qwedsacf/competition_math` (MATH) | train | 2,000 | math |75| `HuggingFaceTB/cosmopedia` | openstax · physics/astronomy | 2,000 | physics & astronomy |76| `openai/gsm8k` | main/train | 1,000 | grade-school math |77 78---79 80## Quick start81 82### llama.cpp (text)83 84```bash85llama-cli \86  -m gemma-cvantic.Q5_K_S.gguf \87  -p "Реши задачу: если цена товара выросла на 20% и составила 480 руб., какой была исходная цена?"88```89 90### Multimodal (vision)91 92```bash93llama-cli \94  -m gemma-cvantic.Q5_K_S.gguf \95  --mmproj gemma-cvantic.BF16-mmproj.gguf \96  -i97```98 99```100> what's in this photo?101```102 103### llama-server (OpenAI-compatible API)104 105```bash106llama-server \107  -m gemma-cvantic.Q8_0.gguf \108  --mmproj gemma-cvantic.BF16-mmproj.gguf \109  --port 8080110```111 112```python113import openai114 115client = openai.OpenAI(base_url="http://localhost:8080/v1", api_key="local")116resp = client.chat.completions.create(117    model="gemma-cvantic",118    messages=[{"role": "user", "content": "Расскажи про эффект Доплера на пальцах"}],119)120print(resp.choices[0].message.content)121```122 123---124 125## Picking a file126 127- **CPU-only, want quality** → `Q5_K_S` (fits ~8 GB RAM/VRAM, sweet spot)128- **16 GB+ / strong GPU** → `Q8_0`129- **Tiny footprint / speed first** → `iQ4_XS` (fits ~6 GB)130- **Vision** → always add the `BF16-mmproj`131 132Sizes are approximate; VRAM usage depends on context length.133 134---135 136## Notes137 138- Working-name **"cvantic"** — experimental build, quality varies per domain. Think of it as a field test of Gemma 4 E4B fine-tuning.139- Text-only SFT; vision/audio behavior inherited from the base model and *not* specifically tuned.140- Base model: [Google DeepMind](https://deepmind.google/models/gemma/) · License: **Apache 2.0** · [Gemma 4 docs](https://ai.google.dev/gemma/docs/core)141 142---143 144*Quantized with `llama.cpp` (BF16 base + imatrix for iQ4_XS).*