costinstroie/RoGemma3-4B-Instruct-GGUF

RoGemma3-4B-Instruct-GGUF (text-only, Q4KM)

Q4KM GGUF quantization of OpenLLM-Ro/RoGemma3-4B-Instruct, converted with llama.cpp's convert_hf_to_gguf.py. Vision tower dropped — text-only language model half, no mmproj file.

Two issues fixed during conversion

1. 1.No `tokenizer.model` (fast/BPE tokenizer only) and this project's llama.cpp didn't have Gemma 3's BPE pre-tokenizer hash registered, so the default GPT-2/BPE vocab path raised NotImplementedError. Worked around by supplying the tokenizer.model (SentencePiece) from google/gemma-3-4b-it — RoGemma3 is continually pretrained from that exact base with an unmodified 262208-token vocab (confirmed via config.json), so the SPM tokenizer is identical; this routes convert_hf_to_gguf.py through the SPM path instead, which is exact, not approximate.
2. 2.LM Studio runtime version mismatch. Two earlier conversions (including a community re-upload from Andarwarm99) failed to load on this project's LM Studio server with a generic Error loading model., despite loading and generating correctly with a locally built llama-cli. Root cause: the GGUF was written by a newer llama.cpp/gguf-py (build ~8808) than LM Studio's bundled CUDA runtime, which reports itself as llama.cpp 2.13.0 — LM Studio's own internal runtime versioning, which corresponds to upstream ggml-org/llama.cpp build b8733 (commit 26229755c, confirmed via llama-cli --version after checking out that exact commit and rebuilding). This file was converted and quantized with llama.cpp checked out at that exact commit, matching LM Studio's runtime.

Non-commercial (cc-by-nc-4.0, inherited from the base model).