SindreLinden/gemma-3-1b-ifs-cloud-gguf

Fine-tuned Gemma 3 1B IT (GGUF)

This is a fine-tuned version of Google's Gemma 3 1B IT model, converted to GGUF format for efficient inference.

Model Details

• —Base Model: google/gemma-3-1b-it
• —Fine-tuning Method: QLoRA (Quantized Low-Rank Adaptation)
• —Format: GGUF (F16 precision)
• —Size: ~1.9GB

Training Details

• —Adapter: QLoRA with rank 32, alpha 16
• —Target Modules: upproj, downproj, gateproj, qproj, kproj, vproj, o_proj
• —Sequence Length: 2048
• —Training Data: Custom dataset with sample packing
• —Epochs: 3
• —Learning Rate: 0.0004 with cosine scheduler
• —Optimizer: AdamW BNB 8-bit

Usage

This GGUF model can be used with various inference engines:

llama.cpp

./llama-server -m model.gguf

Ollama

# Create a Modelfile
FROM model.gguf
TEMPLATE """<start_of_turn>user
{{ .Prompt }}<end_of_turn>
<start_of_turn>model
"""

# Import the model
ollama create my-gemma-model -f Modelfile
ollama run my-gemma-model

Python with llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="model.gguf")
output = llm("Tell me about artificial intelligence", max_tokens=512)
print(output)

Chat Template

This model uses Gemma 3's chat template:

<start_of_turn>user
{user_message}<end_of_turn>
<start_of_turn>model
{assistant_response}<end_of_turn>

Limitations

• —This is a small 1B parameter model with inherent limitations
• —Fine-tuned for specific use cases - performance may vary on other tasks
• —GGUF conversion may introduce minor numerical differences compared to the original model

License

This model inherits the license from the base Gemma model. Please refer to Google's Gemma license for usage terms.