CraneAILabs/swahili-gemma-1b-GGUF

Swahili Gemma 1B - GGUF

Quantized GGUF versions of Swahili Gemma 1B, a fine-tuned Gemma 3 1B instruction model specialized for English-to-Swahili translation and Swahili conversational AI. The model accepts input in both English and Swahili but outputs responses exclusively in Swahili.

📊 Translation Performance

[image]

Model Comparison

*Efficiency = BLEU Score / Parameters (in billions)

Key Performance Insights

🎯 Efficiency Leader: Achieves the highest BLEU-to-parameter ratio (27.6 BLEU per billion parameters) 🚀 Size Advantage: Outperforms Gemma 3 4B (4x larger) by 153% on BLEU score 💎 Competitive Quality: Achieves 94% of Gemma 3 27B performance with 27x fewer parameters ⚡ Practical Deployment: Runs efficiently on consumer hardware while maintaining quality

Evaluation Details

• —Dataset: FLORES-200 English→Swahili (1,012 translation pairs)
• —Metrics: BLEU (bilingual evaluation understudy) and chrF++ (character F-score)
• —Evaluation: Zero-shot translation performance

🚀 Quick Start

# Download the recommended Q4_K_M quantization
pip install huggingface_hub

# Python download
from huggingface_hub import snapshot_download
snapshot_download(
    repo_id="CraneAILabs/swahili-gemma-1b-GGUF",
    local_dir="swahili-gemma-1b-GGUF",
    allow_patterns=["Q4_K_M/*"]  # Download only Q4_K_M version
)

📊 Available Quantizations

💻 Usage with llama.cpp

Basic Translation

# English to Swahili translation
./llama-cli \
  --model swahili-gemma-1b-GGUF/Q4_K_M/swahili-gemma-1b-q4_k_m.gguf \
  --prompt "Translate to Swahili: Hello, how are you today?" \
  --temp 0.3 \
  --top-p 0.95 \
  --top-k 64 \
  --repeat-penalty 1.1 \
  -n 128

🔧 Usage with Ollama

# Create model from GGUF
ollama create swahili-gemma-1b -f Modelfile

# Use for translation
ollama run swahili-gemma-1b "Translate to Swahili: Good morning!"

# Use for conversation  
ollama run swahili-gemma-1b "Hujambo! Je, unaweza kunisaidia?"

Modelfile Example

FROM swahili-gemma-1b-GGUF/Q4_K_M/swahili-gemma-1b-q4_k_m.gguf

TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>

{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

{{ .Response }}<|eot_id|>"""

PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"

🐍 Usage with Python (llama-cpp-python)

from llama_cpp import Llama

# Initialize model
llm = Llama(
    model_path="swahili-gemma-1b-GGUF/Q4_K_M/swahili-gemma-1b-q4_k_m.gguf",
    n_ctx=2048,
    n_threads=8,
    verbose=False
)

# Generate translation
response = llm(
    "Translate to Swahili: Hello, how are you today?",
    max_tokens=128,
    temperature=0.3,
    top_p=0.95,
    top_k=64,
    repeat_penalty=1.1
)

print(response['choices'][0]['text'])

🌍 Language Capabilities

• —Input Languages: English + Swahili
• —Output Language: Swahili only
• —Primary Focus: English-to-Swahili translation and Swahili conversation

📊 Performance Metrics

Translation Quality (BLEU Scores)

Evaluated on 1,012 English-to-Swahili translation samples.

🎯 Capabilities

• —Translation: English-to-Swahili translation
• —Conversational AI: Natural dialogue in Swahili
• —Summarization: Text summarization in Swahili
• —Writing: Creative and informational writing in Swahili
• —Question Answering: General knowledge responses in Swahili

💡 Recommended Parameters

# Optimal settings for translation tasks
--temp 0.3
--top-p 0.95
--top-k 64
--repeat-penalty 1.1
--ctx-size 2048

🔗 Related Models

• —Original Model: CraneAILabs/swahili-gemma-1b - Full precision HuggingFace model
• —LiteRT Mobile: CraneAILabs/swahili-gemma-1b-litert - Mobile deployment
• —Ollama: crane-ai-labs/swahili-gemma-1b - Ready-to-run models

🛠️ Technical Details

• —Base Model: google/gemma-3-1b-it
• —Architecture: Gemma 3
• —Context Length: 4,096 tokens
• —Quantization: GGML format with multiple precision levels
• —Compatible: llama.cpp, Ollama, Jan, LM Studio, and other GGUF engines

🎨 Use Cases

• —Offline Translation: Run Swahili translation without internet
• —Local AI Assistant: Swahili conversational AI on your machine
• —Educational Tools: Language learning applications
• —Content Creation: Generate Swahili content locally
• —Research: Swahili language model experiments

⚠️ Limitations

• —Language Output: Responds only in Swahili
• —Quantization Trade-offs: Lower bit quantizations may reduce quality
• —Context Limit: 4K tokens for optimal performance
• —Specialized Tasks: May need fine-tuning for specific domains

📄 License

This model is released under the Gemma Terms of Use. Please review the terms before use.

🙏 Acknowledgments

• —Google: For the Gemma 3 base model, support and guidance.
• —Community: For Swahili language resources and datasets
• —Gilbert Korir (Msingi AI, Nairobi, Kenya)
• —Alfred Malengo Kondoro (Hanyang University, Seoul, South Korea)

Citation

If you use these GGUF quantizations in your research or applications, please cite:

@misc{crane_ai_labs_2025,
    author    = {Bakunga Bronson and Kato Steven Mubiru and Lwanga Caleb and Gimei Alex and Kavuma Lameck and Roland Ganafa and Sibomana Glorry and Atuhaire Collins and JohnRoy Nangeso and Tukamushaba Catherine},
    title     = {Swahili Gemma: A Fine-tuned Gemma 3 1B Model for Swahili conversational AI},
    year      = {2025},
    url       = {https://huggingface.co/CraneAILabs/swahili-gemma-1b},
    organization = {Crane AI Labs}
}

Built with ❤️ by Crane AI Labs

Swahili Gemma - Your helpful Swahili AI companion, optimized for local deployment