tripathyShaswata/sarvam-1-v0.5-GGUF

Sarvam-1-v0.5 GGUF

GGUF quantized versions of sarvamai/sarvam-1-v0.5 for local inference with llama.cpp, Ollama, LM Studio, and GPT4All.

Sarvam-1 is an Indian multilingual LLM built by Sarvam AI — supporting 22 Indian languages including Hindi, Bengali, Tamil, Telugu, Kannada, Malayalam, Marathi, Punjabi, Gujarati, and Odia. Based on Llama architecture with 3.1B parameters.

Available Quantizations

How to Use

With llama.cpp

./llama-cli -m sarvam-1-v0.5-Q8_0.gguf -p "भारत की राजधानी क्या है?" -n 256

With Ollama

# Create a Modelfile
echo 'FROM ./sarvam-1-v0.5-Q8_0.gguf' > Modelfile
ollama create sarvam -f Modelfile
ollama run sarvam

With LM Studio

1. 1.Download the Q8_0 file
2. 2.Open LM Studio → Load Model → Select the file
3. 3.Start chatting in English or any supported Indian language

Model Details

• —Architecture: Llama
• —Parameters: 3.1B
• —Hidden Size: 2048
• —Layers: 28
• —Attention Heads: 16
• —Context Length: Check original model card
• —Languages: English + 22 Indian languages (Hindi, Bengali, Tamil, Telugu, Kannada, Malayalam, Marathi, Punjabi, Gujarati, Odia, and more)
• —License: Apache 2.0

Original Model

Built by Sarvam AI — India's leading AI research company. See the original model at sarvamai/sarvam-1-v0.5.

Quantized by

Shaswata Tripathy | GitHub | Medium | LinkedIn | Hugging Face