whoy/GigaChat3-10B-A1.8B-GradioChat-q4km

GigaChat3-10B-A1.8B Gradio Chat Interface

A chat interface powered by GigaChat3-10B-A1.8B, a Mixture of Experts (MoE) language model. This Space runs the model using llama.cpp (b7130) for efficient CPU inference with Q8_0 quantization.

🚀 Model Details

• —Model: whoy/GigaChat3-10B-A1.8B-bf16-gguf
• —Architecture: DeepSeek2 with MoE (64 experts, 4 active per token)
• —Parameters: 10.67B total, 1.8B active
• —Quantization: Q4KM (~6.1 GB)
• —Context Length: 8192 tokens (model supports up to 262k)
• —Languages: Primarily Russian and English

⚙️ Technical Stack

• —Inference Engine: llama.cpp (release b7130)
• —Backend: Native CPU with AVX512 support
• —Interface: Gradio ChatInterface
• —Chat Template: Custom Jinja template with proper role handling

📝 Usage

Simply type your message in the chat interface. The model will respond in a conversational manner. Works best with:

• —Question answering
• —Code generation (Python, JavaScript, etc.)
• —Creative writing
• —Technical explanations
• —General conversation in Russian and English

🎯 Examples

• —"What is GigaChat?"
• —"Write a Python function to sort a list"
• —"Explain quantum physics in simple terms"
• —"Tell a story"

⚠️ Limitations

• —Speed: Running on CPU, expect 2-5 tokens/second
• —Context: Limited to 8192 tokens per conversation
• —Language: Optimized for Russian, but supports English
• —Memory: May be slow on first request (model loading)

🔧 Configuration

The model runs with the following settings:

• —Temperature: 0.5
• —Top-p: 0.95
• —Max output tokens: 1024
• —MoE mode: Enabled (-cmoe flag)

📜 License

The model weights are released under MIT license by Sber AI.

🙏 Credits

• —Model: ai-sage/GigaChat3-10B-A1.8B
• —Quantization: whoy
• —Inference: llama.cpp by ggml-org

Note: This is a community-hosted demo. Response quality may vary. For production use, consider running on GPU hardware.