Modelpublic

tahsinahsen/birag-gemma4-e2b-response-only-gguf

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

1likes38downloads

Model Card

BIRAG Gemma 4 E2B Response-Only — GGUF Q4KM

Bu repo, tahsinahsen/birag-gemma4-e2b-response-only merged 16-bit modelinin Unsloth ve llama.cpp kullanılarak oluşturulmuş Q4KM GGUF dağıtımıdır.

—Dataset: tahsinahsen/birag-response-only-tr, revision v0.4
—Quantization: Q4_K_M
—Multimodal projector: BF16
—Doğrulanan context: 16384
—Chat template: GGUF metadata içindeki Gemma 4 template
—

llama.cpp ile kullanım

bash

hf download tahsinahsen/birag-gemma4-e2b-response-only-gguf \
  birag-gemma4-e2b-response-only-Q4_K_M.gguf \
  birag-gemma4-e2b-response-only-mmproj-BF16.gguf \
  --local-dir ./birag-gemma4-e2b-gguf

bash

llama-server \
  --model ./birag-gemma4-e2b-gguf/birag-gemma4-e2b-response-only-Q4_K_M.gguf \
  --mmproj ./birag-gemma4-e2b-gguf/birag-gemma4-e2b-response-only-mmproj-BF16.gguf \
  --ctx-size 16384 \
  --jinja \
  --chat-template-kwargs '{"enable_thinking":false}' \
  --host 127.0.0.1 \
  --port 8080

bash

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "birag-gemma4-e2b-q4",
    "messages": [
      {
        "role": "system",
        "content": "Türkçe, destekleyici ve özerkliği güçlendiren yanıt ver."
      },
      {
        "role": "user",
        "content": "Daha bağımsız karar vermeye nasıl başlayabilirim?"
      }
    ],
    "max_tokens": 256,
    "temperature": 1.0,
    "top_p": 0.95,
    "stream": false
  }'