klei1/bleta-sq-2b

Bleta SQ 2B — Albanian Language Model

Bleta is a fine-tuned Gemma-4 2B model specialized for the Albanian language (Shqip). Built to understand and generate natural, grammatically correct Albanian text.

Bleta (🐝) means "bee" in Albanian — a symbol of diligence and precision.

Model Details

Training Dataset

Fine-tuned on [klei1/bleta-sq-dataset-v1](https://huggingface.co/datasets/klei1/bleta-sq-dataset-v1) — a curated Albanian language instruction dataset covering conversation, grammar, reasoning, and general knowledge.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "klei1/bleta-sq-2b"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

messages = [
    {"role": "user", "content": "Cila eshte kryeqyteti i Shqiperise?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    return_tensors="pt",
    add_generation_prompt=True
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.95,
    repetition_penalty=1.2,
    do_sample=True,
)

response = tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)

Recommended Generation Parameters

Capabilities

• —Albanian conversational AI
• —Grammar correction and explanation
• —Albanian text generation and creative writing
• —Translation (Albanian ↔ English)
• —General knowledge in Albanian
• —Question answering

Limitations

• —2B parameter model — complex reasoning may be limited
• —Primarily trained on Albanian; performance varies by topic
• —May occasionally produce grammatically imperfect outputs

Bleta Model Family

Citation

@model{bleta_sq_2b_2026,
  title  = {Bleta SQ 2B: Gemma-4 Fine-tuned for Albanian Language},
  author = {klei1},
  year   = {2026},
  url    = {https://huggingface.co/klei1/bleta-sq-2b}
}

License

This model is released under the Apache 2.0 License.