Modelpublic

TatarNLPWorld/mistral-7b-tatar-lora-r16

sourceHugging Faceapache-2.0updated 7mo agoView on Hugging Face

0likes10downloads

Model Card

Mistral-7B LoRA Adapter for Tatar Language

This repository contains a LoRA adapter fine-tuned on Tatar language using QLoRA.

📊 Model Details

Property	Value
Base Model	Mistral-7B-v0.3
LoRA Rank	16
Training Data	10,000 samples from Tatar Monocorpus
Test Perplexity	3.98
Training Epochs	2
Learning Rate	2e-4

🚀 Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model
base_model = "mistralai/Mistral-7B-v0.3"
tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load adapter
model = PeftModel.from_pretrained(model, "TatarNLPWorld/mistral-7b-tatar-lora-r16")

# Generate text
prompt = "Минем исемем"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(
    **inputs,
    max_new_tokens=50,
    temperature=0.7,
    top_p=0.9
)
print(tokenizer.decode(outputs[0]))

📈 Generation Examples

Prompt	Generated Text
Минем исемем	Илсур Метшин, башкаладагы даими мэрының тикшерелегенә үз бәяләрен ачты
Бүгенге көндә	ТР Дәүләт Советының VIII сессиясе уздырылачак
Татарстан —	"Ватанлылар бердәмлеге" кандидатлары белән 19 майда уздырыла торган Президенты вазифасына гариза и
Кыш көне	беренче тапкыр һәм ике атнада түбән берничә ялып узган «Тантана» фестивале башкаручы

🧪 Performance

Tested on 500 held-out samples from Tatar Monocorpus:

—Perplexity: 3.98 ± 0.05
—Training time: ~44 minutes on Tesla V100-32GB
—GPU memory: ~16.9 GB during training

📜 License

This adapter is released under Apache 2.0 license. Base model Mistral-7B-v0.3 has its own license.

🤝 Citation

bibtex

@software{mistral_tatar_lora_2026,
    title = {Mistral-7B LoRA Adapter for Tatar Language},
    author = {Arabov Mullosharaf Kurbonovich},
    year = {2026},
    publisher = {Hugging Face},
    url = {https://huggingface.co/TatarNLPWorld/mistral-7b-tatar-lora-r16}
}

👥 Author

—Arabov Mullosharaf Kurbonovich (Kazan Federal University)