Modelpublic

arbib/darija-qwen2.5-multigpu-lora_v3

sourceHugging Faceupdated 7mo agoView on Hugging Face

1likes7downloads

Model Card

Darija Qwen2.5-1.5B — LoRA (LLaMA-Factory + Unsloth + 2x T4)

Fine-tune sur Qwen/Qwen2.5-1.5B-Instruct pour le dialecte marocain (Darija).

Stack

—LLaMA-Factory (orchestration multi-GPU)
—Unsloth (backend rapide)
—2x GPU T4 — fp16 pur (pas de quantization)

Dataset

—MBZUAI-Paris/Darija-SFT-Mixture — 5 000 samples (direction=None)

Config LoRA

—lorarank: 16 | loraalpha: 32 | target: all

Utilisation

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-1.5B-Instruct", torch_dtype=torch.float16, device_map="auto"
)
model     = PeftModel.from_pretrained(base, "REPO_ID")
tokenizer = AutoTokenizer.from_pretrained("REPO_ID")