Modelpublic

DinoStackAI/Qwen3-8b-lora-telco-dpr

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes7downloads

Model Card

Qwen3-8b-lora-telco-dpr

LoRA adapter for Qwen/Qwen3-8B fine-tuned on the telco-dpr RAG generative dataset (DinoStackAI/telco-dpr-rag).

—Best dev metric: eval_loss = 0.6854

Load with PEFT

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "DinoStackAI/Qwen3-8b-lora-telco-dpr")
tokenizer = AutoTokenizer.from_pretrained("DinoStackAI/Qwen3-8b-lora-telco-dpr")

Load with vLLM (LoRA)

python

from vllm import LLM
from vllm.lora.request import LoRARequest

llm = LLM(
    model="Qwen/Qwen3-8B",
    enable_lora=True,
    max_lora_rank=16,
)
outputs = llm.generate(
    prompts,
    lora_request=LoRARequest("telco-dpr", 1, "DinoStackAI/Qwen3-8b-lora-telco-dpr"),
)

Use this adapter with scripts/generation/run_rag_generation.py --lora-path DinoStackAI/Qwen3-8b-lora-telco-dpr.

Training details

—Base model: Qwen/Qwen3-8B
—Fine-tuning dataset: DinoStackAI/telco-dpr-rag
—Method: LoRA (r=16, lora_alpha=32, lora_dropout=0.05)
—Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
—Loss: SFT with completion-only masking (assistant_only_loss=True)
—Best checkpoint selection: dev eval_loss