gokceai/qwen7b_baseline_v1

Qwen 2.5 7B — Enterprise Customer Support LoRA Adapter

QLoRA fine-tuned adapter for enterprise customer support conversations.

Model Details

• —Base model: Qwen/Qwen2.5-7B-Instruct
• —Method: QLoRA (NF4 4-bit quantization + LoRA adapter)
• —LoRA config: r=4, alpha=8, targetmodules=[qproj, v_proj]
• —Dataset: 963 training + 86 validation customer support conversations
• —Trainable parameters: 1,261,568 (0.017% of base)
• —Adapter size: 21 MB

Performance

Capabilities

• —Multi-turn context retention
• —Empathetic clarification (no over-asking)
• —Out-of-scope refusal with polite redirect
• —Escalation handling without fake actions
• —6 customer tones supported (neutral, frustrated, angry, confused, urgent, non-native English)

Training Details

• —Framework: PEFT 0.13.2, TRL 0.12.1, transformers 4.46.3
• —Precision: FP16 (Turing T4 compatibility)
• —Attention: SDPA
• —Optimizer: pagedadamw8bit
• —LR: 2e-4, cosine schedule, 5% warmup
• —Batch: effective 32 (perdevice=1, gradaccum=32)
• —Epochs: 1

Limitations

• —Baseline model trained on small dataset (963 examples, 1 epoch only)
• —No RAG integration — knowledge is only what's in fine-tune data
• —Cannot perform real actions; only describes/initiates them
• —English only; multilingual support planned

License

Apache 2.0 — same as base Qwen 2.5