samarthruckstar/qwen-whatsapp-agent

🤖 Qwen2.5-1.5B WhatsApp Customer Service Agent (Mahadyuti Engineering Solution)

This repository contains the fine-tuned weights and quantized GGUF for the Mahadyuti Engineering Solution AI WhatsApp customer service and catalog agent.

🎯 Model Capabilities

• —Direct Catalog Precision: Grounded responses on industrial machinery (Slipper making machines, camphor slab dies, sambrani cup machines, papad rolling machines).
• —Tax Accuracy: Consistently enforces and declares +18% GST Extra on all machinery price quotes.
• —Image Attribution: Generates verified product image markdown references.
• —Anti-Hallucination: Rejects inquiries for unstocked or unmanufactured goods (e.g. aircraft, electronics, consumer gadgets) without inventing products or pricing.

📦 Formats Included in This Repository

1. 1.PEFT LoRA Adapter (`adapter_model.safetensors`) (~73.9 MB)
2. 2.Base model: Qwen/Qwen2.5-1.5B-Instruct
3. 3.Rank: r=16, alpha=16
4. 4.Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

1. 1.Ollama & llama.cpp GGUF (`qwen_whatsapp_q4_k_m.gguf`) (~986 MB)
2. 2.4-bit medium K-quantization (Q4_K_M) for fast CPU/GPU inference.

💻 Usage with PEFT & Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_MODEL_ID = "Qwen/Qwen2.5-1.5B-Instruct"
LORA_REPO_ID = "samarthruckstar/qwen-whatsapp-agent"

tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_ID)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL_ID,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, LORA_REPO_ID)
model.eval()

prompt = "What is the price, power, and capacity of H-ASM15T Automatic Slipper Making Machine?"
messages = [
    {"role": "system", "content": "You are the official WhatsApp AI assistant for Mahadyuti Engineering Solution."},
    {"role": "user", "content": prompt}
]

inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt").to(model.device)
outputs = model.generate(inputs, max_new_tokens=256, do_sample=False)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

🦙 Usage with Ollama or llama.cpp

Download qwen_whatsapp_q4_k_m.gguf:

hf download samarthruckstar/qwen-whatsapp-agent qwen_whatsapp_q4_k_m.gguf --local-dir .

Run in Ollama:

ollama create qwen-whatsapp -f Modelfile
ollama run qwen-whatsapp "Hi, tell me about your slipper making machines"