tanchaor/Llama-3.2-3B-SaaS-Support-GGUF

<div align="center"> <h1 style="color: #ffbf65;"> SaaS Support Ticket Automation Engine</h1> <h3>Llama-3.2-3B (4-bit GGUF)</h3> </div>

<div align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/68d8e6986a520d90b2470ac1/tgcflbms6edLLDH_jhk.png" alt="SaaS Support UI" width="700"> </div>

A specialized, local-first LLM fine-tuned to act as a highly efficient customer support automation engine. It reads incoming SaaS support tickets, classifies their intent, and instantly drafts a professional, concise JSON response.

This project demonstrates end-to-end applied ML engineering - taking an open-source model, training it on specialized data via QLoRA, and aggressively quantizing it down to a 4-bit GGUF format so it can run flawlessly on consumer edge hardware (like an Apple Silicon MacBook) without expensive cloud GPU costs.

<span style="color: #6366f1;">📊 Model Details</span>

<span style="color: #6366f1;">🎯 Uses & Capabilities</span>

Direct Use

This model is intended to be dropped directly into an automated pipeline (like a FastAPI backend) to handle level-1 customer support triage.

It takes an incoming user email and outputs a strict JSON format with three keys:

• —category (e.g., ACCOUNT, BILLING, SUPPORT)
• —intent (e.g., recoverpassword, refundrequest)
• —response (A professionally drafted, fluff-free reply)

Out-of-Scope Use

Because this model is highly compressed (4-bit) and heavily fine-tuned to penalize conversational "fluff", it is not recommended for complex coding tasks, advanced mathematics, or general open-ended chat. It is a specialized tool for a specific job.

<span style="color: #6366f1;">🛠️ Training Details</span>

Training Data

The model was fine-tuned using the open-source bitext/Bitext-customer-support-llm-chatbot-training-dataset.

Training Procedure

I utilized Parameter-Efficient Fine-Tuning (PEFT) with QLoRA to train the model on a single T4 GPU. The goal was to break the base model's habit of acting like a friendly chatbot and force it into a strict, predictable JSON output structure.

Hyperparameters

<span style="color: #6366f1;">📈 Evaluation & Accuracy</span>

During the final evaluation phase of training, the model showed a strong convergence, successfully learning the JSON structure and professional tone without overfitting.

Final Mean Token Accuracy: 85.58%

<span style="color: #6366f1;">💻 How to Get Started</span>

You can run this model entirely locally using llama.cpp or the llama-cpp-python binding. No cloud API keys required.

from llama_cpp import Llama
llm = Llama(
    model_path="llama-3.2-3b-saas-support.gguf",
    n_ctx=512,
    n_gpu_layers=-1 # Offload to Metal/CUDA
)
prompt = """<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are an emotionless AI routing assistant. Analyze the incoming customer ticket, classify its category and intent, and draft a response. CRITICAL RULES FOR RESPONSE: 1. NEVER apologize. NEVER say 'I am sorry' or 'I apologize'. 2. NEVER use conversational filler (e.g., 'I understand', 'I am here to help'). 3. State facts only. Be extremely direct and ultra-concise (1-2 sentences). You must output the result in a strict JSON format containing three keys: 'category', 'intent', and 'response'.<|eot_id|><|start_header_id|>user<|end_header_id|>
I was charged twice this month.<|eot_id|><|start_header_id|>assistant<|end_header_id|>
"""
output = llm(prompt=prompt, max_tokens=256, temperature=0.1, stop=["<|eot_id|>"])
print(output["choices"][0]["text"])