Modelpublic

pxlnstn/distilbert-ticket-routing

sourceHugging Facecc-by-nc-4.0updated 4mo agoView on Hugging Face

0likes9downloads

Model Card

DistilBERT — Customer-Support Ticket Routing

Fine-tuned `distilbert-base-uncased` that reads a customer-support ticket (subject + body) and predicts which of 10 departments should handle it. Built as a deep-learning term project.

Full project (code + paper + a CrewAI multi-agent demo): https://github.com/pxlnstn/distilbert-text-classification

Results (held-out test set)

Metric	Score
Accuracy	71.3%
Top-2 accuracy	81.6%
Macro-F1	0.68

For reference, the majority-class baseline is ~29% and random is ~10% on these 10 imbalanced classes. Best configuration: 7 epochs, learning rate 3e-5, 256-token inputs.

How to use

python

from transformers import pipeline

clf = pipeline("text-classification", model="pxlnstn/distilbert-ticket-routing",
               top_k=None)
print(clf("My internet has been down all morning and the VPN keeps disconnecting."))
# -> Technical Support (with IT Support close behind)

Labels (departments)

Billing and Payments, Customer Service, General Inquiry, Human Resources, IT Support, Product Support, Returns and Exchanges, Sales and Pre-Sales, Service Outages and Maintenance, Technical Support.

Training

—Base model: distilbert-base-uncased (6 layers, ~66M parameters).
—Data: English tickets from `Tobi-Bueck/customer-support-tickets` (~28k rows), stratified 80/10/10 train/validation/test split.
—Input: ticket subject + body, truncated to 256 tokens.
—Optimiser AdamW, FP16, batch size 16, 7 epochs, learning rate 3e-5.
—Trained on a single NVIDIA RTX 4070 Laptop GPU (8 GB).

Limitations

The data is synthetic and several departments overlap (Technical Support / IT Support / Product Support describe similar tickets), which caps achievable accuracy — this is why top-2 accuracy (82%) is much higher than top-1. Intended for educational and demonstration use.

License

cc-by-nc-4.0, following the non-commercial licence of the training dataset.