Modelpublic

annebanne/distilbert-support-classifier

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes

Model Card

DistilBERT Customer Support Ticket Classifier

Fine-tuned distilbert/distilbert-base-uncased for classifying customer support tickets into 27 intent categories.

Model Description

This model classifies raw customer support ticket text into one of 27 issue-type intents, enabling automated routing, prioritisation, and analytics for customer support pipelines.

—Architecture: DistilBERT-base-uncased (66M parameters, 40% smaller and 60% faster than BERT-base while retaining 97% of its performance)
—Task: Multi-class text classification (27 classes)
—Training data: Bitext Customer Support LLM Chatbot Training Dataset — 26,872 English utterances, nearly perfectly balanced across all classes (imbalance ratio: 1.05×)

Supported Intent Classes

ID	Intent	ID	Intent
0	cancel_order	14	edit_account
1	change_order	15	get_invoice
2	changeshippingaddress	16	get_refund
3	checkcancellationfee	17	newsletter_subscription
4	check_invoice	18	payment_issue
5	checkpaymentmethods	19	place_order
6	checkrefundpolicy	20	recover_password
7	complaint	21	registration_problems
8	contactcustomerservice	22	review
9	contacthumanagent	23	setupshipping_address
10	create_account	24	switch_account
11	delete_account	25	track_order
12	delivery_options	26	track_refund
13	delivery_period

Training Configuration

Hyperparameter	Value
Base model	distilbert/distilbert-base-uncased
Epochs	3
Batch size (per device)	32
Learning rate	2e-5
Weight decay	0.01
Warmup ratio	0.1
Max sequence length	128 tokens
Best model selected by	Macro F1
Optimizer	AdamW
Precision	fp16

Evaluation Results

⏳ Model training pending. Accuracy and Macro F1 will be filled in after training completes.

Evaluated on a held-out test set (15% of data, ~4,031 samples):

Metric	Value
Accuracy	(pending)
Macro F1	(pending)

See confusion_matrix.png for the full per-class breakdown (added after training).

Usage

python

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="annebanne/distilbert-support-classifier",
)

# Single ticket
result = classifier("I need to cancel my order, it hasn't shipped yet.")
print(result)
# [{'label': 'cancel_order', 'score': 0.98}]

# Batch of tickets
tickets = [
    "Where is my refund? It's been 2 weeks.",
    "I can't log into my account after resetting my password.",
    "Please send me an invoice for order #12345.",
    "My payment keeps getting declined.",
]
results = classifier(tickets)
for ticket, res in zip(tickets, results):
    print(f"{res['label']:35s} ({res['score']:.2%}) — {ticket[:60]}")

Dataset

—Source: bitext/Bitext-customer-support-llm-chatbot-training-dataset
—Size: 26,872 utterances
—Language: English
—Balance: Near-perfect — each class has ~950–1,000 examples (imbalance ratio: 1.05×)
—Split: 85% train (22,841) / 15% test (4,031), random seed 42

Limitations

—Trained on synthetic/augmented data. Real-world distribution may differ from production tickets.
—Performs best on short, single-intent utterances (similar to training data style).
—English only.
—Does not handle multi-intent tickets (predicts a single label).

Citation

If you use this model, please cite the base model:

bibtex

@article{sanh2019distilbert,
  title={DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter},
  author={Sanh, Victor and Debut, Lysandre and Chaumond, Julien and Wolf, Thomas},
  journal={arXiv preprint arXiv:1910.01108},
  year={2019}
}

Generated by ML Intern

This model repository was generated by ML Intern, an agent for machine learning research and development on the Hugging Face Hub.

—Try ML Intern: https://smolagents-ml-intern.hf.space
—Source code: https://github.com/huggingface/ml-intern