mahdi2020/fine-tuned-distilbert-customer-intent-router

fine-tuned-distilbert-customer-intent-router

This model is a fine-tuned distilbert-base-uncased model for binary text classification, specifically designed for customer support intent routing. It classifies customer queries into two categories:

• —LABEL_0 (Informative): Queries that can be answered by retrieving information (e.g., FAQs, policies, order tracking).
• —LABEL_1 (Actionable): Queries that require transactional operations or state changes (e.g., canceling an order, changing an address, getting a refund).

Model Description

This model was fine-tuned on the Bitext customer support LLM chatbot training dataset. The original 27 intents were mapped to a binary classification scheme to create a robust router for conversational AI systems.

Intended Use

This model can be used as the first layer in a customer support chatbot architecture to direct user queries to the appropriate backend system:

• —Informative queries (LABEL_0) can be routed to a RAG (Retrieval Augmented Generation) system for knowledge base lookups.
• —Actionable queries (LABEL_1) can be routed to an LLM for direct generation or to a structured API call system.

How to Use

To use this model, you can load it directly from the Hugging Face Hub using the pipeline function from the transformers library:

from transformers import pipeline

classifier = pipeline("text-classification", model="mahdi2020/fine-tuned-distilbert-customer-intent-router")

# Example usage
print(classifier("Where is my package?"))
# Expected output: [{'label': 'LABEL_0', 'score': 0.9828}]

print(classifier("Cancel my package right now"))
# Expected output: [{'label': 'LABEL_1', 'score': 1.0000}]

Training Details

Model Architecture

• —Base Model: distilbert-base-uncased
• —Task: Binary Sequence Classification

Training Data

• —Dataset: Bitext-customer-support-llm-chatbot-training-dataset
• —Preprocessing: The original intent labels were mapped to LABEL_0 (Informative) and LABEL_1 (Actionable). The instruction column was used as the text input.

Training Hyperparameters

• —Learning Rate: 2e-5
• —Batch Size: 16 (per device)
• —Epochs: 3
• —Weight Decay: 0.01
• —Optimizer: AdamW
• —Max Sequence Length: 128 (with truncation)
• —Metrics Optimized: F1-score (for best model selection)

Performance

(Note: The exact final metrics from the `trainer.evaluate()` step should be updated here.)

Limitations

• —The model's performance is highly dependent on the quality and diversity of the training data. While the Bitext dataset is comprehensive, real-world customer queries might contain nuances not present in the training set.
• —It is a binary classifier and might struggle with ambiguous queries that could fall into both categories or new, unforeseen intents.
• —The max_length of 128 tokens assumes customer support inputs are generally short. Longer, more complex queries might be truncated, potentially losing context.