Grinding/ticketsense-category

TicketSense — Category Classifier

Fine-tuned distilroberta-base (82M params) for category classification on real customer-support tweets, with labels distilled from Claude Sonnet 4.6.

Part of the TicketSense project: a tiny specialized classifier that approximates frontier-LLM judgement at a fraction of the cost and latency.

Labels

• —billing
• —technical
• —account
• —feature_request
• —other

Eval

• —Macro F1 (validation): 0.5717

Training

• —Base model: distilroberta-base
• —Method: LoRA (PEFT), then merged
• —LoRA rank: 16, alpha: 32
• —LR: 0.0002, epochs: 4
• —Train data: ~1400 real customer-support tweets, Claude-labeled

Use

from transformers import pipeline
clf = pipeline("text-classification", model="Grinding/ticketsense-category")
clf("my card was charged twice and nobody has responded")

Limitations

• —Trained on noisy real-world Twitter data — picks up typos and slang but may also reflect their biases.
• —Labels are themselves model-generated (Claude Sonnet 4.6) — treat F1 vs Claude as agreement, not gold-standard truth.