BCN001/llm-complexity-router

LLM Complexity Router

A fine-tuned DeBERTa-v3-small classifier that routes queries between gpt-4o-mini (cheap) and gpt-4o (expensive) — saving ~41% cost while improving response quality vs always using the expensive model.

Performance (WildBench — 200 real user queries)

Only router that beats the expensive baseline on quality and saves cost.

Category Breakdown (vs always_expensive)

Usage

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="your-username/complexity-router"
)

result = classifier("What is the capital of France?")
# → [{'label': 'SIMPLE', 'score': 0.98}]  → route to gpt-4o-mini

result = classifier("Prove the Riemann hypothesis step by step")
# → [{'label': 'COMPLEX', 'score': 0.95}]  → route to gpt-4o

Training

• —Base model: microsoft/deberta-v3-small
• —Training data: proprietary (not released)
• —Labels: SIMPLE / COMPLEX
• —Benchmarked against: RouteLLM mf router, length-based baseline

Limitations

• —Weaker on Coding & Debugging (-0.16) and Planning (-0.18)
• —Optimized for gpt-4o vs gpt-4o-mini routing specifically
• —Training data distribution may not match all use cases