bhaiyasingh45/functiongemma-multiagent-router
024
1---2language:3- en4license: gemma5library_name: transformers6tags:7- function-calling8- multi-agent9- router10- gemma11- fine-tuned12- customer-support13base_model: google/functiongemma-270m-it14datasets:15- bhaiyahnsingh45/multiagent-router-finetuning16metrics:17- accuracy18pipeline_tag: text-generation19widget:20- text: "My app keeps crashing when I upload large files"21 example_title: "Technical Issue"22- text: "I need a refund for my subscription"23 example_title: "Billing Request"24- text: "What integrations do you support?"25 example_title: "Product Info"26---27 28# Multi-Agent Router (Fine-tuned FunctionGemma 270M)29 30<div align="center">31 <img src="https://huggingface.co/datasets/huggingface/brand-assets/resolve/main/hf-logo.png" alt="Hugging Face" width="100"/>32 33 **Intelligent routing model for multi-agent customer support systems**34 35 [](https://ai.google.dev/gemma/terms)36 [](https://huggingface.co/google/functiongemma-270m-it)37 [](https://huggingface.co/datasets/bhaiyahnsingh45/multiagent-router-finetuning)38</div>39 40## 📋 Model Description41 42This model is a **fine-tuned version of Google's FunctionGemma 270M** specifically trained for intelligent routing in multi-agent customer support systems. It learns to:43 441. **Classify user intent** from natural language queries452. **Route to the appropriate specialist agent**463. **Extract relevant parameters** (priority, urgency, category)47 48### 🤖 Supported Agents49 50The model routes queries to three specialized agents:51 52| Agent | Handles | Parameters |53|-------|---------|------------|54| 🔧 **Technical Support** | Crashes, bugs, API errors, authentication issues | `issue_type`, `priority` |55| 💰 **Billing** | Payments, refunds, subscriptions, invoices | `request_type`, `urgency` |56| 📊 **Product Info** | Features, integrations, plans, compliance | `query_type`, `category` |57 58## 🎯 Training Details59 60### Base Model61- **Model**: `google/functiongemma-270m-it`62- **Parameters**: 270 Million63- **Architecture**: Gemma with function calling capabilities64 65### Fine-tuning Configuration66- **Training Samples**: 9267- **Test Samples**: 2368- **Epochs**: 1569- **Batch Size**: 470- **Learning Rate**: 5e-0571- **GPU**: NVIDIA T4 (Google Colab Free Tier)72- **Training Time**: ~5-8 minutes73 74### Dataset75Fine-tuned on [bhaiyahnsingh45/multiagent-router-finetuning](https://huggingface.co/datasets/bhaiyahnsingh45/multiagent-router-finetuning) containing 85 realistic customer support queries across three categories.76 77## 📊 Performance78 79| Metric | Before Training | After Training | Improvement |80|--------|----------------|----------------|-------------|81| **Accuracy** | 4.3% | 82.6% | **+78.3%** |82| **Correct Predictions** | 1/23 | 19/23 | +18 |83 84### Per-Agent Performance85- **Technical Support**: High accuracy on crash reports, API errors, authentication issues86- **Billing**: Excellent routing for refunds, payments, subscription management87- **Product Info**: Strong performance on feature queries, integrations, compliance questions88 89## 🚀 Quick Start90 91### Installation92 93```bash94pip install transformers torch95```96 97### Basic Usage98 99```python100from transformers import AutoTokenizer, AutoModelForCausalLM101import re102import json103 104# Load model and tokenizer105model_name = "bhaiyahnsingh45/functiongemma-multiagent-router"106tokenizer = AutoTokenizer.from_pretrained(model_name)107model = AutoModelForCausalLM.from_pretrained(108 model_name,109 device_map="auto",110 torch_dtype="auto"111)112 113# Define your agent tools114from transformers.utils import get_json_schema115 116def technical_support_agent(issue_type: str, priority: str) -> str:117 """118 Routes technical issues to specialized support team.119 120 Args:121 issue_type: Type of technical issue (crash, authentication, performance, api_error, etc.)122 priority: Priority level (low, medium, high)123 """124 return f"Routing to Technical Support: {issue_type} with {priority} priority"125 126def billing_agent(request_type: str, urgency: str) -> str:127 """128 Routes billing and payment queries.129 130 Args:131 request_type: Type of request (refund, invoice, upgrade, cancellation, etc.)132 urgency: How urgent (low, medium, high)133 """134 return f"Routing to Billing: {request_type} with {urgency} urgency"135 136def product_info_agent(query_type: str, category: str) -> str:137 """138 Routes product information queries.139 140 Args:141 query_type: Type of query (features, comparison, integrations, limits, etc.)142 category: Category (plans, storage, mobile, security, etc.)143 """144 return f"Routing to Product Info: {query_type} about {category}"145 146# Get tool schemas147AGENT_TOOLS = [148 get_json_schema(technical_support_agent),149 get_json_schema(billing_agent),150 get_json_schema(product_info_agent)151]152 153# System message154SYSTEM_MSG = "You are an intelligent routing agent that directs customer queries to the appropriate specialized agent."155 156# Function to route queries157def route_query(user_query: str):158 """Route a user query to the appropriate agent"""159 160 messages = [161 {"role": "developer", "content": SYSTEM_MSG},162 {"role": "user", "content": user_query}163 ]164 165 # Format prompt166 inputs = tokenizer.apply_chat_template(167 messages,168 tools=AGENT_TOOLS,169 add_generation_prompt=True,170 return_dict=True,171 return_tensors="pt"172 )173 174 # Generate175 outputs = model.generate(176 **inputs.to(model.device),177 max_new_tokens=128,178 pad_token_id=tokenizer.eos_token_id179 )180 181 # Decode182 result = tokenizer.decode(183 outputs[0][len(inputs["input_ids"][0]):],184 skip_special_tokens=False185 )186 187 return result188 189# Example usage190query = "My app crashes when I try to upload large files"191result = route_query(query)192print(f"Query: {query}")193print(f"Routing: {result}")194```195 196### Expected Output Format197 198```199<start_function_call>call:technical_support_agent{issue_type:crash,priority:high}<end_function_call>200```201 202## 💡 Usage Examples203 204### Example 1: Technical Issue205```python206query = "I'm getting a 500 error when calling the API"207result = route_query(query)208# Output: technical_support_agent(issue_type="api_error", priority="high")209```210 211### Example 2: Billing Request212```python213query = "I need a refund for my annual subscription"214result = route_query(query)215# Output: billing_agent(request_type="refund", urgency="medium")216```217 218### Example 3: Product Question219```python220query = "What integrations do you support for project management?"221result = route_query(query)222# Output: product_info_agent(query_type="integrations", category="project_management")223```224 225## 🔧 Advanced Usage: Parse Function Calls226 227```python228def parse_function_call(output: str) -> dict:229 """Extract function name and arguments from model output"""230 231 pattern = r'<start_function_call>call:(\w+)\{([^}]+)\}<end_function_call>'232 match = re.search(pattern, output)233 234 if match:235 func_name = match.group(1)236 params_str = match.group(2)237 238 # Parse parameters239 params = {}240 param_pattern = r'(\w+):(?:<escape>(.*?)<escape>|([^,{}]+))'241 for p_match in re.finditer(param_pattern, params_str):242 key = p_match.group(1)243 val = p_match.group(2) or p_match.group(3).strip()244 params[key] = val245 246 return {247 "agent": func_name,248 "parameters": params249 }250 251 return {"agent": "unknown", "parameters": {}}252 253# Use it254query = "I was charged twice this month"255result = route_query(query)256parsed = parse_function_call(result)257print(parsed)258# Output: {'agent': 'billing_agent', 'parameters': {'request_type': 'dispute', 'urgency': 'high'}}259```260 261## 🏗️ Integration Example262 263```python264class MultiAgentRouter:265 def __init__(self, model_name: str):266 self.tokenizer = AutoTokenizer.from_pretrained(model_name)267 self.model = AutoModelForCausalLM.from_pretrained(268 model_name,269 device_map="auto",270 torch_dtype="auto"271 )272 self.system_msg = "You are an intelligent routing agent..."273 274 def route(self, query: str) -> dict:275 """Route query and return agent + parameters"""276 messages = [277 {"role": "developer", "content": self.system_msg},278 {"role": "user", "content": query}279 ]280 281 inputs = self.tokenizer.apply_chat_template(282 messages,283 tools=AGENT_TOOLS,284 add_generation_prompt=True,285 return_dict=True,286 return_tensors="pt"287 )288 289 outputs = self.model.generate(290 **inputs.to(self.model.device),291 max_new_tokens=128,292 pad_token_id=self.tokenizer.eos_token_id293 )294 295 result = self.tokenizer.decode(296 outputs[0][len(inputs["input_ids"][0]):],297 skip_special_tokens=False298 )299 300 return parse_function_call(result)301 302# Usage303router = MultiAgentRouter("bhaiyahnsingh45/functiongemma-multiagent-router")304routing = router.route("My payment failed but I don't know why")305print(f"Route to: {routing['agent']}")306print(f"Parameters: {routing['parameters']}")307```308 309## 📈 Evaluation310 311The model was evaluated on a held-out test set of 23 queries:312 313- **Routing Accuracy**: 82.6%314- **False Positive Rate**: 17.4%315- **Average Inference Time**: ~50ms on T4 GPU316 317## ⚠️ Limitations318 3191. **Language**: Currently supports English only3202. **Domain**: Optimized for customer support; may need fine-tuning for other domains3213. **Agents**: Limited to 3 agent types (can be extended with additional training)3224. **Context**: Works best with single-turn queries; multi-turn conversations may need context handling3235. **Edge Cases**: Ambiguous queries may require fallback logic324 325## 🔮 Future Improvements326 327- [ ] Add support for more languages328- [ ] Expand to 5+ agent types (sales, feedback, onboarding)329- [ ] Handle multi-turn conversations330- [ ] Add confidence scores for routing decisions331- [ ] Support for compound queries requiring multiple agents332 333## 📝 Citation334 335```bibtex336@misc{functiongemma_multiagent_router,337 author = {Bhaiya Singh},338 title = {Multi-Agent Router: Fine-tuned FunctionGemma for Customer Support},339 year = {2025},340 publisher = {Hugging Face},341 howpublished = {\url{https://huggingface.co/bhaiyahnsingh45/functiongemma-multiagent-router}}342}343```344 345## 📄 License346 347This model inherits the [Gemma License](https://ai.google.dev/gemma/terms) from the base model.348 349## 🙏 Acknowledgments350 351- Base model: [google/functiongemma-270m-it](https://huggingface.co/google/functiongemma-270m-it)352- Training framework: [Hugging Face TRL](https://github.com/huggingface/trl)353- Dataset: [bhaiyahnsingh45/multiagent-router-finetuning](https://huggingface.co/datasets/bhaiyahnsingh45/multiagent-router-finetuning)354 355## 📧 Contact356 357For questions, issues, or collaboration opportunities:358- Open an issue on the [model repository](https://huggingface.co/bhaiyahnsingh45/functiongemma-multiagent-router)359- Dataset issues: [dataset repository](https://huggingface.co/datasets/bhaiyahnsingh45/multiagent-router-finetuning)360 361---362 363**Built with ❤️ using FunctionGemma and Hugging Face Transformers**364