rescommons/agent-orchestration-intents-dataset

Agent Orchestration Dataset

A dataset for training and evaluating intelligent orchestrator models that route user requests to specialist agents with high-level intent classification.

Dataset Description

This dataset contains conversational examples where an orchestrator must analyze user requests and determine which specialist agents should handle them. Each sample includes intent labels for additional classification tasks.

Key Features:

• —Multi-agent routing scenarios
• —High-level intent classification (10 categories)
• —Customer service and HR/leave management contexts
• —Function-calling format for agent orchestration

Splits

Structure

{
    "messages": [
        {"role": "system", "content": "You are an intelligent orchestrator..."},
        {"role": "user", "content": "I'm incredibly frustrated! I've been charged twice..."},
        {"role": "model", "content": "<start_function_call>call:route_to_agents{...}<end_function_call>"}
    ],
    "ground_truth_agents": ["intent_and_sentiment_extraction_agent", "request_validation_agent", "duplicate_detection_agent"],
    "intent": "billing_issue"
}

Intent Categories

The dataset includes 10 high-level intent categories:

Available Agents

Customer Service Agents

• —request_validation_agent
• —duplicate_detection_agent
• —case_creation_agent
• —informational_queries_agent
• —transactional_query_responder_agent
• —intent_and_sentiment_extraction_agent
• —entity_extraction_agent
• —document_verification_agent
• —email_agent

HR/Leave Management Agents

• —user_information_retriever_agent
• —balance_checking_agent
• —leave_approval_agent

Usage

from datasets import load_dataset

# Load the dataset
dataset = load_dataset("V1rtucious/agent-orchestration-intents-dataset")
train_data = dataset["train"]
test_data = dataset["test"]

# Access a sample
sample = train_data[0]
print(f"Intent: {sample['intent']}")
print(f"User message: {sample['messages'][1]['content']}")
print(f"Routed agents: {sample['ground_truth_agents']}")

Intent Classification

# Extract user messages and intents for classification
user_messages = []
intents = []

for sample in train_data:
    user_msg = next(msg['content'] for msg in sample['messages'] if msg['role'] == 'user')
    user_messages.append(user_msg)
    intents.append(sample['intent'])

# Train your intent classifier
from transformers import AutoTokenizer, AutoModelForSequenceClassification
# Your training code here

Agent Routing

# Train an orchestrator model
for sample in train_data:
    system_prompt = sample['messages'][0]['content']
    user_request = sample['messages'][1]['content']
    agents_to_call = sample['ground_truth_agents']
    
    # Your orchestrator training logic here

Use Cases

• —Intent Classification: Categorize user requests into high-level intents
• —Agent Routing: Build orchestrator systems that route requests to specialist agents
• —Multi-agent Coordination: Develop systems that coordinate multiple AI agents
• —Customer Service Automation: Create intelligent customer support routing
• —Function-calling: Train models for tool/function calling in orchestration scenarios

Dataset Creation

Intent labels were added through pattern-based classification using keyword matching and semantic analysis, ensuring:

• —Stable categories: High-level intents resistant to minor variations
• —Balanced distribution: Reasonable coverage across categories
• —Clear boundaries: Each intent has a distinct purpose
• —Comprehensive coverage: All major user request types represented

Limitations

• —Focused on customer service and employee leave management scenarios
• —English language only
• —Intent categories are broad and may need sub-categorization for specific applications
• —Some samples in general_inquiry could be further categorized

License

MIT License