semantic-router
jailbreak-detection-dataset
Jailbreak Detection Dataset (MLCommons-Aligned)
A comprehensive dataset for training AI safety classifiers, aligned with the MLCommons AI Safety taxonomy.
Dataset Description
This dataset combines multiple sources for robust jailbreak and safety detection:
Primary Sources
nvidia/Aegis-AI-Content-Safety-Dataset-2.0: 18,164 samples with MLCommons-aligned labels
lmsys/toxic-chat: Toxic content detection
jackhhao/jailbreak-classification: Jailbreak attack patterns… See the full description on the dataset page: https://huggingface.co/datasets/llm-semantic-router/jailbreak-detection-dataset.fact-check-classification-dataset
Fact-Check Classification Dataset
🎯 Purpose: Binary classification dataset for determining whether a prompt needs external fact-checking.
Dataset Description
This dataset is designed to train classifiers that can route LLM requests based on whether they require external fact verification. It's part of the vLLM Semantic Router project.
Labels
FACT_CHECK_NEEDED (1): Information-seeking questions requiring external verification
Factual questions about dates… See the full description on the dataset page: https://huggingface.co/datasets/llm-semantic-router/fact-check-classification-dataset.longcontext-haldetect
Long-Context Hallucination Detection Benchmark
A synthetic benchmark dataset for evaluating hallucination detection models on long documents (8K-24K tokens). This dataset is specifically designed to test models that can handle contexts beyond the typical 8K token limit.
Dataset Summary
Property
Value
Total samples
3,366
Token range
8,005 - 23,998
Average tokens
17,852
Hallucinated
1,681 (49.9%)
Supported
1,685 (50.1%)
Splits… See the full description on the dataset page: https://huggingface.co/datasets/llm-semantic-router/longcontext-haldetect.feedback-detector-dataset
Feedback Detector Dataset
A large-scale multilingual dataset for 4-class user feedback classification, labeled using GPT-OSS-120B on AMD MI300X GPU.
Dataset Description
This dataset contains 51,694 examples of user feedback classified into 4 categories:
Label
Description
Count
%
SAT
User is satisfied
8,649
17%
NEED_CLARIFICATION
User needs more information
16,179
31%
WRONG_ANSWER
System gave incorrect response
19,919
39%
WANT_DIFFERENT
User wants… See the full description on the dataset page: https://huggingface.co/datasets/llm-semantic-router/feedback-detector-dataset.mlcommons-ai-safety-synth
MLCommons AI Safety Synthesized Dataset
Synthesized training data for AI safety classifiers based on the MLCommons AI Safety Hazard Taxonomy.
Dataset Description
This dataset contains 12,000 synthesized unsafe prompts across 6 hazard categories, designed to augment training data for content safety classifiers. Each category contains 2,000 balanced samples.
Hazard Categories (MLCommons AI Safety Taxonomy)
Category
Description
Samples… See the full description on the dataset page: https://huggingface.co/datasets/llm-semantic-router/mlcommons-ai-safety-synth.modality-routing-dataset
Modality Routing Dataset
This dataset materializes the dynamic modality routing data builder used by the local
mmBERT-32K modality router training pipeline. The export is intended for review,
versioning, and uploading to a Hugging Face dataset repository.
Labels
Label
ID
Description
AR
0
Text-only requests that should route to an autoregressive LLM.
DIFFUSION
1
Image-generation requests that should route to a diffusion model.
BOTH
2
Requests that benefit… See the full description on the dataset page: https://huggingface.co/datasets/llm-semantic-router/modality-routing-dataset.