Datasetpublic

llm-semantic-router/feedback-detector-dataset

Feedback Detector Dataset A large-scale multilingual dataset for 4-class user feedback classification, labeled using GPT-OSS-120B on AMD MI300X GPU. Dataset Description This dataset contains 51,694 examples of user feedback classified into 4 categories: Label Description Count % SAT User is satisfied 8,649 17% NEED_CLARIFICATION User needs more information 16,179 31% WRONG_ANSWER System gave incorrect response 19,919 39% WANT_DIFFERENT User wants… See the full description on the dataset page: https://huggingface.co/datasets/llm-semantic-router/feedback-detector-dataset.

sourceHugging Faceapache-2.0updated 8mo agoView on Hugging Face

1likes124downloads

Dataset Card

Feedback Detector Dataset

A large-scale multilingual dataset for 4-class user feedback classification, labeled using GPT-OSS-120B on AMD MI300X GPU.

Dataset Description

This dataset contains 51,694 examples of user feedback classified into 4 categories:

Label	Description	Count	%
SAT	User is satisfied	8,649	17%
NEED_CLARIFICATION	User needs more information	16,179	31%
WRONG_ANSWER	System gave incorrect response	19,919	39%
WANT_DIFFERENT	User wants something different	6,947	13%

Splits

Split	Examples
Train	46,524
Validation	5,170

Data Sources

The dataset combines multiple public dialogue and complaint datasets:

Source	Type	Examples	Languages
consumercomplaintsmedium	Complaints	~6,000	English
customer_complaints	Complaints	~2,000	English
multiwoz	Task-Oriented Dialogue	~3,000	English
sgd	Schema-Guided Dialogue	~3,000	English
inscit	Information-Seeking	~2,500	English
mimics	Search Clarification	~2,000	English
hazumi	Affirmative/Negative	~1,500	Japanese
turkish_complaints	Complaints	~1,000	Turkish

Labeling Process

All examples were labeled using OpenAI GPT-OSS-120B served via vLLM on AMD MI300X GPU with:

—Temperature: 0 (deterministic)
—Structured JSON output
—Retry logic with exponential backoff
—Garbage detection and circuit breaker pattern
—Parallel processing with rate limiting

Hardware

Component	Specification
GPU	AMD Instinct MI300X
VRAM	192 GB HBM3
Framework	vLLM with ROCm
Model	OpenAI GPT-OSS-120B

Usage

python

from datasets import load_dataset

dataset = load_dataset("llm-semantic-router/feedback-detector-dataset")

# Access splits
train = dataset["train"]
val = dataset["validation"]

# Example
print(train[0])
# {'text': '...', 'label': 0, 'label_name': 'SAT', 'source': 'multiwoz'}

Schema

Each example contains:

—text (str): The user feedback text
—label (int): Numeric label (0-3)
—label_name (str): Label name (SAT, NEEDCLARIFICATION, WRONGANSWER, WANT_DIFFERENT)
—source (str): Original dataset source

Label Mapping

python

label2id = {
    "SAT": 0,
    "NEED_CLARIFICATION": 1,
    "WRONG_ANSWER": 2,
    "WANT_DIFFERENT": 3
}

Intended Use

—Fine-tuning feedback detection models (ModernBERT, mmBERT, etc.)
—User satisfaction classification in conversational AI
—Customer service automation
—Dialogue system evaluation

Citation

If you use this dataset, please cite:

bibtex

@dataset{feedback_detector_dataset,
  title={Feedback Detector Dataset},
  author={LLM Semantic Router Team},
  year={2025},
  publisher={Hugging Face},
  url={https://huggingface.co/datasets/llm-semantic-router/feedback-detector-dataset}
}

License

Apache 2.0