MCINext/synthetic-persian-chatbot-conversational-sentiment-analysis-friendship

Dataset Summary

Synthetic Persian Chatbot Conversational SA – Friendship is a Persian (Farsi) dataset created for the Classification task, with a focus on detecting the emotion "friendship" in chatbot conversations. It is part of the FaMTEB (Farsi Massive Text Embedding Benchmark). The dataset was synthetically generated using GPT-4o-mini and is derived from the broader Synthetic Persian Chatbot Conversational Sentiment Analysis dataset.

• —Language(s): Persian (Farsi)
• —Task(s): Classification (Emotion Classification – Friendship)
• —Source: Synthetic, generated using GPT-4o-mini
• —Part of FaMTEB: Yes

Supported Tasks and Leaderboards

This dataset evaluates the performance of text embedding models in detecting the presence and intensity of the "friendship" emotion from Persian chatbot conversations. Models can be benchmarked on the Persian MTEB Leaderboard on Hugging Face Spaces (language filter: Persian).

Construction

The dataset was generated using:

• —175 distinct conversation topics
• —9 user and chatbot tone combinations (e.g., formal, casual, childish)
• —One of 9 predefined user emotions (including friendship)
• —Emotion intensity: neutral, moderate, or high
• —GPT-4o-mini to generate the final conversations

Labeling Strategy:

• —Positive Label: Emotion intensity is moderate or high
• —Negative Label: Emotion intensity is neutral

The FaMTEB paper (Table 1) reports a 93% accuracy on the parent SynPerChatbotConvSAClassification dataset based on human evaluation of LLM-generated emotion labels.

Data Splits

The full parent dataset is split as follows:

• —Train: 4,496 samples
• —Dev: 0 samples
• —Test: 1,499 samples

This friendship-specific subset contains 510 examples. Its specific train/test split is not independently detailed and is assumed to be included in the parent dataset splits.