maziyaramini/snappfood-sentiment-refined
Refined SnappFood Persian Sentiment Dataset This dataset contains 13,000 Persian food-delivery reviews from the publicly available SnappFood sentiment dataset, with labels refined using an LLM-assisted annotation procedure described in: Improving Persian Sentiment Classification through LLM-Assisted Label Refinement and Small Language Models Dataset Summary The released dataset contains: Split Positive Negative Total Train 5,000 5,000 10,000… See the full description on the dataset page: https://huggingface.co/datasets/maziyaramini/snappfood-sentiment-refined.
Refined SnappFood Persian Sentiment Dataset
This dataset contains 13,000 Persian food-delivery reviews from the publicly available SnappFood sentiment dataset, with labels refined using an LLM-assisted annotation procedure described in:
Improving Persian Sentiment Classification through LLM-Assisted Label Refinement and Small Language Models
Dataset Summary
The released dataset contains:
The dataset is balanced with respect to the refined labels.
Source Dataset
The reviews originate from the publicly available SnappFood Persian sentiment dataset.
The final subset was constructed by initially sampling reviews from both original sentiment classes, applying LLM-assisted label refinement, and subsequently sampling and refining additional reviews to obtain a final balanced set of 6,500 refined-positive and 6,500 refined-negative examples.
Label Refinement
Labels were refined using the OpenAI API with the gpt-5-mini model alias.
Configuration:
- Model:
gpt-5-mini - Output labels:
positiveandnegative - Same refinement prompt applied to all examples
The exact refinement prompt and implementation are available in the accompanying GitHub repository:
https://github.com/maziyaramini/persian-sentiment-llm-refinement
Annotation Quality
Among the final 13,000 examples, 1,635 labels differed from the original SnappFood labels.
A stratified manual audit of 500 examples was conducted by two native Persian speakers. The audit included 250 changed-label examples and 250 unchanged-label examples.
The two annotators achieved 98.6% raw agreement with Cohen's $\kappa = 0.972$.
The refined labels are therefore treated as a silver-standard resource rather than gold-standard annotations.
Dataset Structure
Each split contains the review text and its refined binary sentiment label.
The released dataset does not include internal annotation columns that were not part of the published study.
Intended Use
The dataset is intended for research on:
- Persian sentiment analysis
- Persian NLP
- sentiment classification
- label-noise reduction
- LLM-assisted annotation
- small language model fine-tuning
Limitations
The dataset consists exclusively of Persian food-delivery reviews and therefore may not generalize to other Persian sentiment domains.
The labels are LLM-refined silver-standard labels rather than fully human-adjudicated gold-standard annotations. Users should take this into account when using the dataset for evaluation.
Because the label refinement was performed using a hosted proprietary model, exact regeneration of the original annotation process may not be possible if the underlying model changes or becomes unavailable.
Related Resources
- Code and label-refinement prompt:
https://github.com/maziyaramini/persian-sentiment-llm-refinement