MR3z4/persian-accents-benchmark

Persian Accents Benchmark

Dataset Summary

A benchmark for Persian automatic speech recognition (ASR): 279 short utterances of informal Persian (Farsi) dialect speech across 16 regional accents, released as a fixed evaluation set. Total audio duration is approximately 4.4 hours. The primary label is the transcription; each utterance also carries an accent label (usable for accent classification as a secondary task) and an emotion label as auxiliary metadata.

This is an evaluation benchmark, not a training corpus — no train/validation/test splits are provided by design.

Dataset Structure

Data Fields

Splits

Fixed evaluation set: 279 rows in metadata.csv, one mp3 per row, organized into 16 accent folders. No train/validation/test split is provided — the benchmark is meant to be evaluated as-is.

Accent Distribution

Benchmark Usage

• —Primary task (ASR): transcribe each utterance. Suggested metrics: WER/CER. Transcripts are informal dialect text, so standard Persian normalization may be needed before scoring.
• —Secondary task (audio classification): classify the accent of each utterance (16 classes). Suggested metrics: per-accent accuracy, macro-averaged F1, and a confusion matrix — per-accent numbers matter because of the heavy class imbalance.
• —Load with load_dataset("audiofolder", data_dir="..."); transcription is the ASR label, accent the classification label.

Dataset Creation

Speech was collected from informal dialect speakers. Emotion was annotated as a one-hot vector and collapsed to a single label; utterances with no emotion annotation are labeled unknown. Cleaning dropped 23 files: 5 corrupted (near-zero duration) files and 22 untranscribed files (4 files fall in both categories).

Bias, Risks, and Limitations

• —Heavy class imbalance: qazvini has 1 file while balochi has 44 and gilani has 42. Aggregate accuracy will be dominated by majority accents; report per-accent metrics.
• —Informal transcripts: transcripts are informal, dialectal, and often ungrammatical; they are not standard written Persian.
• —Speaker overlap: 123 source files are flagged as containing speaker overlap, which may affect ASR alignment quality.
• —Corrupted flags: 6 released files carry corrupted=1 (flagged in the source labels but not among the near-zero-duration broken files); downstream users may want to filter them.

Citation

@misc{persian_accents_benchmark,
  author       = {Saemi, Payam and Mohammadzade, MohammadReza},
  title        = {Persian Accents Benchmark},
  year         = {2026},
  publisher    = {Hugging Face},
  journal      = {Hugging Face Hub},
  howpublished = {\url{https://huggingface.co/datasets/MR3z4/persian-accents-benchmark}}
}

Contact

For questions or issues, contact Payam Saemi at payamsaemi@gmail.com or MohammadReza Mohammadzade at mohammadzade.m.r@gmail.com.