CoolFace
Datasetpublic

farbodbij/Ganjoor-Rhythm-Bench

English | فارسی Ganjoor-Rhythm-Bench A comprehensive dataset of Persian poems along with ther metrs (وزن عروضی) which is intended to be used for benchmarking LLMs, text classification models or any other model tasked with detecting the Rhythm of a Persian poem. Sourced from the Ganjoor dataset. Configs verses: 2,240,985 mesras, 107 metres. Use this for training or other work. field description verse one hemistich rhythm arkān string poem_id Ganjoor… See the full description on the dataset page: https://huggingface.co/datasets/farbodbij/Ganjoor-Rhythm-Bench.

sourceHugging Faceotherupdated 1mo agoView on Hugging Face
1likes115downloads
Dataset Card

English | فارسی

Ganjoor-Rhythm-Bench

A comprehensive dataset of Persian poems along with ther metrs (وزن عروضی) which is intended to be used for benchmarking LLMs, text classification models or any other model tasked with detecting the Rhythm of a Persian poem. Sourced from the Ganjoor dataset.

Configs

`verses`: 2,240,985 mesras, 107 metres. Use this for training or other work.

fielddescription
verseone hemistich
rhythmarkān string
poem_idGanjoor poem id
poet_idGanjoor poet id
couplet_indexcouplet number within the poem (0-based)
mesra_indexmesra order within the poem (verse_order)

`benchmark` (default): 12,713 couplets, 40 metres. This is the measuring set.

fielddescription
first, secondthe two mesras of the couplet
rhythmarkān string
poet, poet_id, book, poem_id, couplet_indextrace-back only; do not put these in the model prompt

Draw a smaller eval split yourself if 12k couplets is more than you need, e.g. N per metre with a fixed seed.

python
from datasets import load_dataset

bench = load_dataset("farbodbij/Ganjoor-Rhythm-Bench", "benchmark", split="train")
verses = load_dataset("farbodbij/Ganjoor-Rhythm-Bench", "verses", split="train")

How it was built

Scripts and configs that produced this release are in `scripts/`.

  1. 1.`extract_verses.py`: Walks the Ganjoor dataset, keeps poetic verses that have a metre.
  1. 1.`build_benchmark.py`:
  2. 2.drop short mesras
  3. 3.first/last couplets
  4. 4.dropes duplicates
  5. 5.drops rare metres
  6. 6.caps per poet per metre
  1. 1.`probe_leakage.py`: Given the first mesra, ask a set of LLMs for the second, correctly reciting the second mesra indicates a potential leak and memorization of properties related to that poem.

Use in Benchmarking

The main benchmarking task intended for this dataset is metr detection. Pass a single mesra to the model with no clue on poet, poem book or such metadata and ask for the Rhythm.

For measuring how accurate the metr was detected various approaches can be used:

  • —string similarity functions
  • —compare each item in arkan (فاعلاتن vs فعلاتن), assign similarity to each one and judge by that
  • —full-string-match only

This choice is left to the user.

Thanks Disclaimer

We are thankful to Ganjoor team for releasing their meticulously curated data publicly, allowing anyone to deliver high-quality Persian datasets to the NLP community. This both helps Persian as a low-resource language and also contributes to distribution of the Iranian culture.

The metrical labels in this dataset are taken from Ganjoor as-is, with no correction or rewriting. Their accuracy therefore depends on the accuracy of Ganjoor’s data.