CoolFace
Datasetpublic

Natashadonoh/yoruba-diacritic-restoration-dataset

Yorùbá Diacritic Restoration Dataset Prompt-completion pairs for restoring correct diacritical marks (tone marks and underdots) in undiacriticized Yorùbá text. Dataset Details Source: Derived from MENYO-20k, a multi-domain English–Yorùbá parallel corpus (news, TED talks, book excerpts, ICT, proverbs; JW-sourced content excluded) Raw dataset: 8,365 rows, annotated with tone_pattern, harmony_class/harmony_breakdown, and focus_tag columns Adapted dataset: 4,968 rows… See the full description on the dataset page: https://huggingface.co/datasets/Natashadonoh/yoruba-diacritic-restoration-dataset.

sourceHugging Facecc-by-nc-4.0updated 2mo agoView on Hugging Face
0likes14downloads
Dataset Card

task_categories:

  • —text2text-generation ---

Yorùbá Diacritic Restoration Dataset

Prompt-completion pairs for restoring correct diacritical marks (tone marks and underdots) in undiacriticized Yorùbá text.

Dataset Details

  • —Source: Derived from MENYO-20k, a multi-domain English–Yorùbá parallel corpus (news, TED talks, book excerpts, ICT, proverbs; JW-sourced content excluded)
  • —Raw dataset: 8,365 rows, annotated with tone_pattern, harmony_class/harmony_breakdown, and focus_tag columns
  • —Adapted dataset: 4,968 rows, generated via Adaption Labs' adaptation pipeline (Instruction Tuning recipe)

Columns

ColumnDescription
yoruba_strippedYorùbá text with tone marks and underdots removed (prompt / model input)
yorubaCorrectly diacritized Yorùbá text (completion / target)
englishParallel English sentence (from MENYO-20k)
tone_patternPer-vowel High/Mid/Low tone sequence
harmony_class / harmony_breakdownWord-level ±ATR vowel harmony classification
focus_tagHeuristic flag for candidate ni-focus constructions (manual verification )

Task

Given yoruba_stripped as input, restore the correctly diacritized form (yoruba) — a standard diacritic-restoration task in Yorùbá NLP.

License

CC BY-NC 4.0, inherited from the MENYO-20k source corpus.

Author

Areo Mafadesere Natasha