Datasetpublic

TigreGotico/bifonia-pt-homographs-wild

bifonia-pt-homographs-wild — real-world OOD evaluation set Real Portuguese sentences containing 112 heterophonic homographs — words spelled identically whose pronunciation (IPA) depends on meaning, e.g. sede (thirst ˈsedɨ vs seat ˈsɛdɨ, both nouns), forma (mould ˈfoɾmɐ vs shape ˈfɔɾmɐ), molho (sauce ˈmoʎu vs bundle ˈmɔʎu). Unlike the synthetic TigreGotico/bifonia-pt-homographs train/test set (LLM-generated), every sentence here is mined from real-world text. It is meant as an… See the full description on the dataset page: https://huggingface.co/datasets/TigreGotico/bifonia-pt-homographs-wild.

sourceHugging Facecc-by-sa-4.0updated 3mo agoView on Hugging Face

0likes15downloads

Dataset Card

bifonia-pt-homographs-wild — real-world OOD evaluation set

Real Portuguese sentences containing 112 heterophonic homographs — words spelled identically whose pronunciation (IPA) depends on meaning, e.g. sede (thirst ˈsedɨ vs seat ˈsɛdɨ, both nouns), forma (mould ˈfoɾmɐ vs shape ˈfɔɾmɐ), molho (sauce ˈmoʎu vs bundle ˈmɔʎu).

Unlike the synthetic `TigreGotico/bifonia-pt-homographs` train/test set (LLM-generated), every sentence here is mined from real-world text. It is meant as an out-of-distribution (OOD) test set — a fair probe of how well a disambiguator generalises beyond synthetic phrasing, for grapheme-to-phoneme / TTS front-ends and word-sense disambiguation.

Schema

field	description
`word`	the ambiguous headword
`sense`	the meaning slug that selects the pronunciation (e.g. `thirst` / `seat`)
`pos`	descriptive part of speech of this reading
`ipa`	European-Portuguese transcription of `word` in this reading
`sentence`	the real sentence containing `word` in that meaning

7,119 sentences, 112 words. The roster was expanded beyond the original 27 with noun/verb and lexical heterophone pairs; the new words were mined the same way (Portuguese Wikipedia) and sense-labelled with Claude Sonnet. Real-world frequency is skewed — common senses dominate and several rare words occur only a few times or not at all. Real-world frequency is skewed: common senses dominate and several minority senses (e.g. the nautical sobre=sail) do not occur at all — which is itself useful signal.

Sources & licensing

Sentences are drawn from:

—Portuguese Wikipedia (CC-BY-SA-4.0) — via the wikimedia/wikipedia dump.
—CommonCrawl — via the cc100 (pt) corpus.

The dataset is released under CC-BY-SA-4.0 to honour Wikipedia's share-alike terms. Attribute Wikipedia and CommonCrawl when redistributing. Text is mixed European and Brazilian Portuguese; the meaning labels are dialect-neutral, while the ipa column follows European Portuguese (pt-PT).

Annotation

Each sentence's sense was labelled by Claude (Sonnet), shown the word's exact sense inventory and asked to choose one meaning or mark it unclear; unclear/invalid labels were dropped. Labels are model-assigned and may contain noise, especially for rare words (e.g. tola n=19, rego n=28, conserto n=62).

Benchmark (sense prediction on this OOD set)

approach	OOD accuracy
most-common (majority sense per word)	47.5%
rule-based (knowledge, no training)	83.2%
Naive-Bayes (corpus-trained)	86.7%
averaged perceptron (corpus-trained)	89.6%

Every approach drops vs the synthetic test set (where the same systems score 94–99%), confirming that synthetic benchmarks overstate accuracy. A corpus-trained perceptron still beats the knowledge-based rules by ~6 points on real text.

Words

acordo, acerto, cerro, choro, colher, começo, conserto, coro, corte, forma, gosto, gozo, jogo, molho, olho, para, pelo, peso, porto, posto, rego, seco, sede, sobre, tola, torre, transtorno.

Citation

bibtex

@misc{bifonia_wild_2025,
  title  = {bifonia-pt-homographs-wild: a real-world OOD set for Portuguese heterophone disambiguation},
  author = {JarbasAI / TigreGotico},
  year   = {2025},
  url    = {https://github.com/TigreGotico/bifonia},
  license = {CC-BY-SA-4.0},
}