ghanaopenai/new-twi-tts-aligned-ipa

new-twi-tts-aligned + IPA phonemes

`ghanaopendata/new-twi-tts-aligned` with a machine-generated IPA phoneme transcription for every clip, produced with `ghananlpcommunity/ghana-speech-phoneme-asr`.

Audio included — this is self-contained, no join with the source dataset needed.

Contents

Columns

Loading it

from datasets import load_dataset

ds = load_dataset("ghanaopendata/new-twi-tts-aligned-ipa", split="train")
row = ds[0]
row["audio"]["array"]      # 24 kHz mono waveform
row["text"]                # orthographic Twi
row["ipa"]                 # "n a n s o pʰ e tʰ o ɾ o ..."
row["ipa"].split(" ")      # phoneme units — see the warning below

Read ipa, not the characters

Many units are multi-character — kʰ, t͡ʃ, k͡p, hʷ, iː. The inventory is 172 units, not 172 characters. Split on spaces:

units = row["ipa"].split(" ")     # correct
units = list(row["ipa"])          # wrong — tears k͡p into three characters

How it was made

Greedy CTC decoding of the fairseq2 checkpoint in bf16 on a single H200, in length-sorted batches — 172 hours in about 14 minutes (~780x realtime). Code, including the validation harness that checks the audio front-end against the reference decoder: https://github.com/GhanaNLP/phoneme-asr-batch

The audio is copied bit-for-bit from the source dataset — never decoded and re-encoded — so these are exactly the waveforms the phonemes were derived from.

Speaker labels and QC flags

The source is Ghanaian news broadcast audio with no speaker column, and an ECAPA-TDNN check confirms it is heavily multi-speaker: 505 sampled clips fragment into 214 clusters at cosine 0.7 with no dominant voice. Training a single-speaker model on that yields an averaged, unstable timbre, so pseudo-speaker labels are provided.

All 161,398 clips were ECAPA-embedded and clustered (k-means over-segmentation into 4,000 centroids, then average-linkage agglomeration of the centroids at cosine >= 0.70), giving 1,427 pseudo-speakers, of which 210 have at least 20 clips and cover 98.5% of the corpus. The ten largest hold 41% between them, so several voices have 7-12 hours each.

These are derived labels, not ground truth. Over-splitting one real speaker into two ids costs a TTS model almost nothing; merging two real speakers is what muddies a voice, so the threshold errs toward splitting.

QC rules

qc_pass is true for 151,488 of 161,398 clips (93.9%), 163.72 h. Nothing is deleted — rows are flagged so you can apply your own cut, since "bad" is architecture-dependent (a 25-second clip is unusable for Piper and fine for F5-TTS).

train = load_dataset("ghanaopendata/new-twi-tts-aligned-ipa", split="train")
train = train.filter(lambda r: r["qc_pass"])

Accuracy, and what to expect

These are model predictions, not verified ground truth. On its own held-out dev set the model scores 17.1% phoneme unit error rate on Asante Twi and 12.4% on Fante. This is a strong starting point for TTS phoneme targets or pronunciation analysis, not a gold lexicon.

Three properties worth knowing before training on it:

• —Punctuation is guessed. The model emits punctuation marks, but they have no acoustic realisation, so its punctuation error rate is high (~30%) even where the phonemes are good. Filter against a known punctuation set if you do not want it.
• —The IPA follows the speech, not the spelling. Where a speaker elides or reduces, the phonemes reflect what was said and will not match a rule-based grapheme-to-phoneme rendering of text. That is the reason to use an acoustic model — and the reason ipa and text will legitimately disagree.
• —Batching adds ~0.2% unit error versus decoding each clip alone, because wav2vec2's convolutional front-end is not padding-masked. Far below the model's own error rate.

The model returned no phonemes for 6 clips (n_units == 0); filter them out if that matters.

License

cc-by-nc-4.0, inherited from the source audio dataset.