Datasetpublic

ghananlpcommunity/voxcpm2-ghana-speech-ipa-latents

VoxCPM2 Ghana — Precomputed AudioVAE Latents Precomputed VoxCPM-2 AudioVAE latents for a Ghanaian multilingual TTS fine-tune, ready for training with the official train_voxcpm_finetune.py (train_manifest: ghana-latents). No audio decoding or VAE encoding needed at train time — the latent feat is fed straight to the VoxCPM-2 model with the IPA transcript. Each language is a dataset subset: from datasets import load_dataset ds =… See the full description on the dataset page: https://huggingface.co/datasets/ghananlpcommunity/voxcpm2-ghana-speech-ipa-latents.

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

0likes369downloads

Dataset Card

VoxCPM2 Ghana — Precomputed AudioVAE Latents

Precomputed VoxCPM-2 AudioVAE latents for a Ghanaian multilingual TTS fine-tune, ready for training with the official train_voxcpm_finetune.py (train_manifest: ghana-latents). No audio decoding or VAE encoding needed at train time — the latent feat is fed straight to the VoxCPM-2 model with the IPA transcript.

Each language is a dataset subset:

python

from datasets import load_dataset
ds = load_dataset("ghanaopendata/voxcpm2-ghana-speech-ipa-latents", "Akuapem_Twi_twi", split="train")

—369,347 clips · ~747 h across 42 subsets
—42 Ghanaian languages (Akuapem Twi, Asante Twi, Anyin, Avatime, Bassar-Ntcham, Bimoba, Birifor-Southern, Bissa, Buli, Chumburung, Dagaare, Dagbani, Dangme, Deg, Ewe, Fante, Fulfulde-Maasina, Gikyode, Gonja, Hausa, Kabiye, Kasem, Konkomba, Konni, Kusaal, Lelemi, Mampruli, Nawuri, Ninkare, Nkonya, Ntrubo, Nzema, Paasaal, Sehwi, Sekpele, Selee, Sisaala-Tumulung, Siwu, Tampulma, Tem, Tuwuli, Vagla)
—AudioVAE from VoxCPM-2: 64-dim, hop 640 (~25 fps)

Subsets (config_name = language code)

—Akuapem_Twi_twi
—Anyin_any
—Asante_Twi_twi
—Avatime_avn
—Bassar_Ntcham_bud
—Bimoba_bim
—Birifor_Southern_biv
—Bissa_bib
—Buli_bwu
—Chumburung_ncu
—Dagaare_dga
—Dagbani_dag
—Dangme_ada
—Deg_mzw
—Ewe_ewe
—Fante_fat
—Fulfulde_Maasina_ffm
—Gikyode_acd
—Gonja_gjn
—Hausa_hau
—Kabiye_kbp
—Kasem_xsm
—Konkomba_xon
—Konni_kma
—Kusaal_kus
—Lelemi_lef
—Mampruli_maw
—Nawuri_naw
—Ninkare_gur
—Nkonya_nko
—Ntrubo_ntr
—Nzema_nzi
—Paasaal_sig
—Sehwi_sfw
—Sekpele_lip
—Selee_snw
—Sisaala_Tumulung_sil
—Siwu_akp
—Tampulma_tpm
—Tem_kdh
—Tuwuli_bov
—Vagla_vag

Phonemization

Transcripts in text are IPA phonemes (space-separated, punctuation kept).

—The 42 Ghanaian languages were phonemized with `GhanaNLP/ghana-g2p` (via the ghana-speech-IPA pipeline).
—English is phonemized with `GhanaNLP/ghana-english-g2p` (curated lexicon + espeak fallback) and published separately at ghanaopendata/voxcpm2-ghana-english-ipa-latents.

Inference note: a TTS fine-tuned on these latents was trained on IPA text. To synthesize new utterances, phonemize the input text into the same space-separated IPA convention (ghana-g2p for Ghanaian languages, ghana-english-g2p for English) and pass that IPA string to the model. Note the training labels are ASR-read pronunciations while a g2p phonemizer produces predicted ones, so the two conventions agree on inventory but can differ on individual words.

Format (each subset: parquet shards `CODE__*.parquet`)

column	type	notes
`id`	string	clip id, joinable back to the source audio dataset (a few source shards repeat ids, so it is not a unique key)
`feat`	binary	fp16 AudioVAE latent, shape `[feat_t, 64]`. Reconstruct: `np.frombuffer(feat, np.float16).reshape(feat_t, 64)`
`feat_t`	int32	number of latent frames (~25 per second)
`text`	string	IPA phoneme transcript (space-separated, punctuation kept) — the training input
`raw_text`	string	the original written transcript in the language's orthography
`text_ids`	list[int32]	Llama token ids of `text` (informational; train re-tokenizes)
`dataset_id`	int32	language id (sorted position of `CODE` among the Ghanaian configs)
`split`	string	`train` / `dev` (first 40 clips per language)

text holds IPA rather than orthography because the finetune script re-tokenizes text directly. raw_text keeps the orthography available for inspection, filtering, and training a grapheme model on the same latents.

Source

Derived from the ghana-speech audio parquets and their ghana-speech-IPA phonemizations, capped at 20 h per language.

How the IPA was produced

Read off the audio by the CTC phoneme recogniser in `GhanaNLP/ghana-ipa-asr` (ghana-ipa-asr batch), feeding the encoder the per-utterance normalised waveforms it was trained on (normalize_audio: true, statistics over valid samples only). The front-end was validated against the reference int8 decoder at 1.8% UER before the run.

Transcripts published before 2026-08-12 were produced without that normalisation, which costs ~12.2% UER against ~1.7%. If you pulled this dataset earlier, re-pull.

Clips whose ASR output is empty or under 2 phonemes/second are excluded (8.3%). Those are source-level segmentation failures rather than recogniser failures — their written transcript does not match the audio length either (1.14 words/sec against 2.95 for the rest).

Pipeline: `GhanaNLP/ghana-speech-english-ipa-latents-data-prep`