Datasetpublic

Professor/dholuo-speech-data

Dholuo Speech Data (Pooled) A ~191.5-hour Dholuo (Luo) speech corpus, pooled from two independent sources and filtered to only genuinely transcribed audio. Part of the AfroNet multi-language TTS data effort. Sources Anv-ke/Dholuo — African Next Voices, a pilot data-collection effort in Kenya led by the KenCorpus Consortium (a coalition of Kenyan universities and research centers), funded by the Gates Foundation. 91,672 clips, 186.1h, source = anv_ke. Gated on… See the full description on the dataset page: https://huggingface.co/datasets/Professor/dholuo-speech-data.

sourceHugging Facecc-by-4.0updated 1mo agoView on Hugging Face

0likes62downloads

Dataset Card

Dholuo Speech Data (Pooled)

A ~191.5-hour Dholuo (Luo) speech corpus, pooled from two independent sources and filtered to only genuinely transcribed audio. Part of the AfroNet multi-language TTS data effort.

Sources

—**Anv-ke/Dholuo** — African Next Voices, a pilot data-collection effort in Kenya led by the KenCorpus Consortium (a coalition of Kenyan universities and research centers), funded by the Gates Foundation. 91,672 clips, 186.1h, source = anv_ke. Gated on HuggingFace — an account must click through the access request on the dataset page before the API can download it.
—Mozilla Common Voice 26.0, Dholuo (luo) locale, accessed via the Mozilla Data Collective mirror (datacollective Python SDK). train split only. 4,033 clips, 5.3h, source = common_voice. Crowdsourced, community-validated read speech, CC0-1.0.

These two sources are independent (different collection institutions and methodologies), so no overlap/deduplication concern applies here, unlike some other sources in this collection.

Structure and what "transcribed" means here

The source splits data into train/dev/dev_test; only `train` is used here (dev/dev_test are held-out evaluation partitions, same policy we apply to DSN's splits for the Nigerian-language releases).

Within train, two categories are pooled together:

—Scripted — read from a prepared script, 100% transcribed by construction. Each row also ships an English translatedText alongside the native transcript (not included in this release's text field, which is native-language only).
—Unscripted — topic-prompted natural speech. The large majority carries a real, reviewed transcript (Anv-ke's own workflow marks each as approved/rejected after review); only transcribed rows are included here.

A text-encoding bug in the source, fixed during ingestion: unscripted transcripts in the raw parquet files are mojibake — UTF-8 bytes that got decoded as Latin-1 somewhere upstream. This is fixed via a encode('latin-1').decode('utf-8') round-trip that's a safe no-op on already-correct text (a genuine non-Latin-1 character can't itself be Latin-1-encoded, so the fix only fires on rows that actually need it) — scripted transcripts, which are correct as shipped, pass through unchanged.

All audio is standardized to 16 kHz mono FLAC (lossless), 1–30 second clips. Source audio is real WAV, embedded directly in the source's parquet files (no WebM-mislabeling issue like some other sources in this collection).

Format

The dataset ships as WebDataset-style tar shards (shards/shard-00000.tar …, ~1 GB each, one {key}.flac file per clip) plus a single manifest (manifest.parquet / manifest.jsonl):

Column	Description
`key`, `shard`	which tar file + entry holds this clip's audio
`text`	transcript (native Dholuo script)
`duration`	seconds
`source`	`anv_ke` or `common_voice`
`dataset_id`	always `0`
`split`	`train` / `val` (250 clips held out for evaluation)
`speaker_id`	Anv-ke's `recorder_uuid`, or Common Voice's `client_id`
`gender`	speaker metadata where available
`domain`	e.g. `scripted/Agriculture and Food`, `unscripted/Education and Technology` (Anv-ke rows only; `null` for Common Voice rows)
`dbfs`, `clip_ratio`, `sil_ratio`	cheap DSP quality proxies: loudness, fraction of clipped samples, fraction of near-silent frames
`has_disfluency`	always `false` — this source doesn't flag disfluencies

Usage

python

from huggingface_hub import hf_hub_download
import pandas as pd, tarfile, io, soundfile as sf

mp = hf_hub_download("Professor/dholuo-speech-data", "manifest.parquet", repo_type="dataset")
df = pd.read_parquet(mp)

row = df.iloc[0]
shard_path = hf_hub_download("Professor/dholuo-speech-data", f"shards/{row.shard}", repo_type="dataset")
with tarfile.open(shard_path) as tar:
    audio_bytes = tar.extractfile(f"{row.key}.flac").read()
arr, sr = sf.read(io.BytesIO(audio_bytes))

The tar shards are also directly readable by the `webdataset` library for streaming training pipelines.

Intended use & limitations

Built for Dholuo TTS/ASR research, in particular as finetuning data for a multilingual TTS model that doesn't natively support Dholuo. Source recordings cover 2 dialects pooled together (per the upstream dataset card); dialect is not preserved as a separate field in this release. This is a research aggregation; usage should respect African Next Voices' own terms.

License

CC BY 4.0 for the Anv-ke portion, per the upstream Anv-ke/Dholuo release; CC0 1.0 for the Common Voice portion, per upstream Common Voice.

Acknowledgments

Deep thanks to the KenCorpus Consortium / African Next Voices for the Anv-ke portion (Gates Foundation-funded), and to Mozilla Common Voice and its Dholuo contributors for the Common Voice portion, hosted via the Mozilla Data Collective.

This dataset was pooled by Victor Olufemi and LyngualLabs as part of the AfroNet multi-language TTS data effort.