IbrahimDayax/somali-combined-asr-stt-dataset

Somali Combined ASR/STT Dataset

A Somali automatic speech recognition (ASR) / speech-to-text (STT) dataset combining synthetic TTS-generated audio and other Somali speech sources, deduplicated by transcript text and split into train/validation/test.

Dataset Summary

• —Language: Somali (so)
• —Task: Automatic Speech Recognition / Speech-to-Text
• —Audio format: WAV, 16 kHz mono
• —Total examples: 8,226 (after deduplication)
• —Total audio: ~6 hours

Source breakdown (post-dedup)

Deduplication

The original combined collection (jelle8000, jelle_data, somali_tts, soomali_asr) contained 80,164 (text, audio) pairs with an 86.9% duplication rate — jelle_data (50,000 samples) turned out to be almost entirely repeated renderings of the same 997 unique texts already present in jelle8000. This release keeps only the deduplicated set (10,504 unique transcripts), with 2,278 text-only rows dropped because their upstream audio generation had failed (no audio file), leaving 8,226 examples with valid audio.

Usage

from datasets import load_dataset

ds = load_dataset("IbrahimDayax/somali-combined-asr-stt-dataset")
sample = ds["train"][0]
print(sample["text"])
sample["audio"]  # {"array": ..., "sampling_rate": 16000}

Note: decoding audio via the datasets library's Audio feature requires torchcodec (pip install torchcodec) plus a compatible FFmpeg. If you'd rather avoid that dependency, read the raw WAV bytes directly with soundfile: ``python import io, soundfile as sf raw = ds["train"].with_format(None)[0]["audio"]["bytes"] data, sr = sf.read(io.BytesIO(raw)) ``

Columns

• —id: example identifier
• —audio: audio data (16 kHz WAV)
• —text: Somali transcript
• —text_length: character length of text
• —source: originating sub-dataset (jelle8000, soomali_asr, or somali_tts)

License

Not specified by the source data — please update this section with your intended license.