Suprio85/Bangla_Speech_Corpus

🎙️ Bengali-Loop: A Long-Form Bangla Speech Corpus

Dataset Summary

Bengali-Loop is a large-vocabulary, long-form Bangla (Bengali) speech corpus designed to push the boundaries of Automatic Speech Recognition (ASR) in low-to-mid resource settings. It comprises 155 hours of naturally occurring Bangla speech sourced from 249 YouTube videos spanning drama serials, audiobooks, and entertainment channels — making it one of the most diverse publicly available Bangla ASR datasets to date.

Unlike most Bangla ASR benchmarks that focus on short utterances (5–15 seconds), Bengali_speecg_corpus is built specifically for long-form, continuous-transcript evaluation, a regime where state-of-the-art models still show significant weaknesses.

Key Statistics

Source Channel Distribution

Recording Duration Distribution

The majority of recordings (54.5%) fall in the 40–60 minute range, making this dataset uniquely suited for long-form speech modeling.

Dataset Structure

Bangla_Speech_Corpus/
├── audio/                  # Raw audio files (.wav or .mp3)
├── subtitles_raw/          # Raw subtitle files (original YT subtitles)
├── transcripts/            # Cleaned, aligned transcripts
├── manifest.jsonl          # Full dataset manifest (audio path + transcript + metadata)
├── video_checklist.csv     # Per-video metadata (channel, duration, source URL, etc.)
└── completed_videos.log    # Processing log

How to Use

Basic Loading with 🤗 Datasets

from datasets import load_dataset

dataset = load_dataset("Suprio85/Bangla_Speech_Corpus")
print(dataset)

Load a Single Split

dataset = load_dataset("Suprio85/Bangla_Speech_Corpus", split="train")

# View a sample
sample = dataset[0]
print(sample["transcript"])

Evaluation with Whisper

from transformers import WhisperProcessor, WhisperForConditionalGeneration
from datasets import load_dataset, Audio
import torch

# Load model
processor = WhisperProcessor.from_pretrained("openai/whisper-large-v3")
model = WhisperForConditionalGeneration.from_pretrained("openai/whisper-large-v3")
model.config.forced_decoder_ids = processor.get_decoder_prompt_ids(language="bengali", task="transcribe")

# Load dataset
dataset = load_dataset("Suprio85/Bangla_Speech_Corpus", split="train")
dataset = dataset.cast_column("audio", Audio(sampling_rate=16_000))

# Transcribe a single sample
sample = dataset[0]
inputs = processor(sample["audio"]["array"], sampling_rate=16_000, return_tensors="pt")
with torch.no_grad():
    predicted_ids = model.generate(inputs.input_features)
transcription = processor.batch_decode(predicted_ids, skip_special_tokens=True)[0]
print("Predicted:", transcription)
print("Reference:", sample["transcript"])

Motivation & Design Philosophy

Most existing Bangla ASR corpora consist of short, isolated utterances recorded in clean studio environments. While valuable for phoneme-level benchmarking, they do not reflect real-world deployment scenarios where:

• —Speech is continuous and multi-minute
• —Background music and overlapping dialogue are present (drama)
• —Speaker styles vary dramatically (expressive acting vs. narration)
• —Vocabulary is rich with colloquial, dialectal, and formal Bangla

Bengali-Loop directly addresses this gap by providing:

1. 1.Long-form audio with full continuous transcripts
2. 2.Natural, in-the-wild speech from broadcast TV and YouTube
3. 3.Diverse channel sources spanning drama, entertainment, and culture

Intended Uses

• —Training and fine-tuning Bangla ASR models
• —Long-form speech recognition benchmarking
• —Bangla language model pretraining (text transcripts)
• —Bangla NLP tasks (NER, POS tagging on transcript text) ---

Limitations

• —Transcripts were derived from YouTube auto-subtitles and may contain minor errors in non-standard speech segments.
• —Audio quality varies across channels; some recordings include background music or audience noise.
• —The corpus is primarily Standard Bangla (শুদ্ধ বাংলা) from Dhaka-dialect drama; dialectal coverage is limited.
• —Speaker metadata (age, gender, region) is not yet annotated.

Citation

If you use Bengali-Loop in your research, please cite:

@dataset{bengali_loop_2025,
  author    = {Suprio85},
  title     = {Bengali-Loop: A Long-Form Bangla Speech Corpus},
  year      = {2025},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/Suprio85/Bangla_Speech_Corpus}
}

Contact

For questions, issues, or collaboration requests, please open an issue on the dataset repository or contact the maintainer via Hugging Face.