MohamedGomaa30/EGYSpeak

EGYSpeak

A curated dataset of 147,979 single-speaker Egyptian Arabic (pure dialect) audio clips with transcriptions, sourced from the fadisarwat/egyptian-arabic-lines Kaggle dataset and processed through a rigorous ASR pipeline.

Quick Start

1. Download the dataset:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="MohamedGomaa30/EGYSpeak",
    repo_type="dataset",
    local_dir="EGYSpeak",
)

2. Extract the dataset:

from EGYSpeak.egyspeak_loader import load_egyspeak
ds = load_egyspeak("EGYSpeak", streaming=True) 

Dataset Details

• —Total samples: 147,979
• —Audio format: WAV (PCM_16, 16 kHz, mono)
• —Transcription language: Egyptian Arabic
• —Metadata delimiter: Pipe (|)
• —License: CC-BY-4.0

Dataset Structure

EGYSpeak/
├── README.md
├── metadata.csv                  # file_name|transcription (pipe-delimited, full dataset)
├── metadata.jsonl                # HF AudioFolder auto-loading format
└── train/
    ├── metadata.jsonl            # HF AudioFolder auto-loading format
    └── audio_shard_XXXX.tar      # 23 tar shards (~1GB each, 147,979 WAV files total)

Audio files are packed into 23 tar shards (~1GB each) to stay within HuggingFace Hub rate limits. Each shard contains WAV files under a wavs/ subdirectory. Use the provided egyspeak_loader.py to stream audio directly from the tars without extracting (saves ~22GB disk space).

metadata.csv Format

Delimiter: pipe (`|`)

metadata.jsonl Format

Each line is a JSON object with file_name and transcription fields, compatible with HuggingFace AudioFolder auto-loading.

Data Preprocessing Pipeline

The raw Kaggle dataset (455,794 index entries, 408,059 MP3 files) was processed through the following pipeline:

Step 1: Duration Filter

• —Reject audio shorter than 0.8s or longer than 30s

Step 2: Audio Processing

• —Resample to 16 kHz mono
• —Loudness normalization to -16 LUFS (using pyloudnorm)
• —Save as PCM_16 WAV
• —Create padded version (0.5s silence on each side for ASR)
• —Reject completely silent files

Step 3: Speaker Diarization

• —Model: pyannote/speaker-diarization-3.1
• —Accept only single-speaker audio; reject multi-speaker clips

Step 4: ASR Transcription

• —Model: NAMAA-Space/EgypTalk-ASR-v2 (FastConformer Hybrid Large, fine-tuned on 200+ hours of Egyptian Arabic)
• —Decoder: Beam search (beam_size=5)
• —Batch processing: 64 files per batch

Step 5: Quality Filters

Five sequential quality filters are applied to transcriptions:

Configuration

Limitations

• —Transcriptions are machine-generated (ASR) and may contain errors
• —The original Kaggle text field was used as a reference only, not as ground truth
• —Numbers appear as Arabic words, not numeric digits
• —Some audio clips may contain overlapping speech fragments at boundaries

Source

Kaggle: fadisarwat/egyptian-arabic-lines

License

This dataset is released under the Creative Commons Attribution 4.0 license.