Datasetpublic

gttsehu/Albayzin-2024-BBS-S2T

Albayzin 2024 Bilingual Basque-Spanish Speech to Text (BBS-S2T) Challenge see Albayzin_2024_BBS-S2T_EvalPlan for a description of the challenge. NOTE: Test data will be released on September 2nd, 2024. The Albayzin 2024 Bilingual Basque-Spanish Speech to Text (BBS-S2T) Challenge training and tuning set is based on the gttsehu/basque_parliament_1 dataset. The database consists of four splits: train : 749945 audio segments (automatically extracted) train_clean : 661871 audio… See the full description on the dataset page: https://huggingface.co/datasets/gttsehu/Albayzin-2024-BBS-S2T.

sourceHugging Facemitupdated 2y agoView on Hugging Face

0likes33downloads

Dataset Card

Albayzin 2024 Bilingual Basque-Spanish Speech to Text (BBS-S2T) Challenge

see [Albayzin_2024_BBS-S2T_EvalPlan](Albayzin_2024_BBS-S2T_EvalPlan.pdf) for a description of the challenge.

NOTE: Test data will be released on September 2nd, 2024.

The Albayzin 2024 Bilingual Basque-Spanish Speech to Text (BBS-S2T) Challenge training and tuning set is based on the gttsehu/basque_parliament_1 dataset. The database consists of four splits:

1.train : 749945 audio segments (automatically extracted)
2.train_clean : 661871 audio segments (automatically extracted, highly reliable transcriptions)
3.dev : 4095 audio segments (manually validated)
4.test : 5152 audio segments (manually validated)

How to download the basque_parliament_1 database

1 - If you can handle yourself comfortably with Huggingface Datasets:

from datasets import load_dataset
ds = load_dataset("gttsehu/basque_parliament_1")

The Dataset contains four splits:

DatasetDict({
    train: Dataset({
        features: ['path', 'audio', 'sentence', 'speaker_id', 'language', 'PRR', 'length'],
        num_rows: 749945
    })
    train_clean: Dataset({
        features: ['path', 'audio', 'sentence', 'speaker_id', 'language', 'PRR', 'length'],
        num_rows: 661871
    })
    validation: Dataset({
        features: ['path', 'audio', 'sentence', 'speaker_id', 'language', 'PRR', 'length'],
        num_rows: 4095
    })
    test: Dataset({
        features: ['path', 'audio', 'sentence', 'speaker_id', 'language', 'PRR', 'length'],
        num_rows: 5152
    })
})

NOTE: The validation split corresponds with the dev split of this challenge.

2 - Manual download:

git clone https://huggingface.co/datasets/gttsehu/basque_parliament_1

NOTE: git-lfs must be installed to be able to handle the download of the large tar files (which include the audio files).

Downloaded database structure:

basque_parliament_1/
├── audio
│   ├── dev_0.tar
│   ├── test_0.tar
│   ├── train_0.tar
│   ├── train_10.tar
│   ├── train_1.tar
│   ├── train_2.tar
│   ├── train_3.tar
│   ├── train_4.tar
│   ├── train_5.tar
│   ├── train_6.tar
│   ├── train_7.tar
│   ├── train_8.tar
│   └── train_9.tar
├── basque_parliament_1.py
├── languages.py
├── metadata
│   ├── dev.tsv
│   ├── test.tsv
│   ├── train_clean.tsv
│   └── train.tsv
├── README.md
└── release_stats.py

Untar all audio files:

ls basque_parliament_1/audio/*.tar | xargs -i tar -xC basque_parliament_1/audio -f {}

The metadata directory contains the index files for the 4 splits. Each index file contains five tab separated fields:

1.The audio file path
2.The language of the segment (es: spanish, eu: basque and bi: bilingual)
3.The speaker id
4.The PhoneRecognitionRate indicating the quality of the transcription
5.The length of the segment (in seconds)
6.The transcription

path    language        speaker_id      PRR     length  sentence
10-007_20130124_01/10-007_20130124_01_83.92_93.84.mp3   eu      0       100.00  9.92    egun on guztioi bilkurari hasiera emango diogu gai zerrendako lehenengo puntua bateraezintasunen
10-007_20130124_01/10-007_20130124_01_95.49_105.34.mp3  eu      416     100.00  9.85    euskadiren izeneko senatari izendatzeko hautagaien bateragarritasun egoerari buruz eztabaida eta behin betiko ebazpena eta hala badagokio senatariak
10-007_20130124_01/10-007_20130124_01_105.35_112.10.mp3 eu      416     98.46   6.75    hautatzeko botazioa batzordeko kidearen batek irizpidearen alde hitz egin nahi du
10-007_20130124_01/10-007_20130124_01_117.61_127.29.mp3 eu      416     100.00  9.68    aurka hitz egin nahi du bost minutuko txanda daukazue eta mistoa upyd hasiko da maneiro
10-007_20130124_01/10-007_20130124_01_149.82_160.12.mp3 es      290     100.00  10.30   buenos días a todas y a todos utilizo este turno para alzar la voz ante la pretensión de eh bildu de que
...