CoolFace
Datasetpublic

facebook/covost2

CoVoST 2, a large-scale multilingual speech translation corpus covering translations from 21 languages into English and from English into 15 languages. The dataset is created using Mozilla’s open source Common Voice database of crowdsourced voice recordings. Note that in order to limit the required storage for preparing this dataset, the audio is stored in the .mp3 format and is not converted to a float32 array. To convert, the audio file to a float32 array, please make use of the `.map()` function as follows: ```python import torchaudio def map_to_array(batch): speech_array, _ = torchaudio.load(batch["file"]) batch["speech"] = speech_array.numpy() return batch dataset = dataset.map(map_to_array, remove_columns=["file"]) ```

sourceHugging Facecc-by-nc-4.0updated 3y agoView on Hugging Face
51likes434downloads
Dataset Card

Dataset Card for covost2

Table of Contents

Dataset Description

  • —Homepage: https://github.com/facebookresearch/covost
  • —Repository: https://github.com/facebookresearch/covost
  • —Paper: https://arxiv.org/abs/2007.10310
  • —Leaderboard: [Needs More Information]
  • —Point of Contact: Changhan Wang (changhan@fb.com), Juan Miguel Pino (juancarabina@fb.com), Jiatao Gu (jgu@fb.com)

Dataset Summary

CoVoST 2 is a large-scale multilingual speech translation corpus covering translations from 21 languages into English \ and from English into 15 languages. The dataset is created using Mozillas open-source Common Voice database of \ crowdsourced voice recordings. There are 2,900 hours of speech represented in the corpus.

Supported Tasks and Leaderboards

speech-translation: The dataset can be used for Speech-to-text translation (ST). The model is presented with an audio file in one language and asked to transcribe the audio file to written text in another language. The most common evaluation metric is the BLEU score. Examples can be found at https://github.com/pytorch/fairseq/blob/master/examples/speechtotext/docs/covost_example.md .

Languages

The dataset contains the audio, transcriptions, and translations in the following languages, French, German, Dutch, Russian, Spanish, Italian, Turkish, Persian, Swedish, Mongolian, Chinese, Welsh, Catalan, Slovenian, Estonian, Indonesian, Arabic, Tamil, Portuguese, Latvian, and Japanese.

Dataset Structure

Data Instances

A typical data point comprises the path to the audio file, usually called file, its transcription, called sentence, and the translation in target language called translation.

{'client_id': 'd277a1f3904ae00b09b73122b87674e7c2c78e08120721f37b5577013ead08d1ea0c053ca5b5c2fb948df2c81f27179aef2c741057a17249205d251a8fe0e658',
 'file': '/home/suraj/projects/fairseq_s2t/covst/dataset/en/clips/common_voice_en_18540003.mp3',
 'audio': {'path': '/home/suraj/projects/fairseq_s2t/covst/dataset/en/clips/common_voice_en_18540003.mp3',
		   'array': array([-0.00048828, -0.00018311, -0.00137329, ...,  0.00079346, 0.00091553,  0.00085449], dtype=float32),
		   'sampling_rate': 48000},
 'id': 'common_voice_en_18540003',
 'sentence': 'When water is scarce, avoid wasting it.',
 'translation': 'Wenn Wasser knapp ist, verschwenden Sie es nicht.'}

Data Fields

  • —file: A path to the downloaded audio file in .mp3 format.
  • —audio: A dictionary containing the path to the downloaded audio file, the decoded audio array, and the sampling rate. Note that when accessing the audio column: dataset[0]["audio"] the audio file is automatically decoded and resampled to dataset.features["audio"].sampling_rate. Decoding and resampling of a large number of audio files might take a significant amount of time. Thus it is important to first query the sample index before the "audio" column, i.e. dataset[0]["audio"] should always be preferred over dataset["audio"][0].
  • —sentence: The transcription of the audio file in source language.
  • —translation: The transcription of the audio file in the target language.
  • —id: unique id of the data sample.

Data Splits

configtrainvalidationtest
en_de2894301553115531
en_tr2894301553115531
en_fa2894301553115531
en_sv-SE2894301553115531
en_mn2894301553115531
en_zh-CN2894301553115531
en_cy2894301553115531
en_ca2894301553115531
en_sl2894301553115531
en_et2894301553115531
en_id2894301553115531
en_ar2894301553115531
en_ta2894301553115531
en_lv2894301553115531
en_ja2894301553115531
fr_en2073741476014760
de_en1278341351113511
es_en790151322113221
ca_en958541273012730
it_en3169889408951
ru_en1211261106300
zh-CN_en708548434898
pt_en915833184023
fa_en5394934453445
et_en178215761571
mn_en206717611759
nl_en710816991699
tr_en396616241629
ar_en228317581695
sv-SE_en216013491595
lv_en233711251629
sl_en1843509360
ta_en1358384786
ja_en1119635684
id_en1243792844
cy_en1241690690

Dataset Creation

Curation Rationale

[Needs More Information]

Source Data

Initial Data Collection and Normalization

[Needs More Information]

Who are the source language producers?

[Needs More Information]

Annotations

Annotation process

[Needs More Information]

Who are the annotators?

[Needs More Information]

Personal and Sensitive Information

The dataset consists of people who have donated their voice online. You agree to not attempt to determine the identity of speakers in this dataset.

Considerations for Using the Data

Social Impact of Dataset

[Needs More Information]

Discussion of Biases

[Needs More Information]

Other Known Limitations

[Needs More Information]

Additional Information

Dataset Curators

[Needs More Information]

Licensing Information

CC BY-NC 4.0

Citation Information

@misc{wang2020covost,
    title={CoVoST 2: A Massively Multilingual Speech-to-Text Translation Corpus},
    author={Changhan Wang and Anne Wu and Juan Pino},
    year={2020},
    eprint={2007.10310},
    archivePrefix={arXiv},
    primaryClass={cs.CL}

Contributions

Thanks to @patil-suraj for adding this dataset.