marianogonzalezgomez/voxpopuli_es-ja

Dataset Card for Spanish-to-Japanese Automatic Speech Recognition Dataset

Dataset Summary

This dataset was created as part of a workshop organized by Yasmin Moslem, focusing on speech-to-text pipelines. The workshop's primary goal is to enable accurate transcription and translation of spoken a source language into a written language (and learn how to do so, of course 😃) The dataset serves as the foundation for developing and evaluating various models, including end-to-end and cascaded approaches for speech-to-text tasks.

Designed for automatic speech recognition (ASR) and translation tasks, this dataset enables the conversion of Spanish speech into Japanese text. It consists of high-quality audio recordings sampled at 16 kHz, paired with Spanish transcriptions (es) and their Japanese translations (ja).

Dataset Processing

The dataset was built using the following processing steps:

1. 1.Base Dataset: The [facebook/voxpopuli](https://huggingface.co/datasets/facebook/voxpopuli) dataset was used as the foundational dataset. It provided the Spanish audio and corresponding transcriptions.
2. 2.Translation to English: Spanish transcriptions were translated to English using the [Helsinki-NLP/opus-mt-es-en](https://huggingface.co/Helsinki-NLP/opus-mt-es-en) machine translation model.
3. 3.Translation to Japanese: The English translations were further translated into Japanese using the [Helsinki-NLP/opus-tatoeba-en-ja](https://huggingface.co/Helsinki-NLP/opus-tatoeba-en-ja) machine translation model.
4. 4.Process Japanese text: The Japanese translations contained unnecessary blank spaces, which are not typical in Japanese writing. These spaces were removed to ensure proper formatting and consistency.
5. 5.Quality Filtering: The Japanese translations were evaluated for quality using the [Unbabel/wmt23-cometkiwi-da-xl](https://huggingface.co/Unbabel/wmt23-cometkiwi-da-xl) MT Quality Estimation model. Low-quality translations were filtered out, ensuring that the final dataset maintained some quality alignments between the audio, the transcriptions, and the translations.
6. 6.Empty Strings Filtering: The Spanish source texts that were empty had to be filtered out.

Dataset Structure

Features

The dataset contains the following features:

• —audio: Audio recordings, sampled at 16 kHz.
• —es: Spanish transcription of the audio.
• —ja: Japanese translation of the Spanish transcription.

Splits

Dataset Size

• —Download Size: 4.85 GB
• —Dataset Size: 5.66 GB

Linked Models

• —[Whisper-Small-es-ja](https://huggingface.co/Marianoleiras/whisper-small-es-ja): An end-to-end model trained on this dataset.
• —[Whisper-Small-es](https://huggingface.co/Marianoleiras/whisper-small-es): The ASR model of the cascaded approach built using this dataset.
• —[NLLB-200-Distilled-es-ja](https://huggingface.co/Marianoleiras/nllb-200-distilled-es-ja): The MT model of the cascaded approach built using this dataset.

Citations

@article{wang2021voxpopuli,
  title={VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation},
  author={Chung-Cheng Chiu and Paden Tomasello and ...},
  year={2021},
  publisher={Meta AI},
  url={https://huggingface.co/datasets/facebook/voxpopuli}
}

@inproceedings{tiedemann-thottingal-2020-opus,
  title={OPUS-MT -- Building Open Translation Services for the World},
  author={J{\"o}rg Tiedemann and Santhosh Thottingal},
  booktitle={Proceedings of the 22nd Annual Conference of the European Association for Machine Translation (EAMT)},
  year={2020},
  url={https://huggingface.co/Helsinki-NLP}
}

@inproceedings{rei-etal-2023-cometkiwi,
  title={COMETKiwi: Advanced Quality Estimation Model for Machine Translation},
  author={Rei, Ricardo and others},
  year={2023},
  url={https://huggingface.co/Unbabel/wmt23-cometkiwi-da-xl}
}

Dataset Card Contact

Mariano González (marianoleiras@hotmail.com)