steven0226/open-asr-leaderboard-multilingual-datasets

Open ASR Leaderboard Chinese Test Set (FLEURS)

This repository holds a test-only copy of the Mandarin Chinese test split of google/fleurs (config cmn_hans_cn, split test). It is used for the Chinese column of the Open ASR Leaderboard (huggingface/open_asr_leaderboard#147).

The layout matches the FLEURS configs in hf-audio/open-asr-leaderboard-multilingual-datasets, so this config may later be merged into that repository.

How to load

from datasets import load_dataset

fleurs_zh = load_dataset("steven0226/open-asr-leaderboard-multilingual-datasets", "fleurs_zh", split="test")
print(fleurs_zh)

Contents

• —fleurs_zh: 945 recordings of 349 unique sentences (most sentences are recorded 2 or 3 times), 16 kHz mono WAV.
• —Columns: file_name, audio, duration (seconds), source_lang and target_lang (both zh), text.
• —Rows and row order are the same as data/cmn_hans_cn/test.tsv in google/fleurs. Nothing is filtered or deduplicated.
• —The audio bytes are the original FLEURS WAV files, not re-encoded.
• —text is the FLEURS raw_transcription (Simplified Chinese, with punctuation), with the language-independent part of the light cleanup seen in the other FLEURS configs of hf-audio/open-asr-leaderboard-multilingual-datasets: text inside square brackets is removed, parentheses are removed but the words inside them are kept (both ASCII and full-width parentheses), and repeated spaces are collapsed. Scoring-time normalization (for example Traditional-to-Simplified conversion and number handling) is done by the leaderboard code, not here.

License and attribution

The data comes from FLEURS by Google, released under the Creative Commons Attribution 4.0 International license (CC-BY-4.0). This copy keeps the same license. Changes from the source: file layout (Parquet); only the columns above are kept (duration, source_lang and target_lang are added, the other FLEURS columns are dropped); and the text cleanup described above.

Citation

@article{fleurs2022arxiv,
  title = {FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech},
  author = {Conneau, Alexis and Ma, Min and Khanuja, Simran and Zhang, Yu and Axelrod, Vera and Dalmia, Siddharth and Riesa, Jason and Rivera, Clara and Bapna, Ankur},
  journal = {arXiv preprint arXiv:2205.12446},
  url = {https://arxiv.org/abs/2205.12446},
  year = {2022}
}