espnet/yodas-granary

Table of Contents

• —Dataset Description
• —Overview
• —Data Distribution
• —How to Use
• —Standard Loading
• —Streaming
• —Dataset Structure
• —Data Instance
• —Data Fields
• —Data Splits
• —Reference

Dataset Card for YODAS-Granary

• —Repository: NeMo-speech-data-processor: Granary
• —Paper: Granary: Speech Recognition and Translation Dataset in 25 European Languages
• —Shared by: ESPnet

Dataset Description

YODAS-Granary is a curated subset of the larger `nvidia/Granary` dataset, focusing on high-quality pseudo-labeled speech data for Automatic Speech Recognition (ASR) and Automatic Speech Translation (AST) across 23 European languages.

Overview

<table style="width:100%; table-layout:auto;"> <tr> <td style="vertical-align:middle; text-align:center;"> <img src="./scheme.png" style="max-width:100%;"> </td> <td style="vertical-align:middle; padding-left:20px;"> <p> Derived from the <a href=https://huggingface.co/datasets/espnet/yodas2) target="blank"><code>espnet/yodas2</code></a> corpus, YODAS-Granary provides high-quality pseudo-labeled speech data, focusing on two core tasks: </p> <ul> <li><strong>Automatic Speech Recognition (ASR)</strong>: covers 23 European languages, with pseudo-labeled transcriptions generated using the <a href="https://huggingface.co/Systran/faster-whisper-large-v3" target="blank"><code>Systran/faster-whisper-large-v3</code></a> model, post-processed to restore punctuation and capitalization using <a href="https://huggingface.co/Qwen/Qwen2.5-7B-Instruct" target="blank"><code>Qwen/Qwen2.5-7B-Instruct</code></a>, and filtered for quality.</li> <li><strong>Automatic Speech Translation (AST)</strong>: covers 22 non-English languages and consists of high-quality translations into English, generated from <code>ASR</code> subset using the <a href="https://huggingface.co/utter-project/EuroLLM-9B-Instruct" target="blank"><code>utter-project/EuroLLM-9B-Instruct</code></a> model and filtered for quality.</li> </ul> </td> </tr> </table>

Data Distribution

The following chart illustrates the distribution of data in the YODAS-Granary dataset across 23 European languages, measured in number of words (left) and total hours of audio (right), for both ASR and AST tasks.

<p align="center"> <img src="./chart.png" width="100%"/><br> <em>🔍 AST data is always a filtered subset of ASR, which is why AST bars are never taller than their ASR counterparts.<br>🗣️ The English subset contains ASR data only.</em> </p>

The table below summarizes the storage footprint and sample counts per language in the YODAS-Granary dataset, broken down into corresponding splits:

• —ast — the size and number of translated samples (X → English),
• —asr_only — samples that exist only in the ASR subset and have no corresponding translation,

and combined size and number of all samples per language (total).

How to use

Standard Loading

You can load the dataset using the datasets library from Hugging Face:

from datasets import load_dataset

🔹 Load the entire dataset:

ds = load_dataset("espnet/yodas-granary")

🔹 Load a single language (e.g., Italian):

ds = load_dataset("espnet/yodas-granary", "Italian")

Streaming

Some language subsets are quite large and may not fit comfortably in memory. For efficient access and analysis without downloading the entire dataset, we recommend using streaming mode:

ds = load_dataset("espnet/yodas-granary", "English", streaming=True)

Dataset Structure

Data Instance

Each utterance in the dataset includes the following fields: utt_id, audio, duration, lang, task, text, translation_en (null in asr_only), original_audio_id, and original_audio_offset.

Typical entry

from data/de101/translation/00000000.parquet

{
  "utt_id": "de101_00000000_Z0_gcPJVTqg_1004_62_1_74",
  "audio": {
    'path': 'de101_00000000_Z0_gcPJVTqg_1004_62_1_74.wav', 
    'bytes': ... 
  }
  "duration": 1.74,
  "lang": "<de>",
  "task": "<ast>",
  "text": "Ich muss mir das Zeug mal aus der Nähe ansehen.",
  "translation_en": "I have to take a closer look at this stuff.",
  "original_audio_id": "Z0_gcPJVTqg",
  "original_audio_offset": 1004.62
}

Data Fields

¹ - utt_id is encoded as <subset>_<shard>_<wav_id>_<start_time_s>_<start_time_decimals>_<duration_s>_<duration_decimals>, where subset, shard², and wav_id match the utterance's location in the original `espnet/yodas2` archive.

² - shard indices reflect those in `espnet/yodas2`, but some shards are missing due to filtering during data processing. In particular, bg000 is missing shard 00000011, en000 is missing shard 00000308, en003 is missing shard 00000221, en118 is missing shard 00000240.

Data Splits

The dataset is organized into language-specific subsets, each containing one or two splits, depending on the language:

• —For non-English languages:
• —<ast> – samples with both high-quality transcriptions and translations into English.
• —<asr_only> – samples that passed transcription quality checks but do not include translations.
• —For `English`:
• —<asr_only> split is available only, since English-to-English translation is not applicable.

Directory example

yodas_granary
└── data
    ├── da000   # subset
    │   ├── asr_only # corresponds to `asr_only` split
    │   │   ├── 00000000.parquet # shard
    │   │   ├── 00000001.parquet
    │   │   ├── 00000002.parquet
    │   │   └── ...
    │   └── ast # corresponds to `ast` split
    │       ├── 00000000.parquet # shard
    │       ├── 00000001.parquet
    │       ├── 00000002.parquet
    │       └── ...
    ├── cs000
    ├── bg000
    └── ...

Reference

@misc{koluguri2025granaryspeechrecognitiontranslation,
      title={Granary: Speech Recognition and Translation Dataset in 25 European Languages}, 
      author={Nithin Rao Koluguri and Monica Sekoyan and George Zelenfroynd and Sasha Meister and Shuoyang Ding and Sofia Kostandian and He Huang and Nikolay Karpov and Jagadeesh Balam and Vitaly Lavrukhin and Yifan Peng and Sara Papi and Marco Gaido and Alessio Brutti and Boris Ginsburg},
      year={2025},
      eprint={2505.13404},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.13404}, 
}