KRAFTON/Raon-OpenTTS-Pool

1---2license: other3license_name: mixed-per-dataset4license_link: LICENSE5language:6  - en7tags:8  - text-to-speech9  - tts10  - speech11  - audio12  - open-data13  - training-data14  - english15task_categories:16  - text-to-speech17pretty_name: Raon-OpenTTS-Pool18size_categories:19  - 100M<n<1B20configs:21  - config_name: all22    data_files:23      - split: pool24        path: "*/metadata_pool.parquet"25      - split: core26        path: "*/metadata_core.parquet"27  - config_name: Raon-YouTube-Commons28    data_files:29      - split: pool30        path: Raon-YouTube-Commons/metadata_pool.parquet31      - split: core32        path: Raon-YouTube-Commons/metadata_core.parquet33  - config_name: Emilia-YODAS234    data_files:35      - split: pool36        path: Emilia-YODAS2/metadata_pool.parquet37      - split: core38        path: Emilia-YODAS2/metadata_core.parquet39  - config_name: Emilia40    data_files:41      - split: pool42        path: Emilia/metadata_pool.parquet43      - split: core44        path: Emilia/metadata_core.parquet45  - config_name: LibriHeavy46    data_files:47      - split: pool48        path: LibriHeavy/metadata_pool.parquet49      - split: core50        path: LibriHeavy/metadata_core.parquet51  - config_name: HiFiTTS52    data_files:53      - split: pool54        path: HiFiTTS/metadata_pool.parquet55      - split: core56        path: HiFiTTS/metadata_core.parquet57  - config_name: VoxPopuli58    data_files:59      - split: pool60        path: VoxPopuli/metadata_pool.parquet61      - split: core62        path: VoxPopuli/metadata_core.parquet63  - config_name: PeoplesSpeech-Clean64    data_files:65      - split: pool66        path: PeoplesSpeech-Clean/metadata_pool.parquet67      - split: core68        path: PeoplesSpeech-Clean/metadata_core.parquet69  - config_name: PeoplesSpeech-Dirty70    data_files:71      - split: pool72        path: PeoplesSpeech-Dirty/metadata_pool.parquet73      - split: core74        path: PeoplesSpeech-Dirty/metadata_core.parquet75  - config_name: LibriTTS-R76    data_files:77      - split: pool78        path: LibriTTS-R/metadata_pool.parquet79      - split: core80        path: LibriTTS-R/metadata_core.parquet81  - config_name: SPGISpeech2-Cut82    data_files:83      - split: pool84        path: SPGISpeech2-Cut/metadata_pool.parquet85      - split: core86        path: SPGISpeech2-Cut/metadata_core.parquet87---88 89# Raon-OpenTTS-Pool90 91<div align="center">92  <img class="block dark:hidden" src="assets/Raon-OpenTTS-Gradient-Black.png" alt="Raon-OpenTTS" width="600">93  <img class="hidden dark:block" src="assets/Raon-OpenTTS-Gradient-White.png" alt="Raon-OpenTTS" width="600">94</div>95 96<p align="center">97  <a href="https://www.krafton.ai/ko/"><img src="https://img.shields.io/badge/Homepage-KRAFTON%20AI-blue?style=flat&logo=google-chrome&logoColor=white" alt="Homepage"></a>98  <a href="https://github.com/krafton-ai/Raon-OpenTTS"><img src="https://img.shields.io/badge/GitHub-Raon--OpenTTS-white?style=flat&logo=github&logoColor=black" alt="GitHub"></a>99  <a href="https://huggingface.co/KRAFTON"><img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-KRAFTON-yellow?style=flat" alt="Hugging Face"></a>100  <a href="https://x.com/Krafton_AI"><img src="https://img.shields.io/badge/X-KRAFTON%20AI-white?style=flat&logo=x&logoColor=black" alt="X"></a>101  <a href="#license"><img src="https://img.shields.io/badge/License-Mixed%20(see%20below)-lightgrey?style=flat" alt="License"></a>102</p>103 104<p align="center">105  <a href="https://arxiv.org/abs/2605.20830">Technical Report</a>106</p>107 108**Raon-OpenTTS-Pool** is a large-scale open English speech corpus for text-to-speech (TTS) training,109constructed from 8 publicly available speech corpora and a set of web-sourced recordings.110It is the training data behind [Raon-OpenTTS](https://github.com/krafton-ai/Raon-OpenTTS),111an open TTS model that performs on par with state-of-the-art closed-data systems.112 113- **615K hours** of speech audio114- **239.7M** speech segments115- **11 source datasets** aggregated into a unified format116- All audio stored as **16 kHz mono Opus (64 kbps)** in [WebDataset](https://github.com/webdataset/webdataset) tar shards117 118We restrict data sources to publicly available English speech datasets with more than 500 hours of audio.119All speech segments are limited to **30 seconds or shorter** to reduce alignment errors, multi-speaker content, and non-speech artifacts.120Existing public datasets (LibriHeavy, Emilia, VoxPopuli, etc.) are included as-is without modification,121with audio standardized to 16 kHz mono Opus 64 kbps for storage efficiency.122The Raon-YouTube-Commons portion is reconstructed from [YouTube-Commons](https://huggingface.co/datasets/PleIAs/YouTube-Commons)123through a dedicated preprocessing pipeline (see [below](#raon-youtube-commons)).124 125With a model-based filtering pipeline applied to Raon-OpenTTS-Pool, we derive126**Raon-OpenTTS-Core**, a curated high-quality subset of **510.1K hours** and **194.5M** segments.127 128For more details, see our paper: [Raon-OpenTTS: Open Models and Data for Robust Text-to-Speech](https://arxiv.org/abs/2605.20830)129 130## Format131 132Each WebDataset tar shard contains pairs of files per sample:133 134```135{sample_key}.opus   # 16 kHz mono Opus 64 kbps audio136{sample_key}.json   # {"text": "...", "duration": 8.42, "source": "..."}137```138 139> **Note:** The dataset viewer shows metadata only (sample_key, text, duration, shard_name).140> Audio is stored in WebDataset tar files — see [Usage](#usage) below to download and load audio.141 142## Splits143 144Each dataset config has two metadata splits:145 146- **pool** — all samples (sample_key, text, duration, shard_name)147- **core** — quality-filtered subset (**Raon-OpenTTS-Core**), retaining ~85% of the data148 149### Raon-OpenTTS-Core Filtering150 151Raon-OpenTTS-Core is constructed by applying three model-based quality filters and removing the bottom 15% of samples by combined score:152 1531. **WER-based**: Transcribe each segment with Whisper-small ASR and compute WER against the existing text annotation. Samples with excessively high WER (> 0.35) indicate severe transcription mismatches.1542. **DNSMOS-based**: Estimate perceptual speech quality using DNSMOS. Samples below 2.24 indicate strong background noise or distortion.1553. **VAD-based**: Estimate speech activity ratio (SAR) using Silero VAD. Samples with SAR below 0.79 are dominated by silence, music, or non-speech audio.1564. **Combined**: Compute an absolute rank for each segment along each criterion (DNSMOS, WER, SAR) and average the ranks into a single combined score. Segments falling below the 15th percentile are discarded.157 158This combined filtering achieves the best overall TTS performance across diverse evaluation benchmarks (see paper, Figure 3).159 160## Available Datasets161 162| Dataset | Source | Size (h) | Avg. Dur. (s) | Segments (M) | Tars | License | DNSMOS | WER | SAR |163|---|---|---|---|---|---|---|---|---|---|164| **Raon-YouTube-Commons** | [YouTube-Commons](https://huggingface.co/datasets/PleIAs/YouTube-Commons) | 335k | 8.5 | 141.70 | 1,017 | CC BY 4.0 | 2.74 | 0.30 | 0.90 |165| **Emilia-YODAS2** | [Emilia](https://huggingface.co/datasets/amphion/Emilia-Dataset) | 92k | 9.2 | 35.97 | 287 | CC BY-NC 4.0 | 2.82 | 0.19 | 0.90 |166| **Emilia** | [Emilia](https://huggingface.co/datasets/amphion/Emilia-Dataset) | 47k | 9.3 | 18.14 | 145 | CC BY 4.0 | 3.02 | 0.18 | 0.89 |167| **LibriHeavy** | [LibriHeavy](https://github.com/k2-fsa/libriheavy) | 42k | 14.2 | 10.77 | 127 | Public Domain | 3.22 | 0.11 | 0.83 |168| **HiFiTTS** | [HiFiTTS2](https://www.openslr.org/hifitts/) | 37k | 10.1 | 13.09 | 109 | CC BY 4.0 | 3.20 | 0.11 | 0.84 |169| **PeoplesSpeech-Dirty** | [People's Speech](https://huggingface.co/datasets/MLCommons/peoples_speech) | 28k | 14.2 | 5.48 | 63 | CC BY 4.0 | 2.63 | 0.25 | 0.86 |170| **VoxPopuli** | [VoxPopuli](https://github.com/facebookresearch/voxpopuli) | 17k | 27.8 | 2.24 | 50 | CC-0 | 2.82 | 0.36 | 0.83 |171| **PeoplesSpeech-Clean** | [People's Speech](https://huggingface.co/datasets/MLCommons/peoples_speech) | 10k | — | 1.50 | 18 | CC BY 4.0 | — | — | — |172| **LibriTTS-R** | [LibriTTS-R](https://www.openslr.org/141/) | 552 | 5.6 | 0.35 | 2 | CC BY 4.0 | 2.96 | 0.06 | 0.91 |173| **SPGISpeech2-Cut** | SPGISpeech 2.0 | 889 | 14.4 | 0.22 | 3 | Kensho UA | 2.72 | 0.08 | 0.90 |174| | | | | | | | | | |175| **Total** | | **615k** | **9.2** | **239.7** | **1,821** | — | 2.83 | 0.24 | 0.89 |176 177### Raon-YouTube-Commons178 179A substantial portion of Raon-OpenTTS-Pool (335K hours) is derived from [YouTube-Commons](https://huggingface.co/datasets/PleIAs/YouTube-Commons).180Since the original release provides only YouTube URLs with noisy or unreliable transcriptions,181we reconstructed it into a high-quality speech-text dataset through the following pipeline:182 1831. **Audio collection**: Download audio from YouTube URLs in the original dataset1842. **Source separation** (UVR-MDX): Suppress background music and non-vocal components1853. **Speaker diarization** (PyAnnote 3.1): Estimate speaker boundaries to ensure single-speaker segments1864. **Voice activity detection** (Silero VAD): Segment continuous speech regions into clips of 3--30 seconds1875. **Automatic transcription** (Whisper-large-v3): Transcribe each segment to obtain aligned speech-text pairs1886. **Standardization**: Resample to 16 kHz mono, encode as 64 kbps Opus189 190The resulting dataset is released as **Raon-YouTube-Commons** in this repository.191 192### Non-redistributable Datasets193 194Two additional datasets used in training cannot be included due to license restrictions.195Users who have agreed to the license on HuggingFace can automatically download and convert them196using `prepare_nonredist_datasets.py`:197 198| Dataset | Size (h) | License | Source |199|---|---|---|---|200| GigaSpeech | 10k | License agreement required | [speechcolab/gigaspeech](https://huggingface.co/datasets/speechcolab/gigaspeech) |201| SPGISpeech | 5k | Non-commercial (Kensho) | [kensho/spgispeech](https://huggingface.co/datasets/kensho/spgispeech) |202 203See [Preparing Non-redistributable Datasets](#preparing-non-redistributable-datasets) for instructions.204 205---206 207## Usage208 209### 1. Metadata (pool / core split)210 211```python212from datasets import load_dataset213 214# Core metadata for a single dataset215meta = load_dataset("KRAFTON/Raon-OpenTTS-Pool", "Raon-YouTube-Commons", split="core")216# Columns: sample_key, text, duration, shard_name217print(meta[0])218 219# All datasets combined220all_core = load_dataset("KRAFTON/Raon-OpenTTS-Pool", "all", split="core")221```222 223### 2. Audio (WebDataset, local tars)224 225Download tars first:226 227```python228from huggingface_hub import snapshot_download229 230local_dir = snapshot_download("KRAFTON/Raon-OpenTTS-Pool", repo_type="dataset",231                               ignore_patterns=["*.parquet"])232```233 234Then load with WebDataset:235 236```python237import webdataset as wds238import json, io, soundfile as sf239 240dataset = (241    wds.WebDataset(f"{local_dir}/LibriTTS-R/lr-{{000000..000001}}.tar")242    .to_tuple("opus", "json")243)244for opus_bytes, json_bytes in dataset:245    meta = json.loads(json_bytes)246    audio, sr = sf.read(io.BytesIO(opus_bytes))247    text = meta["text"]248```249 250### 3. Core-only training251 252The audio tars contain pool and core samples mixed. To train on core only, filter by sample_key:253 254```python255import webdataset as wds256from datasets import load_dataset257import json, io, soundfile as sf258 259# Step 1: load core sample keys from metadata260core_keys = set(261    load_dataset("KRAFTON/Raon-OpenTTS-Pool", "LibriTTS-R", split="core")["sample_key"]262)263 264# Step 2: stream tars, skip non-core samples265dataset = (266    wds.WebDataset(f"{local_dir}/LibriTTS-R/lr-{{000000..000001}}.tar")267    .select(lambda s: s["__key__"] in core_keys)268    .to_tuple("opus", "json")269)270for opus_bytes, json_bytes in dataset:271    meta = json.loads(json_bytes)272    audio, sr = sf.read(io.BytesIO(opus_bytes))273    text = meta["text"]274    duration = meta["duration"]275```276 277---278 279## Preparing Non-redistributable Datasets280 281The script `prepare_nonredist_datasets.py` automatically downloads and converts GigaSpeech282and SPGISpeech into the same WebDataset tar + parquet format used by Raon-OpenTTS-Pool.283 284### Prerequisites285 2861. **Accept the dataset license** on each HuggingFace dataset page:287   - GigaSpeech: https://huggingface.co/datasets/speechcolab/gigaspeech288   - SPGISpeech: https://huggingface.co/datasets/kensho/spgispeech289 2902. **Set your HuggingFace token** (from an account that has accepted the licenses):291   ```bash292   export HF_TOKEN=hf_your_token_here293   ```294 2953. **Install dependencies:**296   ```bash297   pip install "datasets<4.0" soundfile pyarrow numpy tqdm298   ```299   > **Note:** `datasets>=4.0` dropped `soundfile` audio decoding and requires `torchcodec`300   > with system FFmpeg libraries. Use `datasets<4.0` (e.g. `datasets==3.5.0`) to avoid this.301 3024. **ffmpeg** must be in PATH.303 304### GigaSpeech305 306```bash307# Download and convert xl subset from HuggingFace Hub308python prepare_nonredist_datasets.py gigaspeech \309    --output_dir ./GigaSpeech \310    --gigaspeech_subset xl \311    --num_workers 16312 313# Or from a local HF snapshot (no HF_TOKEN needed)314python prepare_nonredist_datasets.py gigaspeech \315    --source_dir /path/to/gigaspeech_local \316    --output_dir ./GigaSpeech \317    --gigaspeech_subset xl318```319 320Available subsets: `xs` (10h), `s` (250h), `m` (1000h), `l` (2500h), `xl` (10000h)321 322### SPGISpeech323 324```bash325# Download and convert L subset from HuggingFace Hub326python prepare_nonredist_datasets.py spgispeech \327    --output_dir ./SPGISpeech \328    --spgispeech_subset L \329    --num_workers 16330 331# Or from a local HF snapshot (no HF_TOKEN needed)332python prepare_nonredist_datasets.py spgispeech \333    --source_dir /path/to/spgispeech_local \334    --output_dir ./SPGISpeech \335    --num_workers 16336```337 338Available subsets: `L` (full ~5000h), `M` (~1000h), `S` (~200h), `dev`, `test`339 340### Output341 342```343<output_dir>/344  {prefix}-000000.tar     # WebDataset shard (~10 GB)345  {prefix}-000001.tar346  ...347  metadata_pool.parquet   # all samples348  metadata_core.parquet   # = pool (no quality filtering without --core_json)349```350 351By default `metadata_core.parquet` equals `metadata_pool.parquet` since quality filtering352requires an internal index file. If you have `pool_indices_filter_remove_15pct_combined.json`353from the Raon-OpenTTS maintainers, pass it with `--core_json` to generate a filtered core split.354 355### Using with Raon-OpenTTS training356 357Once prepared, pass the output directory as a `nonredist_dirs` entry in the training config:358 359```yaml360datasets:361  nonredist_dirs:362    - /path/to/GigaSpeech363    - /path/to/SPGISpeech364```365 366---367 368## License369 370**This repository contains data from multiple sources, each with its own license.**371Users must comply with the license of each individual sub-dataset they use.372 373| Dataset | License | Commercial Use |374|---|---|---|375| Raon-YouTube-Commons | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Yes |376| Emilia | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Yes |377| **Emilia-YODAS2** | **[CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/)** | **No** |378| LibriHeavy | Public Domain (LibriVox) | Yes |379| HiFiTTS | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Yes |380| PeoplesSpeech-Clean / Dirty | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Yes |381| VoxPopuli | [CC-0](https://creativecommons.org/publicdomain/zero/1.0/) | Yes |382| LibriTTS-R | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Yes |383| SPGISpeech2-Cut | [Kensho User Agreement](https://huggingface.co/datasets/kensho/spgispeech) | Non-commercial |384| GigaSpeech (non-redist) | [License agreement required](https://huggingface.co/datasets/speechcolab/gigaspeech) | See terms |385| SPGISpeech (non-redist) | [Kensho User Agreement](https://huggingface.co/datasets/kensho/spgispeech) | Non-commercial |386| Metadata and dataset structure | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) | Yes |387 388> **Note:** Emilia-YODAS2 and SPGISpeech2-Cut are licensed under non-commercial terms.389> If you require fully commercial-use data, exclude these sub-datasets via the `configs` parameter.390 391## Citation392 393```bibtex394@article{kim2026raonopentts,395  title     = {Raon-OpenTTS: Open Models and Data for Robust Text-to-Speech},396  author    = {Kim, Semin and Chung, Seungjun and Moon, Taehong and Lee, Sangheon and Ahn, Minyoung and Lee, Keon and Kim, Nam Soo and Cho, Jaewoong and Schmidt, Ludwig and Lee, Kangwook and Park, Dongmin},397  journal   = {arXiv preprint arXiv:2605.20830},398  year      = {2026},399  url       = {https://arxiv.org/abs/2605.20830}400}401```402 403&copy; 2026 KRAFTON404