humanify/Env-TTS-Clean

Env-TTS-Clean

Environment-aware text-to-speech training corpus (clean release). Each row pairs four short 24 kHz mono FLAC clips with aligned transcripts:

• —an environment sample (different speaker, same acoustic scene),
• —a speaker reference (same speaker as the target utterance),
• —a speaker-enhanced copy of the reference (MossFormer2 enhancement — or, for the DDS source, the real clean-studio recording of the speaker reference),
• —the target speech to synthesise,

so a model can learn to generate an utterance with both a specified voice and a specified environment. This release supersedes the earlier Env-TTS-SD-Corpus with a richer schema, higher sample rate, per-clip ASR for all three contexts, two meeting sources (AMI, AliMeeting), and a controlled read-speech source (DDS / DAPS) that contributes real clean-studio enhancement ground truth.

Dataset statistics

Diarization-source totals were computed exactly on 2026-05-27 (239 shards). The DDS portion (194,400 rows, added 2026-06-09; train groups group_00061– group_00122) is estimated from a sampled shard (mean per row: speech 4.66 s, env 4.24 s, speaker 4.29 s).

Rows by source dataset

Schema

Source corpora

DDS is single-speaker read speech (not a diarization corpus): 20 DAPS speakers re-recorded across 9 rooms × 3 microphones × 6 positions (162 acoustic conditions). For each condition the target speech and the environment_audio_source (a different speaker, same room/mic/position) are the device-degraded recordings, while speaker_audio_source_enhanced is the matching clean-studio recording — a real enhancement ground truth rather than a MossFormer2 estimate. All DDS text comes from the DAPS scripts (text_source = original).

Processing pipeline

Built with the streaming pipeline in `env-tts-data-pipeline`.

Diarization sources — three parallel stages download → process → upload:

1. 1.download — stream each source conversation (HF mirrors, OpenSLR tar streams, etc.) into a bounded local cache; emit a JSON sentinel when ready.
2. 2.process — resample to 24 kHz mono, walk diarisation turns, emit 3–15 s speech slices with a same-speaker reference (≥2.5 s) and a different-speaker environment slice (≥2.5 s). Missing/split transcripts are re-labelled with Qwen3-ASR-1.7B. Snappy parquet shards (~800 rows / shard, 4 shards per HF commit group).
3. 3.upload — HfApi.upload_folder per sealed group, resume-safe.
4. 4.enhance (second pass) — MossFormer2_SE_48K on speaker_audio_source.

DDS uses a dedicated parallel-channel pass (process-dds): each (room, device, position) condition is one acoustic scene; rows are assembled directly from the parallel clean/degraded recordings, and speaker_audio_source_enhanced is filled in-place with the real clean-studio clip (no second-pass MossFormer2).

Licensing

Released under CC-BY-NC-4.0 (non-commercial), inheriting the most restrictive terms among sources. In particular:

• —DDS / DAPS — CC-BY-NC-4.0 (non-commercial). This is the binding term for the whole release.
• —M3SD — academic / non-commercial research only.
• —MSDWILD — X-LANCE research-only agreement.
• —AISHELL-4 (Apache-2.0), CHiME-6 (CC-BY-SA-4.0), AMI, and AliMeeting carry their respective open / research terms.

Redistributing extracted audio requires complying with each upstream licence.

Citation

Please cite the source papers when using this corpus:

@article{li2021dds,
  title={DDS: A new device-degraded speech dataset for speech enhancement},
  author={Li, Haoyu and Yamagishi, Junichi},
  journal={arXiv preprint arXiv:2109.07931},
  year={2021}
}

@article{wu2025m3sd,
  title={M3SD: Multi-modal, Multi-scenario and Multi-language Speaker
         Diarization Dataset},
  author={Wu, Shilong and others},
  journal={arXiv preprint arXiv:2506.14427},
  year={2025}
}

@inproceedings{fu2021aishell4,
  title={AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation,
         Recognition and Speaker Diarization in Conference Scenario},
  author={Fu, Yihui and others},
  booktitle={Interspeech},
  year={2021}
}

@inproceedings{liu2022msdwild,
  title={MSDWILD: Multi-modal Speaker Diarization Dataset in the Wild},
  author={Liu, Tao and others},
  booktitle={Interspeech},
  year={2022}
}

@inproceedings{watanabe2020chime6,
  title={CHiME-6 Challenge: Tackling Multispeaker Speech Recognition for
         Unsegmented Recordings},
  author={Watanabe, Shinji and others},
  booktitle={CHiME Workshop},
  year={2020}
}

ASR re-labelling uses Qwen3-ASR-1.7B. Speaker enhancement uses MossFormer2 (ClearVoice). DDS is built on the DAPS dataset (Mysore, 2015).

Loading

from datasets import load_dataset

ds = load_dataset("ChristianYang/Env-TTS-Clean", split="train", streaming=True)
row = next(iter(ds))
print(row["text"], row["dataset"], row["speech_duration"])
# Audio columns decode automatically when accessed (24 kHz mono).

# Filter to a single source (e.g. the DDS read-speech rows):
dds = ds.filter(lambda r: r["dataset"] == "dds")

Files on disk

data/
  group_00000/ ... group_00122/      # group_00061–00122 are DDS
    manifest.json
    data_000000.parquet
    ...

Each group_* directory is one atomic HF commit bundle (typically 4 × 800-row parquet shards, snappy-compressed FLAC payloads inside).