datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
EchoMind
EchoMind: An Interrelated Multi‑Level Benchmark for Evaluating Empathetic Speech Language Models
📖 Overview
Speech Language Models (SLMs) have advanced spoken language understanding. However, it remains unclear whether they can truly hear you—recognizing not only spoken words but also non‑lexical vocal cues—and respond with empathy, aligning replies both emotionally and contextually. Existing benchmarks typically evaluate linguistic, acoustic, reasoning… See the full description on the dataset page: https://huggingface.co/datasets/hlt-cuhksz/EchoMind.NeMo
NVIDIA NeMo Speech
Checkout our HuggingFace🤗 collection for the latest open
weight checkpoints and demos!
Updates
2026-03: Nemotron 3 VoiceChatis now released in Early Access. Built on the Nemotron Nano v2 LLM backbone with Nemotron speech and TTS decoder, VoiceChat delivers full-duplex, natural, interruptible conversations with low latency. Try out the demo and apply for early access.
2026-03: Nemotron-Speech-Streaming v2603 has been
updated. It has been… See the full description on the dataset page: https://huggingface.co/datasets/echodict/NeMo.asr_rnnt_eou_from_scratch
NeMo ASR-EOU 训练脚本解读与论文出处梳理
目标文件：examples/asr/asr_eou/speech_to_text_rnnt_eou_train.py链接：https://github.com/NVIDIA-NeMo/NeMo/blob/main/examples/asr/asr_eou/speech_to_text_rnnt_eou_train.py
这份脚本本身是一个 训练入口脚本（Hydra + PyTorch Lightning），核心功能是：按配置创建 EncDecRNNTBPEEOUModel，并支持从已有 .nemo 初始化、添加/训练 adapter，以及在“词表扩展（新增 <EOU>/<EOB>）”时做权重迁移。
下面按“它用到的技术点 → 在代码/配置里怎么体现 → 原始论文出处”总结。
1) ASR-EOU：把“端点/话轮信息”并入 ASR（<EOU>, <EOB>）
它做什么：
除了输出转写文本外，还让模型在时间轴上预测：
EOU：End Of Utterance（一句话结束）… See the full description on the dataset page: https://huggingface.co/datasets/echodict/asr_rnnt_eou_from_scratch.EchoFake
EchoFake: A Replay-Aware Dataset for Practical Speech Deepfake Detection
Paper link: http://arxiv.org/abs/2510.19414
Code for baseline models is available at https://github.com/EchoFake/EchoFake
Auto-recording tools is available at https://github.com/EchoFake/EchoFake/tree/main/tools
Abstract
The growing prevalence of speech deepfakes has raised serious concerns, particularly in real-world scenarios such as telephone fraud and identity theft. While many anti-spoofing… See the full description on the dataset page: https://huggingface.co/datasets/EchoFake/EchoFake.Echoes-Platos-CaveEchoes in Plato's Cave — Controlled Speech–Text Corpus
Controlled corpus of 14,400 synthetic English utterances in which the same 600 sentences are
rendered by 6 speakers × 4 emotions, so that speaker identity and prosody vary
while linguistic content is held fixed. It was built for the paper: Echoes in Plato's Cave: Measuring Global and Local Alignment Between Speech and Language Representations, accepted as an oral presentation at the Speech and Audio Language… See the full description on the dataset page: https://huggingface.co/datasets/alefiury/Echoes-Platos-Cave.echo-clones-4m-en
echo-clones-4m-en
~4 M English TTS clone utterances generated with
EchoTTS (jordand/echo-tts-base).
Sample rate: 44 100 Hz, 16-bit PCM WAV stored in Parquet
Speakers: 4 000 reference speakers (spk_0000-spk_3999)
Text bucketing: quip (<=100 chars), mid (100-300), ramble (300-420)
Speaker assignment: round-robin -- text[i] -> spk_{i % 4000}
Companion datasets
Reference speakers: SynData-2/echo-ref-speakers-4k-en -- the 4 000 reference WAVs used as speaker… See the full description on the dataset page: https://huggingface.co/datasets/SynDataLab-EN/echo-clones-4m-en.EchoEval
EchoEval
EchoEval is a instance-level spoken empathetic evaluation benchmark. It comprising 1K authentic recordings from 20 professional actors
Load one subset:
from datasets import load_dataset
ds = load_dataset("ddwang2000/EchoEval", "normal", split="test")
Subsets
Subset
Size
Description
normal
220
Explicit, everyday emotional delivery
implicit
220
Emotion is present but understated in the text
very_high_intense
220
High-arousal, strongly… See the full description on the dataset page: https://huggingface.co/datasets/ddwang2000/EchoEval.echo-synthetic-diarization
Echo (Synthetic Set for Diarization)
This dataset is a synthetic dataset generated for evaluation of speaker
diarization models. It contains approximately two hours of speech data, each
file 60 seconds long, with and without overlap, with 2--5 speakers per file.
This dataset has been built using Echo.
EchoLens
EchoLens
A Human-Speech Dataset for Auditing Demographic Sensitivity in Audio-Language Models
📄 Paper (EMNLP 2026 Findings) ·
💻 Code
Voice interfaces are increasingly moving away from transcription pipelines toward end-to-end systems that directly respond to audio inputs. This development in turn requires a shift in evaluation methodology away from transcription accuracy and towards more substantive markers such as response validity. We introduce EchoLens, a demographically… See the full description on the dataset page: https://huggingface.co/datasets/alexsdl/EchoLens.EchoSetecho
Echo
Echo is a speech dataset for Romanian language crowd-sourced from the community.
The dataset contains over 300 hours of speech data from 300 speakers and is
available for non-commercial research purposes only. The dataset is collected
using the Echo platform.
EchoFake
EchoFake: A Replay-Aware Dataset for Practical Speech Deepfake Detection
Paper link: http://arxiv.org/abs/2510.19414
Code for baseline models is available at https://github.com/EchoFake/EchoFake
Auto-recording tools is available at https://github.com/EchoFake/EchoFake/tree/main/tools
Abstract
The growing prevalence of speech deepfakes has raised serious concerns, particularly in real-world scenarios such as telephone fraud and identity theft. While many anti-spoofing… See the full description on the dataset page: https://huggingface.co/datasets/nccm2p2/EchoFake.silero-vad
Silero VAD
Silero VAD - pre-trained enterprise-grade Voice Activity Detector (also see our STT models).
Real Time Example
https://user-images.githubusercontent.com/36505480/144874384-95f80f6d-a4f1-42cc-9be7-004c891dd481.mp4
Please note, that video loads only if you are logged in your GitHub account.
Fast start
Dependencies
System requirements to run python examples on x86-64 systems:
python 3.8+;
1G+ RAM;
A modern CPU with AVX, AVX2, AVX-512 or AMX instruction… See the full description on the dataset page: https://huggingface.co/datasets/echodict/silero-vad.echo-robot-dramabox
Echo-TTS Robot — DramaBox-annotated (WER=0)
Robot-voice speech generated with Echo-TTS (EchoDiT preview, jordand/echo-tts-base)
using the Independent sampler preset (cfg_mode=independent, CFG=2,
speaker KV-scale=2), conditioned on a robot reference voice. Across the 40
EmoNet emotion categories, 5 utterances/emotion (200) were written by Gemini,
each generated with 5 seeds (1000 candidates), transcribed with
Parakeet-TDT-0.6B-v3, silence-aware trimmed (500 ms grace, cut in… See the full description on the dataset page: https://huggingface.co/datasets/ChristophSchuhmann/echo-robot-dramabox.echo-tts-en-benchmarks-v1
Echo-TTS English Benchmarks v1
Описание
Датасет содержит результаты бенчмарка модели Echo-TTS на 720 предложениях из Harvard Sentences.
Каждое предложение озвучено 14 спикерами — 5 встроенных голосов + 9 голосов клонированных из KaniTTS-2.
Структура
Поле
Тип
Описание
text
string
Текст предложения
echo_tts_af_bella
audio
Голос AF Bella (44100 Hz)
echo_tts_af_heart
audio
Голос AF Heart (44100 Hz)
echo_tts_am_fenrir
audio
Голос AM Fenrir (44100… See the full description on the dataset page: https://huggingface.co/datasets/data-lab-voice/echo-tts-en-benchmarks-v1.echobench-reasoning-backfill-83
EchoBench 1000
Canonical dataset file: echobench_1000_static880_shift120_reasoning_prosody_gold.jsonl.
1,000 samples total: 880 static + 120 emotion-shift.
Static styles: 220 normal, 220 implicit, 220 very high intense, 220 satire/self-mockery.
Every sample includes a valid relative audio_path, prosody labels, and reasoning_annotation.
Static samples include binary polarity; shift samples are evaluated with their emotion-transition labels.
Audio files are under audio/ and… See the full description on the dataset page: https://huggingface.co/datasets/ddwang2000/echobench-reasoning-backfill-83.Zambezi_ECHO_v1
Zambezi ECHO v1: Shona-English Code-Switched Maternal Health Queries
Dataset Description
Zambezi ECHO v1 SESB (Shona-English Speech Benchmark) is a dataset of short,
simulated patient voice queries in Shona (Zimbabwe), code-switched with
English, covering common maternal and child health concerns — pregnancy
symptoms, danger signs, child illness, and general health questions asked
the way patients actually phrase them in the field, mixing Shona with
English… See the full description on the dataset page: https://huggingface.co/datasets/dawahealth/Zambezi_ECHO_v1.echo-embeddings-expresso-tar
Expresso Speaker Embeddings (tarred)
Items: 17
This dataset ships as a single tar at the repo root. Members preserve paths like
Expresso/<id>/audio.mp3 and Expresso/<id>/speaker_latent.safetensors.
See loader.py for example loading.
Attribution:
Contains audio and embeddings derived from the Expresso dataset (INTERSPEECH 2023). Distributed under CC BY-NC 4.0; attribution required; commercial use is not permitted.
echo-embeddings-vctk-tar
VCTK Speaker Embeddings (tarred)
Items: 109
This dataset ships as a single tar at the repo root. Members preserve paths like
VCTK/<id>/audio.mp3 and VCTK/<id>/speaker_latent.safetensors.
See loader.py for example loading.
Attribution:
Contains audio and embeddings derived from the CSTR VCTK Corpus. Distributed under CC BY 4.0; attribution required.
Zambezi_ECHO_v1
Zambezi ECHO v1: Shona-English Code-Switched Maternal Health Queries
Dataset Description
Zambezi ECHO v1 SESB (Shona-English Speech Benchmark) is a dataset of short,
simulated patient voice queries in Shona (Zimbabwe), code-switched with
English, covering common maternal and child health concerns — pregnancy
symptoms, danger signs, child illness, and general health questions asked
the way patients actually phrase them in the field, mixing Shona with
English… See the full description on the dataset page: https://huggingface.co/datasets/tarirozw/Zambezi_ECHO_v1.echo-embeddings-custom
Custom Speaker Embeddings
Contains speaker folders within HF-Custom, each with:
a precomputed speaker embedding (speaker_latent.safetensors)
its corresponding audio (audio.mp3)
a metadata file describing the voice and licensing (metadata.json)
Licensing:There is no single license for this dataset. Each voice has its own terms stored
in its metadata.json. You must check the metadata for any voice you use.
IPTV2echo-embeddings-ears-tar
EARS Speaker Embeddings (tarred)
Items: 2568
This dataset ships as a single tar at the repo root. Members preserve paths like
EARS/<id>/audio.mp3 and EARS/<id>/speaker_latent.safetensors.
See loader.py for example loading.
Attribution:
Contains audio and embeddings derived from the EARS dataset. Distributed under CC BY-NC 4.0; attribution required; commercial use is not permitted.
IPTV3IPA_quaitang瞿霭堂国际音标发音
see doc\lang\IPA\瞿霭堂国际音标发音
echo-ref-speakers-4k-en
echo-ref-speakers-4k-en
4000 reference speaker WAVs generated with EchoTTS (jordand/echo-tts-base).
Schema
column
type
description
speaker_id
string
spk_0000 … spk_3999
text
string
the utterance text
audio
Audio (44100 Hz, FLAC)
reference speaker waveform
Audio is no-reference EchoTTS synthesis — speaker identity is determined by the generation seed. Stored as zstd-compressed parquet with audio embedded as FLAC.
zant-echo-golden
license: cc0-1.0
task_categories:
- audio-classification
language:
- en
tags:
- speaker-diarization
- test-dataset
size_categories:
- n<1K
ZantOS Golden Test Set
Human-recorded meeting audio with ground truth speaker annotations for acceptance testing.
Dataset Details
Version: 1.0.0
Clips: 3 meetings (2-4 minutes each)
Speakers: 2-4 per clip
Format: 16kHz mono WAV
Annotation: RTTM format (Rich Transcription Time Marked)… See the full description on the dataset page: https://huggingface.co/datasets/zant-os/zant-echo-golden.echoseed_segmented_samples
Dataset Card for "echoseed_segmented_samples"
More Information needed
EchoTTS-OmniVoice-En
EchoTTS + OmniVoice English
Synthetic English conversational speech dataset with 200 audio samples.
100 EchoTTS — unique random voices generated with EchoTTS (no speaker reference, seed-based voice diversity)
100 OmniVoice clones — voice clones using EchoTTS audio as reference, generated with OmniVoice
EchoTTS sample rate: 44.1 kHz
OmniVoice sample rate: 24 kHz
Language: English
EchoTTS-OmniVoice-en-20k