datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
duplexgen-spoken
DuplexGen Spoken
Rendered spoken audio for the DuplexGen turn-taking dialogues — the exact
set of clips used to fine-tune the full-duplex model (PP-DG) in
DuplexGen: Adaptive Synthesis of Human–AI Turn-Taking Dialogues.
Each clip is a full render of one generated dialogue variation: the mixed
two-speaker dialogue audio, the isolated per-turn utterances, and the inserted
backchannel clips, plus per-clip metadata. Audio is synthesized with
Chatterbox TTS; the dialogue
text it… See the full description on the dataset page: https://huggingface.co/datasets/DuplexGen/duplexgen-spoken.ml_spoken_wordsMultilingual Spoken Words Corpus is a large and growing audio dataset of spoken
words in 50 languages collectively spoken by over 5 billion people, for academic
research and commercial applications in keyword spotting and spoken term search,
licensed under CC-BY 4.0. The dataset contains more than 340,000 keywords,
totaling 23.4 million 1-second spoken examples (over 6,000 hours). The dataset
has many use cases, ranging from voice-enabled consumer devices to call center
automation. This dataset is generated by applying forced alignment on crowd-sourced sentence-level
audio to produce per-word timing estimates for extraction.
All alignments are included in the dataset.SpokenWOZ-Test-Audiospoken-squad-t2aspoken-multiturn-sft
Spoken Multi-turn SFT Japanese
Japanese spoken multi-turn SFT dataset generated from kanhatakeyama/AutoMultiTurnByCalm3-22B using CosyVoice2 TTS.
Dataset Description
This dataset contains Japanese multi-turn SFT (Supervised Fine-Tuning) data with spoken questions.
q1: First question (text + audio)
a1: First answer (text only)
q2: Follow-up question (text + audio)
a2: Second answer (text only)
Samples
ID
Q1
Q1 Audio
A1
Q2
Q2 Audio
A2
0
鉄は強磁性体ですか？… See the full description on the dataset page: https://huggingface.co/datasets/Atotti/spoken-multiturn-sft.spoken-alpaca-gpt4SpokenNativQA
SpokenNativQA: Multilingual Everyday Spoken Queries for LLMs
The SpokenNativQA dataset consists of question-answer (QA) pairs, where queries are sourced from real users and answers are manually reviewed and edited. The dataset covers a diverse range of 18 topics that reflect culturally and regionally specific knowledge, as well as everyday queries. These topics include animals, business, clothing, education, events, food and drinks, general knowledge, geography, immigration… See the full description on the dataset page: https://huggingface.co/datasets/QCRI/SpokenNativQA.SpokenTOD
SpokenTOD
SpokenTOD is an English spoken task-oriented dialogue dataset created with the SpokenTOD augmentation pipeline, and used to train SpokenUS: A Spoken User Simulator for Task-Oriented Dialogue. The pipeline augments task-oriented dialogues with phenomena such as cross-turn slots, barge-in, disfluency, and emotion labels, and synthesizes speech conditioned on reference audio. Speech synthesis uses Qwen3-TTS with its Voice Design then Clone workflow.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/holi-lab/SpokenTOD.SpokenWOZ-Train-Text
What is SpokenWOZ?
SpokenWOZ is a large-scale multi-domain speech-text dataset for spoken task-oriented dialogue modeling, which consists of 203k turns, 5.7k dialogues and 249 hours audios from realistic human-to-human spoken conversations.
Why SpokenWOZ?
The majority of existing TOD datasets are constructed via writing or paraphrasing from annotators rather than being collected from realistic spoken conversations. The written TDO datasets may not be representative of the… See the full description on the dataset page: https://huggingface.co/datasets/ssz1111/SpokenWOZ-Train-Text.spoken_squad_testThis dataset is licensed under the terms of the CC-BY-SA-4.0 license.
https://github.com/Chia-Hsuan-Lee/Spoken-SQuAD/blob/master/LICENSE.md
Author: @michaellee886
@article{li2018spoken,
title={Spoken SQuAD: A study of mitigating the impact of speech recognition errors on listening comprehension},
author={Li, Chia-Hsuan and Wu, Szu-Lin and Liu, Chi-Liang and Lee, Hung-yi},
journal={arXiv preprint arXiv:1804.00320},
year={2018}
}
@article{wang2024audiobench,
title={AudioBench: A… See the full description on the dataset page: https://huggingface.co/datasets/AudioLLMs/spoken_squad_test.SpokenSwag
SpokenSwag
We present here SpokenSwag as described in the paper "Slamming: Training a Speech Language Model on One GPU in a Day".
This dataset is based on allenai/swag and synthetised with 4 speakers from hexgrad/Kokoro-82M.
We show that perfoming DPO over the dataset can really improve performance of Speech Language Models.
We encourage you to also see the following resources, for further information:
Project Page: https://pages.cs.huji.ac.il/adiyoss-lab/slamming/ Paper:… See the full description on the dataset page: https://huggingface.co/datasets/slprl/SpokenSwag.SpokenMedicalQAdddspoken_norm_pattern
Vietnamese Inverse Text Normalization
Inverse text normalization (ITN) is the task that transforms spoken to written styles. It is particularly useful in automatic speech recognition (ASR) systems where proper names are often miss-recognized by their pronunciations instead of the written forms. By applying ITN, we can improve the readability of the ASR system’s output significantly. This dataset provides data for doing ITN task in the Vietnamese language.
For example:
Spoken… See the full description on the dataset page: https://huggingface.co/datasets/nguyenvulebinh/spoken_norm_pattern.spoken-multihop-rag
Spoken Multi-hop QA: ASR Transcripts Across Four English Accents
ASR transcriptions of 3,000 multi-hop QA questions, each spoken in four
English accents and transcribed with Whisper-large-v3. Released as the
data companion to Better Retrieval, Worse Robustness: How Multi-hop RAG
Amplifies Upstream ASR Errors
(EMNLP 2026, Main Conference).
The dataset exists to make one thing cheap to study: what happens to a
retrieval pipeline when its query arrives through ASR rather than as… See the full description on the dataset page: https://huggingface.co/datasets/orcarouter/spoken-multihop-rag.Neapolitan-Spoken-Corpus
Neapolitan Spoken Corpus (NSC)
A corpus of read Neapolitan speech for ASR evaluation, with a validated
Neapolitan–Italian lexicon, LOSO fine-tuning splits, trained LoRA adapters,
metric implementations, per-clip results, and error annotations.
This release supersedes the earlier 141-clip single-speaker version of this
repository. The earlier release corresponds to Speaker S1 of the present
corpus; the old audioData/ and transcripts.csv are replaced by
data/audio/ and… See the full description on the dataset page: https://huggingface.co/datasets/michaelcacioli/Neapolitan-Spoken-Corpus.SpokenWOZ-Train-Audiospoken-magpie-ja
Spoken-magpie
LLMの日本語Instruction Tuning用データllm-jp/magpie-sft-v1.0をCosyVoice2 TTSを使用して音声化した商用利用可能な日本語の音声言語モデルのSFT用データセットです。
ある程度の話者多様性を持つように生成されています。
Respone Audioは500文字以下の場合にのみ生成されています。
NVIDIA H200を10枚を使用しvllmで推論しました。
Samples
最初の50サンプルを掲載します。
ID
Instruction
Instruction Audio
Response
Response Audio
0
カボチャを使ったスイーツのレシピをいくつか教えてください。
もちろんです、カボチャを使ったスイーツは秋にぴったりですね。以下にいくつかのレシピをご紹介します。1. カボチャのスフレパウンドケーキ- 材料：カボチャ 200g、生クリーム 50ml、牛乳 50ml、卵 3個、砂糖 100g、薄力粉 70g、バニラエッセンス 少々-… See the full description on the dataset page: https://huggingface.co/datasets/Atotti/spoken-magpie-ja.SpokenTermDetection_Tedlium2Train
Dataset Card for "SpokenTermDetection_Tedlium2Train"
More Information needed
SpokenVisITSpokenVisIT
SpokenVisIT is a real-world visual-speech interaction benchmark built upon VisIT-Bench, designed to evaluate the visual-grounded speech interaction capabilities of omni large multimodal models (LMMs).
Our deepest acknowledgment goes to VisIT-Bench — A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use — which collects a diverse set of real-world visual instructions. SpokenVisIT builds on this foundation by converting the textual instructions into spoken… See the full description on the dataset page: https://huggingface.co/datasets/ICTNLP/SpokenVisIT.spokenwoz_dst
Dataset: spokenwoz_whisper_dst
Short description
Prepared SpokenWoZ training dataset adapted for Whisper-style DST (dialog state tracking) tasks. Each example contains an utterance, normalized text, and aligned audio (16 kHz).
Key metadata
Number of examples: 73,950
Total size on disk: ~9.08 GB
Splits: train, validation
validation examples: 7,284
Features:
text (string): original utterance text
normalized_text (string): normalized form of the… See the full description on the dataset page: https://huggingface.co/datasets/vendrkat/spokenwoz_dst.spoken_mathspoken_norm_assignment
VietAI assignment: Vietnamese Inverse Text Normalization dataset
Dataset Description
Inverse text normalization (ITN) is the task that transforms spoken to written styles. It is particularly useful in automatic speech recognition (ASR) systems where proper names are often miss-recognized by their pronunciations instead of the written forms. By applying ITN, we can improve the readability of the ASR system’s output significantly. This dataset provides data for doing ITN… See the full description on the dataset page: https://huggingface.co/datasets/VietAI/spoken_norm_assignment.spokenwoz-whisper
SpokenWOZ With Whisper Transcripts
This is a custom version of the SpokenWOZ dataset with all the original transcripts replaced by transcripts generated by Whisper-large-v3. We manually
transcribed 1000 utterances from the test set and like this estimate the WER of the original ASR to be ~29% and the WER of the Whisper transcripts to
be ~5.3%.
Citation Information
If you happen to use these transcripts for your work, we will be glad if you cite the following works:… See the full description on the dataset page: https://huggingface.co/datasets/pirxus/spokenwoz-whisper.The_Spoken_Wikipedia_Corpora_Dutch_ASR_Hiidden
Dataset Card for "The_Spoken_Wikipedia_Corpora_Dutch_ASR_Hiidden"
Homepage: Spoken Wikipedia CorporaRepository: Bitbucket RepositoryPaper: Publication at LREC 2016Leaderboard: Interspeech 2018 PaperPoint of Contact: nats@nats.gitlab.io
Dataset Summary
The Spoken Wikipedia Corpora (SWC) is a collection of aligned spoken Wikipedia articles including articles in Dutch. It includes approximately 210 hours of audio, transcriptions, and metadata. The corpus is licensed… See the full description on the dataset page: https://huggingface.co/datasets/Jaspernl/The_Spoken_Wikipedia_Corpora_Dutch_ASR_Hiidden.chatbot-arena-spoken-voicesSpokenWOZ-Test-Text-Fixedspoken-dpo-49k
Spoken DPO Japanese
Japanese spoken DPO dataset generated from ryota39/dpo-49k using CosyVoice2 TTS.
Dataset Description
This dataset contains Japanese DPO (Direct Preference Optimization) data with spoken prompts.
prompt: Multi-turn conversation (human turns have audio)
chosen: Preferred response (text only)
rejected: Non-preferred response (text only)
Samples
ID
Conversation
Chosen
Rejected
0
Human: ドイツに社会問題があるのはなぜだと思いますか？
ドイツは高度に工業化された国なので、公害が多い。… See the full description on the dataset page: https://huggingface.co/datasets/Atotti/spoken-dpo-49k.ghomala-spoken-bible
Ghomálá' Spoken New Testament — aligned audio + trilingual text
Part of the Lingo / NativeAI language-preservation project. This is
~20 hours of spoken Ghomálá' (Ghomala, ISO bbj; a Grassfields Bantu language of
West Cameroon) — recorded readings of the New Testament — aligned chapter-by-chapter
with parallel text in Ghomálá', French, and English.
Spoken-language data is exactly what oral-first Cameroonian languages lack, which makes
this a rare resource for building ASR, TTS… See the full description on the dataset page: https://huggingface.co/datasets/flagship-ai/ghomala-spoken-bible.myanmar_spoken_corpus
Credits and Acknowledgments
This dataset is built upon the foundational work of the Myanmar Spoken Corpus by freococo (Wynn).
Original Dataset Source: freococo/myanmar_spoken_corpus
Modifications: * Curated and filtered for specific training needs of the ShweYon model.
Integrated with 3% English subset for bilingual proficiency maintenance.
Re-formatted into 36 shards for optimized Continued Pre-training (CPT).
We are deeply grateful to freococo for their contribution to the… See the full description on the dataset page: https://huggingface.co/datasets/URajinda/myanmar_spoken_corpus.spoken_squad
Dataset Card for Spoken-SQuAD
Citation
@article{lee2018spoken,
title={Spoken SQuAD: A Study of Mitigating the Impact of Speech Recognition Errors on Listening Comprehension},
author={Lee, Chia-Hsuan and Wu, Szu-Lin and Liu, Chi-Liang and Lee, Hung-yi},
journal={Proc. Interspeech 2018},
pages={3459--3463},
year={2018}
}