Datasetpublic

beatsprom/audio-speech-realtime-voice-agents-2026

🎙️ Audio, Speech Foundation Models & Real-Time Voice Agents Dataset (2026 Edition) A structured research dataset featuring 1,722 domain-verified research papers and 298 official code repositories focused on Full-Duplex Speech-to-Speech LLMs, Real-Time Voice Agents (<200ms Latency), Zero-Shot TTS, Voice Cloning, OpenAI Whisper-v3, Neural Audio Codecs (EnCodec/DAC/SNAC), and Generative Music (2023–2026). Built with Universal Scientific Engine V18.1 Diamond, providing 48 schema… See the full description on the dataset page: https://huggingface.co/datasets/beatsprom/audio-speech-realtime-voice-agents-2026.

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes43downloads

Dataset Card

🎙️ Audio, Speech Foundation Models & Real-Time Voice Agents Dataset (2026 Edition)

A structured research dataset featuring 1,722 domain-verified research papers and 298 official code repositories focused on Full-Duplex Speech-to-Speech LLMs, Real-Time Voice Agents (<200ms Latency), Zero-Shot TTS, Voice Cloning, OpenAI Whisper-v3, Neural Audio Codecs (EnCodec/DAC/SNAC), and Generative Music (2023–2026).

Built with Universal Scientific Engine V18.1 Diamond, providing 48 schema attributes with verified repository attribution, 8 AI topological semantic clusters, pre-calculated Top-3 Semantic Nearest Neighbors Graph, structured benchmark leaderboards, and native 384-dimensional dense PyTorch embeddings.

📊 Dataset Schema Highlights (48 Columns)

Field	Type	Description
`paper_id`	String	Unique ArXiv identifier
`title`	String	Research paper title
`cluster_topic_name`	String	1 of 8 AI Topological Semantic Clusters
`audio_speech_task_paradigm`	String	Task paradigm (Full-Duplex Voice LLM, Zero-Shot TTS, ASR, Neural Codec)
`speech_latency_profile`	String	Latency profile (<200ms Streaming, Streaming Chunk ASR, <500ms Duplex)
`acoustic_sampling_rates`	List[String]	Supported sampling rates (16 kHz, 24 kHz, 44.1 kHz, 48 kHz Hi-Fi)
`speech_foundation_backbone`	String	Model backbone (Whisper-v3, Moshi, CosyVoice, ChatTTS, F5-TTS, MusicGen)
`tested_benchmarks`	List[String]	Evaluated benchmarks (LibriSpeech, CommonVoice, Seed-TTS, AudioCaps)
`benchmark_leaderboard_json`	List[Struct]	Structured WER, MOS, RTF, and Latency scores
`semantic_nearest_neighbors_top3`	List[Struct]	Pre-calculated top-3 related papers with cosine similarity
`commercial_ip_safety_score`	Integer	0–100 commercial compliance index (95% Enterprise Safe)
`tldr_neural_summary`	String	15-word executive summary of key innovation
`title_vector_384d`	List[Float]	384d PyTorch embedding (`all-MiniLM-L6-v2`)
`abstract_vector_384d`	List[Float]	384d dense contextual PyTorch embedding
`reproduction_recipe`	String	1-line bash setup command

🧩 8 AI Semantic Clusters Breakdown

1.Neural Audio Codecs & Discrete Acoustic Tokenizers (375 papers)
2.End-to-End Multilingual Speech Recognition (ASR & Whisper) (254 papers)
3.Speech Enhancement, Denoising & Source Separation (247 papers)
4.Audio-Visual Speech Synthesis, Talking Heads & Lip-Sync (233 papers)
5.Generative Music, Audio Diffusion & Sound Synthesis (198 papers)
6.Low-Latency Zero-Shot TTS & Instant Voice Cloning (191 papers)
7.Spoken Dialogue Evaluation & Acoustic Benchmarking (137 papers)
8.Full-Duplex Voice Agents & Speech-to-Speech LLMs (87 papers)

📊 Interactive OpenAngels Visual Dashboard Included

Open DATASET_ANALYTICS_DASHBOARD_100_SAMPLE.html directly in your browser (Chrome/Edge/Safari) to explore the interactive visual intelligence directory with real-time filtering, search, and audio latency metrics.

💻 1-Click Python Quickstart

python

import pyarrow.parquet as pq

# Load 100-Sample Teaser
table = pq.read_table("AUDIO_SPEECH_FOUNDATION_MODELS_REALTIME_VOICE_AGENTS_2026_100_SAMPLE.parquet")
df = table.to_pandas()

print(f"Loaded {len(df)} sample Audio & Speech AI papers.")
print(f"Top Paper: {df['title'].iloc[0]}")
print(f"Task Paradigm: {df['audio_speech_task_paradigm'].iloc[0]}")
print(f"Latency Profile: {df['speech_latency_profile'].iloc[0]}")
print(f"Top-3 Nearest Neighbors: {df['semantic_nearest_neighbors_top3'].iloc[0]}")

🚀 Get the Full 1,722-Paper Enterprise Edition

The complete commercial production dataset (1,722 papers in Parquet with 384d vectors, SQLite DB, Clean CSV, Interactive OpenAngels HTML Dashboard, and JSON) is available here:

👉 [BeatsProm Audio, Speech & Real-Time Voice Agents Dataset Full Edition](https://beatsprom.gumroad.com/l/audio-speech-real-time-voice-agents-2026)