CoolFace
Apppublic

prism-v/open_vocoder_leaderboard

sourceHugging Facemitupdated 10d agoView on Hugging Face
0likes
App README

๐ŸŽ™๏ธ PRISM-V: Open Evaluation of Pretrained Neural Vocoders

Quality ยท Robustness ยท Generalization ยท Efficiency A multidimensional benchmark for understanding how modern pretrained neural vocoders perform across datasets, architectures, acoustic conditions, and deployment constraints.

Overview

PRISM-V evaluates 15 publicly-released, pretrained neural vocoder checkpoints in a fully training-free, zero-shot setting across 4 diverse English corpora (LJSpeech, LibriTTS, VCTK, Free_ST), with GPU hardware profiling under Edge Profiling.


Key Features

DimensionDetails
๐Ÿ† Models15 architectures: BigVGAN-v2, Vocos, ComVo, PeriodWave, Flow2GAN, BridgeVoC, RFWave, WaveFM, RNDVoC, FreeV, HiFi-GAN, Griffin-Lim
๐Ÿ“Š CorporaClean studio (LJSpeech), Multi-speaker (LibriTTS), Accented (VCTK), Real-world Mobile (Free_ST)
๐Ÿ”ฌ Phoneme DiagnosticsMFA phone-level alignment โ†’ per-ARPAbet class LSD, F0 error, boundary distortion
โšก Hardware ProfileEdge Profiling with synchronous CUDA barriers
๐ŸŽง Audio ExplorerInteractive side-by-side listening: Ground Truth vs any two models
๐Ÿ“ˆ Pareto FrontierPareto-optimal models identified across Quality (PESQ/UTMOS) vs Efficiency (RTF/VRAM/Params)

Benchmark Navigation

PagePurpose
๐Ÿ† LeaderboardMain model ranking with interactive column selectors and dataset toggles
โš–๏ธ CompareDirect side-by-side model comparison with radar charts and metric difference tables
๐ŸŽง Audio ExplorerListen to reference & model reconstructions across corpora
๐Ÿ“Š Quality & RobustnessCross-corpus heatmaps and generalization analyses
โšก EfficiencyThroughput (xRT), real-time factor (RTF), latency, and peak VRAM
๐Ÿง  ArchitecturesGAN vs Flow vs Fourier/Transformer vs Diffusion structural breakdown
๐Ÿ”ฌ DiagnosticsPhoneme-level error heatmaps and failure analyses
๐Ÿ“ˆ Pareto FrontierMulti-objective trade-off frontiers
๐Ÿ“ฆ ModelsIndividual model cards with checkpoint and code provenance
๐Ÿงช MethodologyBenchmark protocol and metric definitions
โž• Submit ModelProcedure to add new vocoder checkpoints
๐Ÿ“œ Changelog & CitationVersion history and copyable citation

Citation

bibtex
@misc{purohit2026prismv,
  author       = {Ravindrakumar M. Purohit and Hemant A. Patil},
  title        = {{PRISM-V}: Multidimensional Evaluation of Pretrained
                  Neural Vocoders for Speech Synthesis},
  year         = {2026},
  howpublished = {Hugging Face Space},
  note         = {Open neural vocoder evaluation leaderboard}
}