CoolFace
Datasetpublic

ampixa/neptts-bench

NepTTS-Bench Dataset First comprehensive benchmark for evaluating Nepali text-to-speech systems. Contents sentences.json — 365 phonologically-designed Nepali sentences with metadata audio/ — TTS outputs from 12 systems (~2000 audio files) results/ — Pre-computed evaluation results (SCOREQ, Chirp2, MMS, XLS-R, Whisper) baselines.json — Aggregate scores for all baseline systems model/ — NepaliMOS predictor checkpoint (Spearman 0.587) Systems Evaluated… See the full description on the dataset page: https://huggingface.co/datasets/ampixa/neptts-bench.

sourceHugging Facemitupdated 4mo agoView on Hugging Face
0likes126downloads
Dataset Card

NepTTS-Bench Dataset

First comprehensive benchmark for evaluating Nepali text-to-speech systems.

Contents

  • —sentences.json — 365 phonologically-designed Nepali sentences with metadata
  • —audio/ — TTS outputs from 12 systems (~2000 audio files)
  • —results/ — Pre-computed evaluation results (SCOREQ, Chirp2, MMS, XLS-R, Whisper)
  • —baselines.json — Aggregate scores for all baseline systems
  • —model/ — NepaliMOS predictor checkpoint (Spearman 0.587)

Systems Evaluated

SystemHuman MOSType
Natural Speech3.91Human
TingTing Asmita3.49Nepali-specific
ElevenLabs v33.48Cloud
Piper3.47Open source
TingTing Subina3.42Nepali-specific
Edge TTS Hemkala3.31Cloud (Microsoft)
Edge TTS Sagar3.28Cloud (Microsoft)
Gemini Flash3.19Cloud (Google)
gTTS2.56Cloud (Google Translate)

Usage

python
from datasets import load_dataset

ds = load_dataset("ampixa/neptts-bench")

Or use the evaluation CLI:

bash
pip install neptts-eval
neptts-eval --wav_dir ./my_tts_outputs/

Citation

bibtex
@article{neptts-bench-2026,
  title={NepTTS-Bench: A Comprehensive Benchmark for Nepali Text-to-Speech Evaluation},
  author={Ampixa},
  year={2026}
}

Links