TTS-AGI/moss-reference-voices-consolidated

MOSS reference voices — consolidated (6,064 voices)

6,064 reference voices for casting MOSS-VA-v2 voice-acting generations. Every voice was auto-annotated by Gemini (name, tagline, language, accent, age/gender read, register, timbre, distinctive features, emotional range, casting suggestions for 4 genres, free-text tags, search text) and scored on 99 measured dimensions: 57 VoiceNet voice-quality axes (timbre/prosody/register/speaking-style, e.g. brightness, roughness, warmth, formality, ASMR-ness, narration-ness…), 40 Empathic-Insight emotion axes (amusement, distress, anger, longing, …), plus genuineness (does it sound like a real human recording) and burst-blend (how cleanly non-speech vocal bursts blend into the speech). Each voice ships with three audio variants and per-variant DNSMOS, so a consumer can pick the best-sounding take per voice.

Repository is private. Research use.

Live search UI: https://projects.laion.ai/moss-reference-voice-search/ — code + reproducer under `search_tool/` in this repo.

Composition

Counts by source (= meta.json[i]["source"], also the cid prefix):

Language/accent are populated for the voices where Gemini could identify one from the audio (4,727 of 6,064 non-null; predominantly English (4,250) and German (473), the latter almost entirely the Mediathek family). Gender read (meta.json[i]["gender"]): Male 4,262 · Female 1,675 · Androgynous 109 · Non-human 8 · a handful of Masculine/Feminine/null edge cases.

The three audio variants — and why

Each voice has three parallel renders of the same demo clip:

Finding (see `annotations/dnsmos_stats.json`): at the dataset level, mean DNSMOS-OVRL is essentially tied across variants — orig 3.343, sidon 3.346, cbx 3.344 — so no single variant is best on average. But per-voice, ~62% of voices are improved by one of the two processed variants over the raw original (win-rate: orig 38.2%, sidon 32.0%, cbx 29.8% — i.e. sidon or cbx wins on 61.8% of voices). Which variant wins is voice-specific, not predictable in aggregate — use the precomputed `best_version` field (argmax DNSMOS per voice) shipped in every per-sample record and in metadata.parquet to pick the best take for a given voice, rather than defaulting to one variant dataset-wide.

File layout

data/voices-0000.tar … voices-0011.tar   # WebDataset shards, ~505-506 voices each, ~2 GB total
metadata.parquet                          # flat index, one row per voice (see below)
annotations/
  dims.npy            # (6064, 99) float32 — 99-dim scores on the ORIGINAL (orig) audio
  dims_enh.npy        # (6064, 99) float32 — 99-dim scores on the SIDON (sidon) audio
  dim_catalog.json    # the 99-dim schema: [{i, code, name, group, desc}, ...]
  dnsmos.json         # {cid: {orig, sidon, cbx}} DNSMOS-OVRL per variant, all 6064 voices
  dnsmos_stats.json   # dataset-level DNSMOS summary (mean + win% per variant)
search_tool/
  server/server.py, server/API.md, server/dim_catalog.json   # FastAPI search server (BM25 / embedding / VoiceCLAP + 99-dim filters)
  docs/index.html                                             # static demo search page
  pipeline/            # the scripts that built this dataset end-to-end (SIDON, Chatterbox, DNSMOS, dim scoring, VoiceCLAP, assembly)
  README.md            # how to reconstruct a runnable DS_DIR from this repo and run the server
README.md              # this file

WebDataset shards (data/*.tar)

Each tar holds a whole number of voices; members for one voice are contiguous:

<cid>.orig.mp3     # original demo clip
<cid>.sidon.mp3    # SIDON-denoised + loudness-normalized
<cid>.cbx.mp3      # Chatterbox self-conversion of the sidon clip
<cid>.json         # full per-voice record (see below)

The <cid>.json record = the voice's full meta.json entry (name, tagline, gender, age, language, accent, register, timbreprofile, distinctivefeatures, emotionalrange, casting {classicfantasy, scifi, mysteryhorror, contemporary}, tags, search_text, legacy scores, source) plus:

{
  "dnsmos": {"orig": 3.44, "sidon": 3.40, "cbx": 3.29},
  "best_version": "orig",
  "dims_raw": [ /* 99 floats, order = annotations/dim_catalog.json, scored on orig audio */ ],
  "dims_enh": [ /* 99 floats, same order, scored on sidon audio */ ]
}

Load with WebDataset:

import webdataset as wds

ds = wds.WebDataset(
    "hf://datasets/TTS-AGI/moss-reference-voices-consolidated/data/voices-{0000..0011}.tar"
).decode()

for sample in ds:
    cid = sample["__key__"]
    orig_mp3 = sample["orig.mp3"]     # bytes
    sidon_mp3 = sample["sidon.mp3"]
    cbx_mp3 = sample["cbx.mp3"]
    rec = sample["json"]
    print(cid, rec["name"], rec["best_version"])

(Requires huggingface_hub's hf:// support, or hf_hub_download each tar locally first and point WebDataset at the local glob.)

metadata.parquet — flat index

One row per voice, for fast filtering/joining without touching audio: cid, name, gender, age, language, accent, tagline, tags, source, dnsmos_orig, dnsmos_sidon, dnsmos_cbx, best_version, shard, plus key 99-dim values (both orig-scored and _enh = sidon-scored) for the dimensions most people filter/sort on: dim_GEND, dim_AGEV, dim_GENU, dim_BLEND, dim_BKGN, dim_VALN, dim_AROU, dim_WARM (+ _enh variants). Full 99-dim vectors live in annotations/dims.npy / dims_enh.npy (row i = metadata.parquet row i, same order as meta.json) and in every tar sample's dims_raw/dims_enh.

import pandas as pd
df = pd.read_parquet("metadata.parquet")
loud_masculine = df[(df.dim_GEND > 4) & (df.dnsmos_orig > 3.4)]

99-dim schema

annotations/dim_catalog.json is a list of 99 {i, code, name, group, desc} entries, group ∈ {emonet (40), voicenet (57), quality (2)}. Key codes used in metadata.parquet: GEND (perceived gender, higher = more masculine), AGEV (perceived age), GENU (genuineness — sounds like a real human recording), BLEND (vocal-burst blend quality), BKGN (background noise level), VALN/AROU (emotional valence/arousal), WARM (vocal warmth). dims.npy rows are scored on the orig audio, dims_enh.npy on the sidon audio — compare the two to see how denoising shifted a voice's measured profile.

DNSMOS

annotations/dnsmos.json = {cid: {orig, sidon, cbx}}, DNSMOS-OVRL (0–5, higher = better perceived audio quality) computed independently per variant for all 6,064 voices. annotations/dnsmos_stats.json has the dataset-level means and per-variant win-rates referenced above.

License

CC-by-4.0

Search

See `search_tool/` for the full FastAPI search server (BM25 / sentence-embedding / VoiceCLAP text→audio similarity, with optional AND-filters over any of the 99 dimensions), the pipeline scripts that produced this dataset end-to-end (SIDON enhancement, Chatterbox self-conversion, DNSMOS scoring, 99-dim scoring, VoiceCLAP embedding, dataset assembly), and a live-demo reproducer README. Live instance: https://projects.laion.ai/moss-reference-voice-search/.