laion/tts-scaling-ladder-de-en
Balanced Expressive Speech: German–English Scaling Views A nested set of ten cumulative training views for reproducible expressive TTS and audio-model studies. Eight base tiers were designed to be approximately 50/50 German/English and, within each language, 50/50 real/synthetic; each cell reserves half its volume for coverage of 40 EmoNet emotion categories and half for 57 VoiceNet dimensions × 10 value bins. Two later, non-overlapping extensions yield cumulative views of 49… See the full description on the dataset page: https://huggingface.co/datasets/laion/tts-scaling-ladder-de-en.
Balanced Expressive Speech: German–English Scaling Views
A nested set of ten cumulative training views for reproducible expressive TTS and audio-model studies. Eight base tiers were designed to be approximately 50/50 German/English and, within each language, 50/50 real/synthetic; each cell reserves half its volume for coverage of 40 EmoNet emotion categories and half for 57 VoiceNet dimensions × 10 value bins. Two later, non-overlapping extensions yield cumulative views of 49,569.2 and 98,920.1 measured hours. The larger view falls short of the nominal 100,000-hour build target; it is not a 100,000-hour measurement. Every smaller training view is contained in the next larger one. The 80.1-hour holdout is separate.
The release is intended to support future from-scratch TTS training, scaling comparisons, and exploration of broad-to-selective curricula. The from-scratch Humaneness Voice follow-up was not completed at the time of the manuscript submission; the existing Humaneness Voice baseline is a fine-tune of MOSS, not a model trained on this full ladder. Source-specific audio rights apply throughout, including EuroSpeech's parliament-specific terms. The full ladder is not uniformly commercially permissive even though its metadata are publicly documented.
Every element ships audio (MP3, byte-identical to its source repository), MOSS-Audio-Tokenizer-v2 codes (uint16 [T, 12], 12.5 fps) and a JSON record with the procedural voice-acting prompt in the `LAION-AI/procedural-voice-captions` format (GENERAL line + SCRIPT with per-sentence delivery cues, [pause X.Xs] markers and inline (Vocal Burst) labels), plus the raw scores it was built from, the language, the origin, the source, the source uid, the source licence and the duration.
How to read this table. Durations are measured cumulative views, not ten additive datasets. stage50 contains all of Core plus 29,567.7 extension hours; stage100 contains all of Expanded plus 49,350.9 further extension hours. The existing configuration names are retained for code compatibility; Broad, Expanded and Core are explanatory display names. An illustrative, untested curriculum would train for one epoch on Broad, then one epoch on each next-smaller view, down to Tier 0. Because sets nest, high-ranked clips are deliberately revisited. The schedule has not been validated as better than alternatives and does not describe training of the current MOSS-based Humaneness Voice checkpoint. Filter per-record licence fields before any use that requires a narrower rights set; doing so changes measured hours and balance.
Held-out split. 20 h per cell (10 h per pipeline) drawn with exactly the same rules as the tiers — the next elements behind the tier-7 boundary of every bucket/category in master-rank order, same normalisation, same quotas — and disjoint from all tiers (tier = 8, verified by uid: zero overlap). It lives under data/*/*/heldout/ and index/*/*/heldout/, loader config heldout. Use it for the scaling-law loss; never train on it.
Hours are summed over dur_s. Tier N ⊂ Tier N+1 strictly — by construction, not by check: the whole release is stored once (the tier-7 stock plus held-out, 6,692,637 elements, 20,081.6 h, 2,694 shards, 1.04 TB of tars) with a column tier = the smallest tier index that contains the element, and the tierN loader configs simply filter tier <= N.
Two assumptions made by the executor (the specification left them open)
- 50/50 split between the two pipelines. Inside every cell (language × origin) half of the volume is filled by pipeline A (uniform over 40 EmoNet categories) and half by pipeline B (uniform over 57 VoiceNet dimensions × 10 deciles). The specification fixed the strata but not the share; halves were the coordinator's assumption.
- One stock, one `tier` column. The release is stored exactly once (the tier-7 stock plus the held-out split) and every element carries
tier= the smallest tier that contains it. Tier K is therefore defined astier <= K(and, on disk, as the tar directoriestier0…tierK); nesting is guaranteed by construction rather than by shipping eight overlapping copies.
The remaining executor's choices (pool definition, decile edges, deduplication order, bucket shortfall handling, language verdicts) are marked [A] in the rules below.
Loading — one example per tier
Every tierN config is the union of the tar/parquet directories tier0 … tierN; heldout is the evaluation split.
from datasets import load_dataset
R = "laion/tts-scaling-ladder-de-en"
t0 = load_dataset(R, "tier0") # 100 h (50 DE + 50 EN; 25 h real + 25 h synth per language)
t1 = load_dataset(R, "tier1") # 250 h
t2 = load_dataset(R, "tier2") # 500 h
t3 = load_dataset(R, "tier3") # 1,000 h compute-optimal anchor
t4 = load_dataset(R, "tier4") # 2,500 h
t5 = load_dataset(R, "tier5") # 5,000 h
t6 = load_dataset(R, "tier6") # 10,000 h
t7 = load_dataset(R, "tier7") # 20,000 h everything
ev = load_dataset(R, "heldout") # 80 h evaluation, disjoint from every tier
de = load_dataset(R, "de") # one language, all tiers: filter de["train"]["tier"] <= N yourselfEach row is one element (metadata, scores, prompt); audio and MOSS codes are in the tar with the same basename:
Audio and MOSS codes live in WebDataset tars next to the index, one tar per index parquet, same basename:
import io, json, tarfile, numpy as np
from huggingface_hub import hf_hub_download
row = idx["train"][0] # any index row
p = hf_hub_download("laion/tts-scaling-ladder-de-en", f"data/{row['lang']}/{row['origin']}/tier{row['tier']}/{row['shard']}.tar", repo_type="dataset")
with tarfile.open(p) as tf:
mp3 = tf.extractfile(f"{row['uid']}.mp3").read() # bytes, decode with soundfile/torchaudio
moss = np.load(io.BytesIO(tf.extractfile(f"{row['uid']}.moss.npy").read())) # uint16 [T, 12]
rec = json.load(tf.extractfile(f"{row['uid']}.json")) # prompt, words, sentences, bursts, scores
print(rec["prompt"])Tar members per element: <uid>.mp3, <uid>.moss.npy, <uid>.json. Every tar of tier block tierN holds only elements whose smallest tier is N, so tier K = all tars under tier0 … tierK. For streaming with webdataset:
import webdataset as wds
K = 3
urls = [f"https://huggingface.co/datasets/laion/tts-scaling-ladder-de-en/resolve/main/data/{lang}/{origin}/tier{n}/{shard}.tar"
for (lang, origin, n, shard) in shards_of_tier(K)] # from the index: columns lang, origin, tier, shard
ds = wds.WebDataset(urls).decode().to_tuple("mp3", "moss.npy", "json")Decoding note. The voice-profile sources (vprof_original, vprof_repair) are 48 kHz MP3 with two identical channels; decode with soundfile/torchaudio and take mean(axis=channels) — never reshape(-1) (that yields a half-speed signal of twice the length). All other sources are mono. audio_sr and audio_channels are in the index.
Selection rules (verbatim from the specification, with the executor's assumptions marked [A])
Licence filter (before any scoring)
Only sources with documented provenance and release terms entered this published pool; this is not a claim of uniform commercial reuse rights. Real: emolia (DE+EN), mls (DE; the corpus has no English MLS), eurospeech (DE+EN, explicitly released by the project lead), kartoffelphon (DE), commonvoice (DE and EN subsets only). Synthetic: laion-voice-profiles-annotated (origin=original, origin=repair, vc_sidon — 500 LAION voice profiles, DE/EN in halves), laion/voice-acting-edge-top3 (formerly laion/dramabox-edge-top3-voice), laion/voice-acting-reinterpretations-top3 (formerly laion/dramabox-reinterpretations-top3) (MOSS takes). Excluded: podcast, snippets, evasnippets (research repo, non-commercial), everything Mediathek, everything without a documented licence, laion/voice-acting-burst-audio (no score columns), laion-emotional-trajectory-t80 (chains concatenated from vc_sidon clips — duplicates — and 93 % language-mixed), laion/moss-voice-identity-repairs (identical material to origin=repair, which carries the MOSS codes and scores).
Scoring — all metrics min-max normalised per language × origin cell
S_gen = Norm(genuineness_0_6), S_vb = Norm(blend_0_10), S_emo,e = Norm(emo_e) for the 40 EmoNet dimensions, S_vn,d = Norm(vn_d_reg) for the 57 VoiceNet dimensions. R_quality = S_gen + S_vb; R_emotion(e) = 3·S_emo,e + R_quality. [A] min/max are exact over the whole cell pool (all sources of the cell together).
Stratification
Per cell (language × origin) the cell volume is split in halves between two pipelines ([A], coordinator's assumption).
- Pipeline A — EmoNet, 40 categories. For category e: candidates sorted by
R_emotion(e); the intensity pool [A] is defined as nested bands: band k (k = 0…7) = the top prefix whose hours reach m · q_A(k) with m = 3 (the pool covers the quota threefold), qA(k) = pipeline-A volume of tier k / 40. **Master rank** in A = (band ascending, then `Rqualitydescending). Tier 0 is therefore the best-quality third of the most extreme band; every larger tier adds less extreme emotion and again ranks strictly by quality inside. Quota per category = 1/40 of the pipeline-A volume. If a category's pool is exhausted by deduplication, the ranking continues below band 7 inR_emotion(e)order (band = 8`, counted in the report). - Pipeline B — VoiceNet, 57 dimensions × 10 deciles = 570 buckets. For each dimension the value range
[min, max]of the cell is cut into 10 equal-width intervals ([A]: equal width, not quantiles); inside each bucket the master rank is strictlyR_quality. Quota per bucket = 1/570 of the pipeline-B volume, so extreme values (very low voice, very young age) are not disadvantaged. [A] A bucket with fewer available hours than its quota takes everything it has; the shortfall is redistributed proportionally to the other deciles of the same dimension (then, if needed, across all buckets of the pipeline). - Deduplication. An element counts once. [A] Fixed order: pipeline A first, then B; inside A the 40 categories in alphabetical order (all pools are equal-sized); inside B the 570 buckets in ascending order of available candidate hours (scarcest first, which protects the extreme deciles), ties by (dimension, decile). An element already taken is skipped by later buckets. Elements that would have fallen into more than one window are counted (
n_candidacies). - Nesting. Per bucket/category the deduplicated list is walked in master-rank order, hours accumulated, and each element receives
tier = min{N : cum_h ≤ q(N) + dur/2}(rounding to the nearest quota); elements beyond q(7) are dropped. [A] Bucket rounding noise is then balanced at the tier boundaries by moving boundary elements (last of tier n ↔ first of tier n+1) until every (cell, pipeline, tier) is within 0.25 % of its target — the prefix property is preserved. Stored once withtier,rank_in_bucket,bucket_id,pipeline,emotionorvn_dim/decile, plusband(A) andn_candidacies. - [A] Language = the corpus
langverdict (upstream label + ASR LID,lang_src == agreedon 99.6 %); for the DramaBox sources fastText lid.176 over the ASR transcript. Duplicate uids inside a source (Emolia ships some clips in two upstream archives) are dropped before selection, first occurrence kept. No duration filter beyonddur_s > 0.
Licences per source
The license column carries this per element. If your use case cannot accept other, filter license to CC-BY-4.0 / CC-0 (this removes eurospeech; the tiers stay nested but no longer exactly balanced).
Balance (hours, from the shipped index)
Targets per (cell, pipeline), tiers 0–7: 12.5 / 31.25 / 62.5 / 125 / 312.5 / 625 / 1,250 / 2,500 h. Largest deviation over the 64 tier values: 0.204 %; over the eight held-out values: 0.238 %. All are within 0.25 %.
Hours per source (tier 7 plus held-out)
The prompt (what rec["prompt"] / index prompt contains)
Produced offline from the stored annotations with the captioner of `LAION-AI/procedural-voice-captions` at main (PR #4: 17-class x2 head and energy-based pause correction), tags template, deterministic seed = stable_hash(uid):
GENERAL: very warm, very soft-onset, …, middle-aged, low-register, very slow, pining, helplessness, semi-genuine, with-bursts
SCRIPT:
(very flat, very smooth, no interest, very fatigue, genuine) Huh. (Chuckle)
(very flat, very warm, helplessness, concern, semi-genuine) [pause 0.4s] What have we here?- GENERAL — top-5 VoiceNet dimensions by |z|·reliability, top-3 emotions, genuineness, burst blend, always Age/Gender/Register/Tempo, z-scored against the bundled in-domain baseline.
- SCRIPT — one line per sentence:
(delivery cue) sentence text. Sentences are cut at sentence-final punctuation of the aligned words. [A] The cue of every sentence is derived from the clip's global scores (the corpora carry no per-sentence scores; the record stores the sentence timings so per-sentence re-scoring can be dropped in later). Wording rotates deterministically per sentence. - `[pause X.Xs]` — a gap ≥ 0.30 s between consecutive words or sentences, measured from the stored word timestamps after the energy-based word-end correction (each word's
endis trimmed back to where its speech actually stops, computed from the decoded audio;end_spankeeps the raw value). Word timestamps are the corpus alignments (wordsin the source JSON — MMS-FA for the LAION corpora, Parakeet for the DramaBox sources); Parakeet was not re-run. - `(Vocal Burst)` — the kept locator/classifier-v2 spans of the source annotation, written at the word gap where they happen (
BURST_LABEL_SOURCE=v2).prompt_x2is the same prompt with the 17-class x2 head naming the spans (BURST_LABEL_SOURCE=x2, recall-tiered wording: exact class /Class?/ family word /Vocal Burst) — present where thevb2_*re-annotation exists (3,689,357 of 6,692,637 elements: the five real sources,vc_sidonshards with a re-annotation,dramabox_reint, anddramabox_edgewhere its re-annotation had landed),nullelsewhere. - Totals in this release: 8,508,596 pause markers, 3,928,534 kept bursts.
rec["sentences"],rec["words"],rec["bursts"]andrec["scores"]are the recordaugment.pyneeds to regenerate varied phrasings of the same content at training time (augment_script(rec, seed)).
Columns
See `COLUMNS.md`. Every index row carries the raw scores (genuineness_0_6, blend_0_10, 40 emo_*, 57 vn_*_reg and, where the source has them, 57 vn_*_bucket), lang, origin, source, uid (= source uid), hf_source, license, dur_s, the selection fields and the prompt.
Provenance and verification
Built on JUPITER (JSC) from the local copies of the source repositories; nothing was re-encoded — every MP3 is the byte-identical member of its source tar, every MOSS code array is the source's (laion-tts-annotated-v1, laion-voice-profiles-annotated) or, for the two DramaBox sources, newly computed with OpenMOSS-Team/MOSS-Audio-Tokenizer-v2 at 48 kHz, n_vq = 12, with the frame-rate canary moss_frames / (decoded_duration × 12.5) ∈ [0.97, 1.03] checked on every element (the packer refuses otherwise). Upload with upload_large_folder; every file verified byte-exact against the local copy via repo_info(files_metadata=True). Selection code, statistics and the full report: ladder.json in this repo; the working report is scaling_ladder.md in the project.
Citation
If you use this ladder, credit LAION and the upstream sources listed above.
50k and 100k extensions (2026-09-19)
This release extends the frozen 20k ladder with two incremental, non-overlapping container sets. The files are stored under logical x50/ and x100/ directories; the build system's incidental physical tier5//tier1/ source directories are not part of the public layout.
stage50 loads the 20k tier stock plus x50 (approximately 49.6k measured hours); stage100 additionally loads x100 (approximately 98.9k measured hours). The latter is deliberately called stage100, not a claim of exactly 100,000 hours: 51 planned shards were not packed after a documented 12-hour timeout (4 de/synth, 47 en/synth), leaving the final total about 1,082 hours below its target. This is a transparent quota shortfall, not a missing manifest entry.
The original tier0–tier7 and held-out containers are unchanged. The x50/x100 records use the same WebDataset triplets and MOSS-v2 convention as the base release.
Extension release checks
Before publication, every extension Parquet was paired with its TAR and .tar.ok marker; deterministic canaries in every stage/language/origin cell verified complete {mp3, moss.npy, json} triplets, loadable uint16 [T,12] NumPy MOSS matrices, matching JSON frame counts, and a 12.5-Hz duration ratio in [0.95, 1.05]. The release audit report and canary SHA-256 values are retained with the training run.
License remains mixed per source
The repository-level identifier intentionally remains other, not CC-BY-4.0. The extension contains CC-BY-4.0 and CC-0 records and, in de/real x100, the existing eurospeech value other (parliament terms; released by project lead). The per-record license column and the source table above are authoritative. Filter to CC-BY-4.0 / CC-0 if the other terms are unsuitable for a use case.
Annotation sidecars for expressive TTS and audio-language training
Release note. This section specifies the additive annotation release that accompanies the ladder update. Publish the README and the paths below in the same commit; the annotations are not part of the older public commit 92ae1518f80088a64372138dcfd450a0290e6e66.The completed annotation pass targets 3,682,649 unique clips, consisting of 3,425,099 clips from the base ladder and 257,550 clips from the synthetic voice-conversion sidecar. Its production directory and historical physical manifest are named S3, but the corrected ten-stage curriculum calls this full-coverage set logical S4, or tier6 + sidecar tier6. It is not the whole public tier7 stock. Every target UID has a non-empty BUD-E Whisper V1.1 caption, Timbre-Whisper output, Voice-Tagging-Whisper output, and one validated Gemma-4-E4B DramaBox-style instruction.
Public file contract
The strings are stored once in aligned Parquet containers and joined by uid; the smaller stages use compact membership Parquets instead of copying millions of captions:
annotations/
s3/
whisper/whisper.part-00000.parquet ... whisper.part-00255.parquet
gemma/gemma.part-00000.parquet ... gemma.part-00255.parquet
ANNOTATION_DATASET.json
FINAL_SUMMARY.json
ladder-membership/
S5.membership.parquet
S6.membership.parquet
S7.membership.parquet
S8.membership.parquet
S9.membership.parquet
S10.membership.parquet
FINAL_SUMMARY.json
bude-v1/bude_caps.parquetThe two files with a matching five-digit part number contain the same UIDs in the same order. ANNOTATION_DATASET.json records row counts and SHA-256 hashes for all 256 pairs. Each membership row is simply {uid: string, annotation_part: int16}. The authoritative v2 descriptor maps the logical curriculum as follows:
Only logical S5-S10 require membership Parquets. Logical S4 is exactly the annotated tier6 UID set, while logical S1-S3 are larger supersets with partial annotation coverage. The published logical filenames normalize older internal physical labels: S5 sources physical/S4, S6 sources physical/S5, S7 sources physical/S5b, S8 sources physical/S6, S9 sources physical/S5c, and S10 sources physical/S7.
The original audio and codec targets remain in the WebDataset TARs under data/<lang>/<origin>/<tier-or-extension>/<shard>.tar. Sidecars never duplicate audio. The Whisper Parquet columns record_path, record_member, and audio_member are provenance locators from the build workspace; portable loaders should resolve a clip from its ladder index fields (lang, origin, tier, shard, uid) rather than relying on those absolute build paths.
What each annotation means
The model revisions used for this release are BUD-E-Whisper_V1.1@9a4b11214f2f37e87a05dc75a51edd472e7bba59, timbre-whisper@0eba2ef9f74ef8a37a31774dbf9fbbd40568e6fe, voice-tagging-whisper@dc223dda0fabb29a90a9d8c16b1e8ae5ff0988fe, and gemma-4-E4B-it@ee0ef6023621cff504d758262d4e04895a5af4a2. The original BUD-E V1.0 join contains 10,629,084 unique UIDs and is useful as a historical alternative; it is not silently substituted for V1.1.
Related public datasets are laion/Emolia, laion/laion-tts-annotated-v1, laion/laion-voice-profiles-annotated, laion/voice-acting-edge-top3, and laion/voice-acting-reinterpretations-top3. The synthetic curriculum sidecar does not yet have a verified public repository; do not infer or invent a Hub identifier for it.
One real, sanitized aligned example
The following text comes from one completed Emolia row. The UID and local filesystem locators are omitted here; stored strings are otherwise shown as produced. Raw model outputs are retained even when their wording is imperfect.
Verified transcript
Die größten Herausforderungen waren im letzten Jahr.Original BUD-E Whisper V1.0 caption (historical Emolia join)
A medium-quality recording of a male speaker talking in a conversational tone.
He sounds somewhat neutral, possibly slightly interested. The audio quality is
decent, with no noticeable background noise.BUD-E Whisper V1.1 caption
InIn a slightly subdued, neutral tone, an adult male voice expresses a mild
sense of Contentment and Interest, with a hint of Concentration. The delivery
is natural and spontaneous, featuring a slightly low-pitched, moderate-to-mid
pitch range, a somewhat monotonous and steady tempo, and clear articulation.
The language is likely German with a neutral accent, recorded in a quiet
environment with slight room reverberation.Timbre-Whisper tags
male_baritone, slightly_soft_neutral, dark_neutral, slightly_breathy,
slight_nasal_touch, near_neutral_relaxed, slight_roughness, chest_mixed,
near_neutral_heavy, mild_wear, mostly_natural, stableTimbre-Whisper prose
Pleasant middle-aged adult male voice with medium-pitched delivery, exhibiting
a slightly soft and dark timbre. There's a touch of breathiness and nasality,
coupled with a relaxed vocal production. The voice presents a slight roughness
and a chest-mixed resonance, possessing a near-neutral heavy vocal weight. Mild
wear is detectable, yet the voice remains mostly natural and stable.Voice-Tagging-Whisper tags
Suitable for Work, natural speaking, fluent, conversational style, modal voice,
neutral airflow, normal loudness, slightly dynamic, precise articulation,
natural speakingProcedural prompt
GENERAL: very understated, very relaxed, very smooth, very monotone, very dark,
adult, masculine, low-register, slightly slow, no fascination, no reflection,
no provoking lightly, genuine
SCRIPT:
(very understated, very relaxed, very smooth, no intrigue, no pensiveness,
no provoking lightly) Die größten Herausforderungen waren im letzten Jahr.Gemma-4-E4B rendered instruction (`NR_TIMBRE_SENT`)
A very dark, low-register, smooth adult male voice speaks with a very
understated and relaxed tone. Speak with a very understated, very relaxed,
very smooth delivery. "Die größten Herausforderungen waren im letzten Jahr."Parquet schemas and joining
The Whisper Parquets contain:
uid, source, mode, dur_s, record_path, record_member, audio_member, variant,
lang, record_json, bude_caption, timbre_caption, timbre_tags_json,
timbre_prose, voice_tags_caption, voice_tags_json, voice_tags_proseThe aligned Gemma Parquets contain:
uid, source, mode, variant, record_path, record_member, instruction,
raw_generation, render_json, validation_json, pass,
quote_punctuation_canonicalized, generated_tokens,
batch_gpu_seconds_per_output, model_pathMinimal zero-copy-style part join with PyArrow:
import json
import pyarrow.parquet as pq
part = 0
w = pq.read_table(f"annotations/s3/whisper/whisper.part-{part:05d}.parquet")
g = pq.read_table(f"annotations/s3/gemma/gemma.part-{part:05d}.parquet")
assert w.column("uid").to_pylist() == g.column("uid").to_pylist()
# The files are aligned, so columns can be appended without a global shuffle.
row = {name: w[name][0].as_py() for name in w.column_names}
row["instruction"] = g["instruction"][0].as_py()
record = json.loads(row["record_json"])
text = record["text"]
procedural_prompt = record["prompt"]For S5-S10, first scan the appropriate membership Parquet and group rows by annotation_part; then read only those of the 256 aligned part pairs. This keeps both metadata I/O and the number of open files bounded.
MOSS Audio Tokenizer v2: frames, codebooks and token rate
This release stores the 12-codebook operating point of OpenMOSS-Team/MOSS-Audio-Tokenizer-v2:
.moss.npyisuint16with shape `[T, 12]`.- The frame rate is 12.5 frames/s, so one row represents 80 ms.
- Every frame has 12 RVQ codebook indices. If a model flattens all indices, that is
12.5 × 12 = 150discrete indices per second. - It is therefore incorrect to describe these files as “12 tokens per second”. They contain 12 indices per 80-ms frame.
- MOSS v2 supports other operating points up to 32 quantizers, but this dataset intentionally precomputed 12. The omitted higher residual codebooks are not present and cannot be reconstructed from the
[T,12]arrays.
Recommended training uses
For instruction-conditioned TTS, use record_json.text as the immutable speech target, choose either the procedural prompt or instruction as conditioning, and predict the [T,12] MOSS target (parallel codebooks, delayed codebooks, a Talker head, or a flattened 150-index/s sequence are all possible model-side choices). Never train the model to speak the unquoted English directions in a Gemma instruction.
For reference-conditioned TTS, select rows with mode == "reference"; the REF_* variants intentionally omit speaker identity/timbre wording because the reference audio supplies it. For no-reference voice design, use NR_TIMBRE_* or NR_ARCHETYPE_SENT. For audio-language or omni-model training, the MP3 can be the audio input and any of the captions/tags/instructions can be supervised text, retrieval metadata, or auxiliary targets. Keep heldout evaluation-only, retain raw annotations for audit, and treat disagreements between captioners as QA evidence rather than silently overwriting the verified transcript.
