VoiceHub/dacvae-tts-tr-w512-stage2-hq

DACVAE-TTS Turkish run C stage 2 (width 512, high-quality subset)

Generated audio of every evaluated checkpoint of the training run tr-w512-stage2-hq (Turkish zero-shot voice-cloning TTS, dacvae-tts, frozen Meta DACVAE latents, 48 kHz). This repository holds model outputs and metrics, not training data. Training data: Vyvo/tr-dataset-12 (Turkish podcast segments).

Warm start from VoiceHub/dacvae-tts-tr-w512-clean (step 60k, 66.5M parameters), 10k more updates on the high-quality subset (Whisper CER <= 0.05, DNSMOS OVRL >= 3.0, quality_score >= 70; 21k rows) at LR 2e-4, batch expansion 4, frame budget 6000, one RTX 4090.

How to listen

• —monitor/prompt-<uid>.wav: the reference voice given to the model (a real validation recording of an unseen speaker, decoded through the DACVAE codec, so it also shows the codec's own quality ceiling).
• —monitor/<step-folder>/NNN.wav: the model's synthesis of case NNN — the text of another recording of the same speaker, in the prompt's voice. monitor/<step-folder>/results.jsonl lists per case: text (target transcript), prompt_uid (which prompt WAV), hypothesis (Whisper-large-v3 transcript of the synthesis), wer, cer, speaker_similarity, duration_ratio. NNN.json holds the sampler settings and timings.
• —Folder name suffix -gG-uU-nN-sS-dD-kK: guidance G, guidance applied while t < U, initial-noise scale N, sway S, duration scale D, Euler steps K (no suffix = guidance 2, 16 steps).
• —monitor/cases.json: the 48 (prompt, target) cases; monitor.jsonl: one summary row per evaluated checkpoint/setting; train.jsonl: training/validation curves; config.json: the full training configuration.

WER/CER: faster-whisper large-v3 (Turkish), Turkish text normalization (numbers spelled out, İ/ı-aware lower-casing, punctuation removed). SIM: microsoft/wavlm-base-plus-sv cosine between synthesis and the codec-decoded prompt. Corpus WER/CER over 48 cases of 10 held-out speakers; the ASR floor on real codec-decoded speech is WER ≈ 0.05 / CER ≈ 0.016.

Results

Configuration

{
  "latent_dim": 128,
  "width": 512,
  "depth": 12,
  "heads": 8,
  "text_depth": 4,
  "patch_size": 1,
  "ff_mult": 3,
  "cond_dropout": 0.2,
  "reference_encoder": "mlp",
  "reference_pooling": "mean",
  "reference_paths": "both",
  "duration_features": "baseline",
  "positions": "rope",
  "qk_norm": true,
  "text_attention": 4,
  "prediction": "edm",
  "text_layout": "joined",
  "duration": "rule",
  "ctc_layer": 8,
  "adaln_rank": 64
}

{
  "steps": 10000,
  "batch_size": 96,
  "accumulation": 1,
  "learning_rate": 0.0002,
  "warmup": 300,
  "weight_decay": 0.01,
  "optimizer": "muon",
  "muon_momentum": 0.95,
  "ema_decay": 0.9999,
  "precision": "bf16",
  "workers": 4,
  "worker_threads": 1,
  "prefetch_factor": 4,
  "loader_start_method": "spawn",
  "cuda_prefetch": true,
  "checkpoint_every": 1000,
  "validate_every": 1000,
  "log_every": 100,
  "grad_checkpoint": true,
  "compile": false,
  "seed": 42,
  "flow_reduction": "frame",
  "duration_weight": 0.1,
  "speaker_balance": 0.0,
  "diagnostics_every": 0,
  "pairing": "within",
  "prompt_fraction_min": 0.1,
  "prompt_fraction_max": 0.6,
  "prompt_dropout": 0.3,
  "time_sampling": "logit_normal",
  "batch_expansion": 4,
  "keep_every": 2500,
  "ctc_weight": 0.1,
  "contrastive_weight": 0.2,
  "contrastive_margin": 0.1,
  "wandb_project": "dacvae-tts-tr"
}