anak10thn/jalak-vits-multispeaker-slr44

Jalak VITS — Sundanese/Javanese/Indonesian multi-speaker TTS (fine-tuned)

A Coqui VITS multi-speaker model fine-tuned from the jalak-model pretrained checkpoint (83 speakers) + 41 new Sundanese speakers from OpenSLR SLR44, for a total of 124 speakers. Input is IPA phonemes (espeak-ng id voice).

Quality (honestly reported)

Eval mel-reconstruction loss on a held-out 525-clip split (original speakers only, never trained on), measured by an independent evaluator (eval_model.py):

Adding SLR44 data was neutral on the original-speaker mel metric (21.19 ≈ 21.19, within the ~0.15 run-to-run noise floor) but adds 41 new Sundanese speakers the pretrained model could not synthesize at all (no speaker embeddings). Both original and new speakers produce valid, audible speech (verified via generation sanity checks).

Caveat — new speakers are less verified: the 41 SLR44 speaker embeddings were trained for a single (diluted) epoch; their perceptual quality is not rigorously held-out-evaluated (only confirmed non-silent/non-garbage). The original 83 speakers are well-trained and held-out-verified. If you need only the metric-best on original speakers, the 83-spk variant is equivalent.

Training recipe

• —Fine-tuned fresh from the pretrained checkpoint (pre-extended emb_g 83→124 so all layers restore; the 41 new embeddings start from small random and train).
• —1 epoch over combined data (~16.9k clips: 12.6k original + 4.2k SLR44), batch 24.
• —lr 5e-5, cosine decay 5e-5 → 1e-5 over the epoch (per-step), fp32.
• —freeze_encoder=True (text encoder frozen; acoustic/decoder adapt).
• —No clip-length cap, no speaker-weighted sampler, no grad clip.
• —TTS 0.22.0, torch cu126, Tesla T4.

How to use

Requires espeak-ng installed (system), Coqui TTS, and this repo's make_config.py:

# plain text -> IPA -> audio (original speaker)
python inference.py --ckpt model.pth --text "halo selamat datang" \
    --speaker SU-00060 --out out.wav

# a new SLR44 Sundanese speaker
python inference.py --ckpt model.pth --text "panonton ting haruleng ningali tipi" \
    --speaker SLR44_suf_00297 --out slr.wav

# list all 124 speakers
python inference.py --ckpt model.pth --list-speakers

inference.py auto-detects the 124-speaker checkpoint and loads speakers.pth.

Files

• —model.pth — VITS checkpoint (124 speakers, step ~1260704)
• —config.json — derived training/inference config
• —speakers.pth — speaker name → id map (124)
• —inference.py — generation script

License

CC BY-SA 4.0 (inherited from OpenSLR SLR44; the base jalak pretrained model and the original dataset licenses also apply — verify before commercial use).

Reproducibility

Full pipeline in the source repo: preprocess_slr44.py (data), make_config.py + run_train.py (training), eval_model.py (evaluation), inference.py (inference).