Modelpublic

sulabhkatiyar/indian-ne-multilingual-tts

sourceHugging Facecc-by-4.0updated 12d agoView on Hugging Face

3likes69downloads

Model Card

NE-TTS: 16 North-East India languages

A single multi-speaker, multilingual text-to-speech model covering 16 low-resource North-East India languages in 23 speaker voices, trained from scratch on romanized text.

This is a first version, trained on the data currently available for these languages. The audio samples below are what it sounds like today.

Model description

—Architecture: VITS (end-to-end). The model goes straight from text to waveform, so there is no separate vocoder stage to load.
—Training: from scratch. There is no pretrained base model anywhere in its lineage.
—Text front-end: character-level, use_phonemes=false. Input is lowercase romanized Latin text over a 34-character vocabulary ( abcdefghijklmnopqrstuvwxyz·âêîûüṭ) with no punctuation. Characters outside that set are dropped.
—Output: mono waveform at 22 050 Hz.
—Coverage: 16 languages, 23 speaker voices, selected per utterance by name.
—Release checkpoint: own step 62000 (cumulative step 208757). The file holds 86.5 M parameters, of which 39.8 M are the generator that runs at inference. The other 46.7 M are the training discriminator, which is kept in the file and is why the download is 346 MB.

Speakers

Seven languages ship two voices and nine ship one, for 23 in total. Pass the voice name as speaker_name and the ISO code as language_name.

Language	ISO	Voices
Tagin	`tgj`	`tgj_female_papumpare`
Nyishi	`njz`	`njz_male_papumpare`
Angami Naga	`njm`	`njm_male_kohima`
Rongmei Naga	`nbu`	`nbu_female_kohima`
Chokri Naga	`nri`	`nri_female_kohima`
Ao Naga	`njo`	`njo_female_kohima`
S. Rengma Naga	`nre`	`nre_female_kohima`, `nre_male_kohima`
Karbi	`mjw`	`mjw_female_karbianglong`, `mjw_male_karbianglong`
Idu Mishmi	`clk`	`clk_female_lowerdibangvalley`
Kok Borok	`trp`	`trp_male_westtripura`
Wancho	`nnp`	`nnp_female_longding`, `nnp_male_longding`
Mizo	`lus`	`lus_female_aizawl`, `lus_male_aizawl`
Nagamese	`nag`	`nag_female_dimapur`, `nag_male_kohima`
Garo	`grt`	`grt_female_westgarohills`, `grt_male_southgarohills`
Chakma	`ccp`	`ccp_female_dhalai`, `ccp_female_unakoti`
Sumi	`nsm`	`nsm_female_kohima`

Each voice is a gender × district speaker pool from the source corpus, not one individual. The recordings behind a single voice come from several speakers of the same gender in the same district, so a voice can drift in accent within a language.

Datasets

—Training data: `sulabhkatiyar/ne-tts-coqui-multilingual`

This checkpoint was trained from scratch on the v1 snapshot of that corpus, built from the Vaani recordings as follows. Clips were filtered to an estimated signal-to-noise ratio of at least 20 dB, and 23 of the 63 available speaker groups were kept, one or two per language, ranked by how many clips each group had left after that filter. Clips longer than 10 seconds were then dropped, which removed 454 of 41,425 clips and left about 41,000 clips and 55 hours of audio.

Data source & attribution. The training and evaluation data is derived from the ARTPARK-IISc Vaani project (https://vaani.iisc.ac.in/), released under CC-BY-4.0. Please retain that attribution when you use these models.

How to load

Part of the NE Speech AI V1 collection.

This repo contains the full VITS weights. No external base model is required. Everything needed for inference is here: model.pth (346 MB), config.json, speakers.pth, language_ids.json and tts_release_meta.json.

Install, on Python 3.12:

bash

pip install "coqui-tts[cpu]==0.27.5" "torch==2.8.0" "torchaudio==2.8.0" "transformers<5" "numpy<2" huggingface_hub

The versions are pinned because the install breaks without them.

python

import os
from huggingface_hub import snapshot_download
from TTS.utils.synthesizer import Synthesizer

model_dir = snapshot_download(
    repo_id="sulabhkatiyar/indian-ne-multilingual-tts",
    local_dir="indian-ne-multilingual-tts",
    allow_patterns=["model.pth", "config.json", "speakers.pth",
                    "language_ids.json", "tts_release_meta.json"],
)

# config.json resolves speakers.pth and language_ids.json relative to the cwd
os.chdir(model_dir)

synth = Synthesizer(
    tts_checkpoint="model.pth",
    tts_config_path="config.json",
    tts_speakers_file="speakers.pth",
    tts_languages_file="language_ids.json",
    use_cuda=False,          # set True if you have a CUDA GPU
)

wav = synth.tts(
    "ia noksao anga dal·begipa rikgimin nokko nikenga",   # lowercase romanized text
    speaker_name="grt_female_westgarohills",              # see the Speakers table
    language_name="grt",
)
synth.save_wav(wav, "out.wav")   # 22 050 Hz mono

tts_release_meta.json carries the speaker and language maps, the sample rate and the vocabulary fingerprint, so you can check a voice or language name without loading the model.

Audio samples

Two sentences per voice, 46 clips in total, at the model's native 22 050 Hz. Every sentence is held-out text that the model was not trained on. Where a language has two voices, both speak the same two sentences so they can be compared directly.

Generated with the inference settings baked into config.json (use_sdp=true, inference_noise_scale=0.667, inference_noise_scale_dp=1.0, length_scale=1.0) and seed 1234. If the players below do not render in your client, the files are in the `samples/` folder.

Tagin (tgj)

`tgj_female_papumpare`

ara aoh achin daku heh doku daku dopeh

hoke rail shi rail station shi bag anyi go dopeh

Nyishi (njz)

`njz_male_papumpare`

building agu pute jabu kongpo pa

ho building pugbo chao hoko bv sire gvh yalle nago bv chocolate colour nah sire yallo leparada pungnv yarcha pah

Angami Naga (njm)

`njm_male_kohima`

ro le di kibu nu phichümia kenie rei ngu ba

kibu kezhau kheki pfhenhie kekra ki kecü yo puo rei chü pie keba ngu ba

Rongmei Naga (nbu)

`nbu_female_kohima`

tüng gari number ta up charuk panguh b b chanei kathumh chakiu kathumh weh

mesi ne bam na lao kapiru deferao ne tao na wall karu neiw te

Chokri Naga (nri)

`nri_female_kohima`

fencing lü post ko shi cement za post shizü sü ngo

lü thünomine thünomi miküpüh kho stage lü no dance shiva ngova

Ao Naga (njo)

`njo_female_kohima`

aserbo bike anasaji long aika yua aliba ngutar

iba jaka ya central reserve police force indang jaka amai ka ngudar

S. Rengma Naga (nre)

`nre_female_kohima`

ro blackboard katsü tha nden hon tsü price den zen binyon

ro pechenyu gu shyeka mineral water bottle nme le binyon

`nre_male_kohima`

ro blackboard katsü tha nden hon tsü price den zen binyon

ro pechenyu gu shyeka mineral water bottle nme le binyon

Karbi (mjw)

`mjw_female_karbianglong`

so atm arlong monit bangkethom do

lake isi adukan ladak inut amonit arjap lun do

`mjw_male_karbianglong`

so atm arlong monit bangkethom do

lake isi adukan ladak inut amonit arjap lun do

Idu Mishmi (clk)

`clk_female_lowerdibangvalley`

ah ho done acha ma mecha

brange anga ameyi gane jigaha

Kok Borok (trp)

`trp_male_westtripura`

ani bwskango kaisa mampli tongo aw mampli o nugjago nini twi reejak obo ni pore ke nugjago aw nini oro kwbangma murti haikei taskai rok

aw jaga o ang nugmanka omo unka kaisa tok rok aw jaaga tok tongo aw jaaga

Wancho (nnp)

`nnp_female_longding`

khanak chu le tai teya saiken kho ma ngo lah le

ee kauohang kauphang du jaji saman taman li chanley ley taga

`nnp_male_longding`

khanak chu le tai teya saiken kho ma ngo lah le

ee kauohang kauphang du jaji saman taman li chanley ley taga

Mizo (lus)

`lus_female_aizawl`

mi pakhat ka hmu a kawr gray a ha a a hnuai ah kamis pawl a ha a

he thlalakah chuanin tawlhpahrit eng a awm a chu chu tualzawlah a a awm a a sir vel

`lus_male_aizawl`

mi pakhat ka hmu a kawr gray a ha a a hnuai ah kamis pawl a ha a

he thlalakah chuanin tawlhpahrit eng a awm a chu chu tualzawlah a a awm a a sir vel

Nagamese (nag)

`nag_female_dimapur`

aro rot bhi ase aro eta khan bhi jaga jaga teh rakhina ase

kan laga bahar teh etu mat rakha thaka tu dikhi ase

`nag_male_kohima`

aro rot bhi ase aro eta khan bhi jaga jaga teh rakhina ase

kan laga bahar teh etu mat rakha thaka tu dikhi ase

Garo (grt)

`grt_female_westgarohills`

dingtang dingtang rongchi gadangni gadango rikgimin nokrangkoba nikata

ia noksao anga dal·begipa rikgimin nokko nikenga

`grt_male_southgarohills`

dingtang dingtang rongchi gadangni gadango rikgimin nokrangkoba nikata

ia noksao anga dal·begipa rikgimin nokko nikenga

Chakma (ccp)

`ccp_female_dhalai`

iskulano pijedi sigun robbar gaj obak obode parang degongor

tay mui ei photobot degongottey olodey ekkan room goro bidirey room ekkan tay rummot

`ccp_female_unakoti`

iskulano pijedi sigun robbar gaj obak obode parang degongor

tay mui ei photobot degongottey olodey ekkan room goro bidirey room ekkan tay rummot

Sumi (nsm)

`nsm_female_kohima`

hilehi room lakhi ithuluani red and white lono

kitimi hipou yae alchi gha pusii egho eki ane

Limitations

—Quality varies substantially across the 16 languages and tracks how much data each had. The languages with only a few hundred clips are noticeably weaker.
—Each voice is a gender × district speaker pool rather than one individual, so a voice can drift in accent within a language.
—Input is lowercase romanized Latin text over a 34-character vocabulary with no punctuation. Characters outside that set are dropped.
—No formal listening evaluation (MOS or similar) was carried out.
—CC-BY-4.0 covers copyright, not voice or personality rights. The Vaani speakers' consent form text is not published, so its exact scope is unverified. Consider this before using these voices to represent a person or organisation.

Citation

If you use this model, please cite the VITS paper:

bibtex

@inproceedings{kim2021conditional,
  title        = {Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech},
  author       = {Kim, Jaehyeon and Kong, Jungil and Son, Juhee},
  booktitle    = {Proceedings of the 38th International Conference on Machine Learning},
  pages        = {5530--5540},
  year         = {2021},
  eprint       = {2106.06103},
  archivePrefix= {arXiv},
  primaryClass = {cs.SD}
}

The Vaani corpus the model is trained on:

bibtex

@misc{pulikodan2026vaanicapturinglanguagelandscape,
      title={VAANI: Capturing the language landscape for an inclusive digital India},
      author={Sujith Pulikodan and Abhayjeet Singh and Agneedh Basu and Nihar Desai and Pavan Kumar J and Pranav D Bhat and Raghu Dharmaraju and Ritika Gupta and Sathvik Udupa and Saurabh Kumar and Sumit Sharma and Vaibhav Vishwakarma and Visruth Sanka and Dinesh Tewari and Harsh Dhand and Amrita Kamat and Sukhwinder Singh and Shikhar Vashishth and Partha Talukdar and Raj Acharya and Prasanta Kumar Ghosh},
      year={2026},
      eprint={2603.28714},
      archivePrefix={arXiv},
      primaryClass={eess.AS},
      url={https://arxiv.org/abs/2603.28714},
}

And the NE-TTS release (placeholder; replace when the canonical publication is available):

bibtex

@misc{katiyar2026netts,
  author       = {Katiyar, Sulabh},
  title        = {NE-TTS: Multilingual VITS Text-to-Speech for North-East India Languages},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/sulabhkatiyar/indian-ne-multilingual-tts}},
  note         = {Placeholder citation; replace with the canonical publication when available.}
}

License

CC-BY-4.0. The license chain has a single hop: the model is trained from scratch, with no pretrained ancestor, so the only upstream license is the one on training data.

—Training data: ARTPARK-IISc Vaani, released under CC-BY-4.0.
—Model weights: CC-BY-4.0.
—The training framework, coqui-tts, is MPL-2.0. That is a file-level license on the framework's own source and does not extend to the weights produced by running it.