Epikwhale/speecht5_finetuned_telugu_charan

SpeechT5 Fine-tuned for Telugu Text-to-Speech

This model is a fine-tuned version of microsoft/speecht5_tts adapted for Telugu text-to-speech synthesis. The model was trained on SPRINGLab's IndicTTS Telugu dataset using a custom Telugu-to-Latin transliteration layer so the English-trained SpeechT5 tokenizer could process Telugu input.

Approach

SpeechT5's tokenizer is trained on English/Latin script and does not natively handle Devanagari-family scripts. Rather than retraining the tokenizer, this model uses a transliteration shim: Telugu input text is converted to Latin-script approximations before being passed to the model. The model then learns to map those transliterated sequences to Telugu speech during fine-tuning.

This is a pragmatic approach — it sidesteps the tokenizer problem but inherits any information loss from transliteration. It's not a substitute for native script support or phoneme-aware modeling.

Intended Uses

• —Telugu text-to-speech experimentation and research
• —Educational and accessibility prototypes for Telugu content
• —Baseline for further fine-tuning on richer Telugu datasets

Limitations

• —No formal speech-quality evaluation. This model was evaluated by training/validation loss only. There is no MOS study, no objective audio metric (MCD, PESQ, etc.), and no listener test. Quality should be judged by listening to outputs.
• —Latin-transliteration intermediate step. Some Telugu phonemic distinctions may not survive transliteration cleanly.
• —Single-speaker / dataset-bound. Performance is bounded by the IndicTTS Telugu dataset's speaker and content diversity.
• —No handling of code-mixed or technical text. English words, numerals, or mixed-script input are not handled robustly.
• —Compute-bound inference. Standard SpeechT5 inference requirements apply.

Training Data

SPRINGLab IndicTTS Telugu — 8,576 Telugu audio-text pairs. Used as-is, without additional filtering or augmentation.

Training Procedure

Hyperparameters

• —Learning rate: 1e-3
• —Train batch size: 4 (effective 32 with gradient accumulation = 8)
• —Eval batch size: 2
• —Optimizer: AdamW (β₁=0.9, β₂=0.999, ε=1e-8)
• —Scheduler: Linear with 100 warmup steps
• —Total steps: 1,000
• —Mixed precision: Native AMP
• —Seed: 42

Training Results

Validation loss decreased from 0.6689 → 0.4496 over 1,000 steps:

Loss-based convergence only — see limitations.

Usage

from transformers import SpeechT5Processor, SpeechT5ForTextToSpeech
import torch

processor = SpeechT5Processor.from_pretrained("Epikwhale/speecht5_finetuned_telugu_charan")
model = SpeechT5ForTextToSpeech.from_pretrained("Epikwhale/speecht5_finetuned_telugu_charan")

# Telugu text must be transliterated to Latin script before tokenization.
# See the transliteration utility in the repo for the mapping used during training.
text = "your transliterated telugu text here"
inputs = processor(text=text, return_tensors="pt")

with torch.no_grad():
    speech = model.generate_speech(inputs["input_ids"], speaker_embeddings=None)

Speaker embeddings (e.g., SpeechBrain x-vectors) and a vocoder (e.g., HiFi-GAN) are required for the full inference pipeline.

Framework Versions

• —Transformers 4.47.0
• —PyTorch 2.5.1+cu121
• —Datasets 3.3.1
• —Tokenizers 0.21.0

Citation

@misc{speecht5_telugu_charan,
  title  = {SpeechT5 Fine-tuned for Telugu Text-to-Speech},
  author = {Rama Charan Pisupati},
  year   = {2025},
  url    = {https://huggingface.co/Epikwhale/speecht5_finetuned_telugu_charan}
}

Acknowledgments

• —Microsoft Research — SpeechT5 architecture
• —SPRINGLab, IIT Madras — IndicTTS Telugu dataset
• —Hugging Face — transformers library and hosting

Contact

• —Email: rama.charan.official@gmail.com
• —GitHub: Epik-Whale463