Modelpublic

KickItLikeShika/NileTTS-XTTS

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

4likes63downloads

Model Card

Nile-XTTS Model 🇪🇬

Paper: https://arxiv.org/abs/2602.15675

Nile-XTTS is a fine-tuned version of XTTS v2 optimized for Egyptian Arabic (اللهجة المصرية) text-to-speech synthesis with zero-shot voice cloning capabilities.

Model Description

This model was fine-tuned on the NileTTS dataset, comprising 38 hours of Egyptian Arabic speech across medical, sales, and general conversation domains.

Key Features

—Egyptian Arabic optimized: Trained specifically on Egyptian dialect, not MSA or Gulf Arabic
—Zero-shot voice cloning: Clone any voice with just a 6-second reference audio
—Improved intelligibility: 29.9% reduction in WER compared to base XTTS v2
—Better pronunciation: 49.4% reduction in CER for Egyptian Arabic

Performance

Metric	XTTS v2 (Baseline)	Nile-XTTS-v2 (Ours)	Improvement
WER	26.8%	18.8%	29.9%
CER	8.1%	4.1%	49.4%
Speaker Similarity	0.713	0.755	+5.9%

Usage

**Interactive Demo**

Installation

bash

pip install TTS

Usage (Direct Model Loading)

python

import torch
import torchaudio
from TTS.tts.configs.xtts_config import XttsConfig
from TTS.tts.models.xtts import Xtts

# load config and model
config = XttsConfig()
config.load_json("config.json")

model = Xtts.init_from_config(config)
model.load_checkpoint(
    config,
    checkpoint_path="model.pth",
    vocab_path="vocab.json",
    use_deepspeed=False
)
model.cuda()
model.eval()

# get speaker latents from reference audio
gpt_cond_latent, speaker_embedding = model.get_conditioning_latents(
    audio_path="reference.wav",
    gpt_cond_len=6,
    max_ref_length=30,
    sound_norm_refs=False
)

# synth speech
out = model.inference(
    text="مرحبا، إزيك النهارده؟",
    language="ar",
    gpt_cond_latent=gpt_cond_latent,
    speaker_embedding=speaker_embedding,
    temperature=0.7,
)

# save output
torchaudio.save("output.wav", torch.tensor(out["wav"]).unsqueeze(0), 24000)

Training Details

—Base model: XTTS v2
—Training data: NileTTS dataset (38 hours, 2 speakers)
—Epochs: 8 (early stopping)
—Learning rate: 5e-6

Limitations

—Limited to 2 speaker voices in training data
—Optimized for Egyptian Arabic; may not perform as well on other Arabic dialects
—Zero-shot cloning quality depends on reference audio quality

Citation

If you use this model, please cite:

@inproceedings{khamis-ahmed-2026-llm,
    title = "{LLM}-to-Speech: A Synthetic Data Pipeline for Training Dialectal Text-to-Speech Models",
    author = "Khamis, Ahmed  and
      Ahmed, Hesham Ali",
    booktitle = "Proceedings of the 2nd Workshop on {NLP} for Languages Using {A}rabic Script",
    month = mar,
    year = "2026",
    address = "Rabat, Morocco",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.abjadnlp-1.6/",
    pages = "47--54"
}

License

This model is released under the Apache 2.0 license, following the original XTTS v2 license.

Acknowledgements

—Coqui TTS for the XTTS v2 base model
—The NileTTS team for the dataset creation