CoolFace
Apppublic

alakxender/tts-dhivehi-demo-mms

sourceHugging Faceupdated 1y agoView on Hugging Face
4likes
App README

Fine-tuned Text-to-Speech Model for Divehi

Overview

This project involves fine-tuning a Text-to-Speech (TTS) model specifically for the Divehi language using HuggingFace's powerful tools and pre-trained models, including the Massively Multilingual Speech (MMS) and VITS frameworks. Divehi, also known as Maldivian, is an Indo-Aryan language spoken in the Maldives. The aim of this fine-tuning process is to improve the TTS system's ability to generate natural and accurate Divehi speech from text inputs.

Model Description

Base Models:

  • —Massively Multilingual Speech (MMS): A pre-trained model from Facebook AI Research designed to handle TTS tasks across multiple languages. MMS provides a robust foundation with extensive language support and pre-learned phonetic nuances.
  • —VITS (Variational Inference and Text-to-Speech): A state-of-the-art TTS model that integrates variational inference and generative adversarial networks (GANs) to produce high-quality, natural-sounding speech.

Fine-tuning Process:

  1. 1.Data Collection and Preparation:
  2. 2.Text Corpus: Compilation of a large and diverse Divehi text corpus to capture the language's phonetic and syntactic properties.
  3. 3.Audio Samples: Collection of high-quality Divehi audio recordings, including:
  4. 4.Common Voice Dataset: Leveraging Mozilla's Common Voice dataset, which includes a substantial number of Divehi audio samples.
  5. 5.Synthesized Data: Utilizing over 16 hours of synthesized Divehi speech data to augment the training set.
  6. 6.Alignment: Ensuring the text-audio pairs are accurately aligned for effective training.
  1. 1.Training Setup:
  2. 2.HuggingFace's Transformers Library: Utilizing HuggingFace’s easy-to-use interface for loading pre-trained models and managing the fine-tuning process.
  3. 3.Tokenization: Employing a tokenizer suitable for Divehi to convert text into token sequences that the model can process.
  4. 4.Model Configuration: Adjusting model parameters to optimize performance for Divehi, including learning rates, batch sizes, and epochs.
  1. 1.Fine-tuning:
  2. 2.VITS Fine-tuning: Leveraging the VITS model’s architecture to fine-tune on Divehi-specific text-audio pairs, focusing on improving the model’s ability to generate Divehi phonetics accurately.
  3. 3.MMS Fine-tuning: Further fine-tuning the MMS model to enhance its multilingual capabilities with a specific focus on Divehi.
  1. 1.Evaluation and Testing:
  2. 2.Quality Assessment: Using objective metrics like Mean Opinion Score (MOS) and subjective listening tests to evaluate the naturalness and accuracy of the generated Divehi speech.
  3. 3.Error Analysis: Identifying and rectifying common errors such as mispronunciations, intonation issues, and unnatural pacing.
Benefits
  • —High-Quality Speech Synthesis: Produces natural and intelligible Divehi speech, suitable for applications in virtual assistants, audiobooks, and accessibility tools.
  • —Cultural Preservation: Supports the digital presence and preservation of the Divehi language through advanced speech technology.
  • —Customizability: Fine-tuning allows for further adjustments and improvements based on specific use cases and user feedback.
Conclusion

The fine-tuned Divehi Text-to-Speech model represents a significant advancement in the accessibility and usability of speech technology for the Divehi-speaking community. By combining the strengths of the MMS and VITS models with the flexibility of HuggingFace's tools, and leveraging a rich dataset including the Common Voice dataset and synthesized data, this project delivers a high-quality, linguistically accurate TTS solution tailored to the unique characteristics of the Divehi language.