CoolFace
Apppublic

honey126/VoxAI

sourceHugging Facemitupdated 9mo agoView on Hugging Face
0likes
App README

🎙️ VoxAI Studio - AI Voice Cloning & Text-to-Speech

![Hugging Face Spaces](https://huggingface.co/spaces) ![License: MIT](https://opensource.org/licenses/MIT)

State-of-the-art voice cloning and text-to-speech platform powered by F5-TTS and E2-TTS models.

Clone any voice with just 3-10 seconds of audio and generate natural-sounding speech in seconds!

🚀 Features

  • —🎯 Zero-Shot Voice Cloning: Clone any voice with minimal audio samples (3-10 seconds)
  • —⚡ Fast Generation: GPU-accelerated inference for quick results
  • —🎨 Dual Interface:
  • —Modern React web app with authentication and voice management
  • —Simple Gradio interface for quick testing
  • —🔐 User Management: JWT authentication with tier-based access
  • —📚 Voice Library: Save and manage your voice clones
  • —🎛️ Advanced Controls: Fine-tune speed, pitch, and other parameters
  • —🌐 Hybrid TTS: Supports both custom cloned voices and 31 standard voices via Google Gemini

🎮 How to Use

Option 1: React Web Interface (Recommended)

  1. 1.Visit the main app at / (root URL)
  2. 2.Register/Login to create your account
  3. 3.Go to VoiceLab to upload voice samples:
  4. 4.Record or upload 3-10 seconds of clear audio
  5. 5.Provide accurate transcription
  6. 6.Save to your voice library
  7. 7.Go to Studio to generate speech:
  8. 8.Select your cloned voice or a standard voice
  9. 9.Enter the text you want to generate
  10. 10.Click generate and download your audio!

Option 2: Gradio Interface (Quick Test)

  1. 1.Visit `/gradio` for the simple interface
  2. 2.Upload reference audio (3-10 seconds)
  3. 3.Enter reference text (transcription of the audio)
  4. 4.Enter generation text (what you want to say)
  5. 5.Click Generate and listen to the result!

📋 API Endpoints

The FastAPI backend provides a full REST API:

  • —POST /api/auth/register - Create account
  • —POST /api/auth/login - Authenticate
  • —POST /api/voices - Upload voice clone
  • —GET /api/voices - List your voices
  • —POST /api/tts - Generate speech
  • —GET /docs - Full API documentation

🛠️ Technology Stack

Backend

  • —F5-TTS: Flow Matching-based TTS model for voice cloning
  • —E2-TTS: Enhanced emotional voice synthesis
  • —FastAPI: High-performance Python web framework
  • —SQLAlchemy: Database ORM with SQLite/PostgreSQL support
  • —JWT Authentication: Secure user authentication
  • —Gradio: Alternative simple UI

Frontend

  • —React 19: Modern component-based UI
  • —TypeScript: Type-safe development
  • —Vite: Lightning-fast build tool
  • —Google Gemini API: Standard voice TTS

AI/ML

  • —PyTorch: Deep learning framework
  • —Vocos/BigVGAN: Neural vocoders
  • —Conditional Flow Matching: Generative modeling
  • —Transformer Architecture: DiT, MMDiT, UNetT backbones

💡 Usage Tips

For Best Results:

  1. 1.Reference Audio Quality:
  2. 2.Use clear, noise-free audio
  3. 3.3-10 seconds of speech
  4. 4.Single speaker
  5. 5.Good microphone quality
  1. 1.Reference Transcription:
  2. 2.Provide exact transcription
  3. 3.Include punctuation
  4. 4.Match the audio precisely
  1. 1.Generation Text:
  2. 2.Keep it reasonable length (< 500 chars recommended)
  3. 3.Use proper punctuation for natural prosody
  4. 4.Similar style to reference works best
  1. 1.Speed Settings:
  2. 2.Start with 1.0 (normal speed)
  3. 3.Adjust between 0.5-2.0 for effect
  4. 4.Slower speeds can improve quality

🎯 User Tiers

TierVoice SlotsRate LimitCharacter Limit
Free1 voice10 req/hour5,000 chars
Pro3 voices100 req/hour20,000 chars
EnterpriseUnlimited1,000 req/hour20,000 chars

🔧 Environment Variables

For deployment, configure these variables:

bash
# Server
PORT=7860
HOST=0.0.0.0
ENVIRONMENT=production

# Database
DATABASE_URL=sqlite:///./f5tts.db

# Authentication
SECRET_KEY=your-secret-key-here
ACCESS_TOKEN_EXPIRE_MINUTES=15
REFRESH_TOKEN_EXPIRE_DAYS=7

# CORS
CORS_ORIGINS=https://your-space.hf.space

# Optional: Google Gemini API
GEMINI_API_KEY=your-gemini-api-key

📦 Local Development

Prerequisites

  • —Python 3.10+
  • —Node.js 18+
  • —FFmpeg
  • —Git

Setup

  1. 1.Clone the repository
bash
git clone <your-repo-url>
cd E2-F5-TTS
  1. 1.Install Python dependencies
bash
pip install -r requirements.txt
  1. 1.Install frontend dependencies
bash
cd voxai-studio
npm install
npm run build
cd ..
  1. 1.Run the application
bash
python hf_app.py
  1. 1.Access the app
  2. 2.Main app: http://localhost:7860
  3. 3.Gradio: http://localhost:7860/gradio
  4. 4.API docs: http://localhost:7860/docs

🐳 Docker Deployment

Build and run with Docker:

bash
# Build the image
docker build -t voxai-studio .

# Run the container
docker run -p 7860:7860 voxai-studio

For Hugging Face Spaces:

Simply push this repository to your HF Space. The Dockerfile will handle everything automatically!

📚 Model Information

This application uses the F5-TTS (Flow Matching-based TTS) and E2-TTS models:

Model Architecture:

  • —Conditional Flow Matching (CFM) for audio generation
  • —Transformer backbones (DiT, MMDiT, UNetT)
  • —Mel spectrogram intermediate representation
  • —Vocos/BigVGAN vocoder for high-quality audio

⚖️ License

MIT License - see LICENSE file for details

🙏 Acknowledgments

  • —F5-TTS Team for the amazing voice cloning models
  • —Hugging Face for the Spaces platform
  • —FastAPI and React communities
  • —All contributors to this project

🐛 Issues & Support

For issues, questions, or feature requests, please visit:

🌟 Star History

If you find this project useful, please consider giving it a star! ⭐

Made with ❤️ by the VoxAI Studio Team

Powered by F5-TTS, FastAPI, React, and Gradio