honey126/VoxAI

1# F5-TTS REST API Setup Guide2 3Complete guide to integrate F5-TTS with your React application.4 5## Architecture Overview6 7```8┌─────────────────┐         HTTP/REST API          ┌──────────────────┐9│   React App     │ ◄──────────────────────────► │  FastAPI Server  │10│  (Frontend)     │     POST /api/tts            │   (Backend)      │11│                 │     - FormData               │                  │12│  - Upload audio │     - Reference audio        │  - F5-TTS Model  │13│  - Input text   │     - Reference text         │  - PyTorch       │14│  - Play result  │     - Generation text        │  - GPU/CPU       │15└─────────────────┘                               └──────────────────┘16```17 18## Why This Architecture?19 20✅ **Separation of Concerns**: UI logic separate from ML inference21✅ **Performance**: PyTorch runs on server GPU, not client browser22✅ **Security**: Model weights stay on server23✅ **Scalability**: Can scale backend independently24✅ **Cross-Platform**: Any client can use the API (React, mobile, etc.)25 26---27 28## Backend Setup (Python FastAPI)29 30### 1. Install Dependencies31 32First, ensure you have the F5-TTS package installed:33 34```bash35# Already done if you ran earlier36pip install -e .37 38# Install FastAPI and server39pip install fastapi uvicorn python-multipart40```41 42### 2. Start the API Server43 44```bash45# Start the server46python api_server.py47```48 49The server will start on `http://localhost:8000`50 51**Available Endpoints:**52- `GET /` - API info53- `GET /health` - Health check54- `GET /docs` - Interactive API documentation (Swagger UI)55- `POST /api/tts` - Full TTS generation with all parameters56- `POST /api/tts/quick` - Quick TTS with default parameters57 58### 3. Test the API59 60Open your browser to `http://localhost:8000/docs` to see interactive API documentation and test endpoints.61 62Or use curl:63 64```bash65curl -X POST "http://localhost:8000/api/tts/quick" \66  -F "ref_audio=@path/to/reference.wav" \67  -F "ref_text=Hello, this is my voice" \68  -F "gen_text=This is the text I want to generate" \69  -o output.wav70```71 72---73 74## Frontend Setup (React)75 76### 1. Copy the React Component77 78Copy `react-client-example.tsx` into your React project:79 80```bash81# In your React project82cp react-client-example.tsx src/components/F5TTSGenerator.tsx83```84 85### 2. Configure API URL86 87Create a `.env` file in your React project:88 89```env90REACT_APP_TTS_API_URL=http://localhost:800091```92 93For production, change to your production API URL.94 95### 3. Use the Component96 97```tsx98// In your App.tsx or any page99import { F5TTSGenerator } from './components/F5TTSGenerator';100 101function App() {102  return (103    <div className="App">104      <F5TTSGenerator />105    </div>106  );107}108```109 110### 4. Start Your React App111 112```bash113npm start114# or115yarn start116```117 118---119 120## API Reference121 122### POST /api/tts123 124Generate speech with full control over parameters.125 126**Request (multipart/form-data):**127 128| Field | Type | Required | Default | Description |129|-------|------|----------|---------|-------------|130| `ref_audio` | File | ✅ Yes | - | Reference audio file (WAV, MP3, etc.) |131| `ref_text` | String | ✅ Yes | - | Transcription of reference audio |132| `gen_text` | String | ✅ Yes | - | Text to generate in reference voice |133| `remove_silence` | Boolean | No | false | Remove silence from output |134| `target_rms` | Float | No | 0.1 | Audio normalization level |135| `cross_fade_duration` | Float | No | 0.15 | Cross-fade duration (seconds) |136| `speed` | Float | No | 1.0 | Speech speed multiplier |137| `nfe_step` | Integer | No | 32 | Quality (higher = better, slower) |138| `cfg_strength` | Float | No | 2.0 | Classifier-free guidance strength |139| `sway_sampling_coef` | Float | No | -1.0 | Sway sampling (-1 = disabled) |140| `seed` | Integer | No | -1 | Random seed (-1 = random) |141 142**Response:**143 144Returns audio file (WAV format) with headers:145- `X-Seed`: The seed used for generation146- `X-Sample-Rate`: Audio sample rate (24000 Hz)147 148**Example JavaScript/TypeScript:**149 150```typescript151const formData = new FormData();152formData.append('ref_audio', audioFile);153formData.append('ref_text', 'Hello world');154formData.append('gen_text', 'This is generated text');155formData.append('speed', '1.2');156 157const response = await fetch('http://localhost:8000/api/tts', {158  method: 'POST',159  body: formData,160});161 162const audioBlob = await response.blob();163const audioUrl = URL.createObjectURL(audioBlob);164```165 166---167 168## Production Deployment169 170### Backend (FastAPI)171 172**Option 1: Docker** (Recommended)173 174```dockerfile175# Dockerfile176FROM python:3.11-slim177 178WORKDIR /app179 180# Install dependencies181COPY requirements.txt pyproject.toml ./182RUN pip install -e .183RUN pip install fastapi uvicorn python-multipart184 185# Copy application186COPY . .187 188# Expose port189EXPOSE 8000190 191# Run server192CMD ["python", "api_server.py"]193```194 195Build and run:196 197```bash198docker build -t f5tts-api .199docker run -p 8000:8000 --gpus all f5tts-api200```201 202**Option 2: Direct Deployment**203 204```bash205# Install production server206pip install gunicorn207 208# Run with multiple workers209gunicorn api_server:app -w 4 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000210```211 212### Frontend (React)213 214Build and deploy as usual:215 216```bash217npm run build218# Deploy the build/ folder to your hosting service219```220 221Update `.env.production`:222 223```env224REACT_APP_TTS_API_URL=https://your-api-domain.com225```226 227### CORS Configuration228 229For production, update `api_server.py` line 24:230 231```python232# Change from:233allow_origins=["*"]234 235# To:236allow_origins=["https://your-react-app.com", "https://www.your-react-app.com"]237```238 239---240 241## Performance Optimization242 243### Backend244 2451. **Use GPU**: Ensure PyTorch detects your GPU246   ```python247   # In api_server.py, the model automatically uses GPU if available248   # Check with: print(tts_engine.device)249   ```250 2512. **Model Caching**: The model loads once at startup (already implemented)252 2533. **Async Processing**: For multiple requests, consider task queues (Celery/Redis)254 2554. **File Cleanup**: Temporary files auto-cleanup (already implemented)256 257### Frontend258 2591. **Streaming**: For long text, consider implementing streaming responses260 2612. **Chunking**: Split very long text into chunks on the frontend262 2633. **Caching**: Cache generated audio for repeated requests264 2654. **Loading States**: Show progress (already implemented in example)266 267---268 269## Troubleshooting270 271### Common Issues272 273**Issue: "Module not found: f5_tts"**274```bash275# Solution: Install the package276pip install -e .277```278 279**Issue: "CORS error in browser"**280```python281# Solution: Check CORS settings in api_server.py282# Ensure your React app URL is in allow_origins283```284 285**Issue: "Connection refused"**286```bash287# Solution: Ensure API server is running288python api_server.py289 290# Check it's accessible291curl http://localhost:8000/health292```293 294**Issue: "Slow generation"**295```python296# Solution: Reduce nfe_step parameter (default: 32)297# Lower values = faster but lower quality298# Try: nfe_step=16 for 2x speed299```300 301**Issue: "Out of memory"**302```python303# Solution: Use smaller batch sizes or CPU mode304# For very long text, split into chunks client-side305```306 307---308 309## Example Usage Scenarios310 311### 1. Voice Cloning Chatbot312 313```typescript314// In your chatbot component315const generateBotResponse = async (text: string) => {316  const formData = new FormData();317  formData.append('ref_audio', savedVoiceFile); // User's voice sample318  formData.append('ref_text', savedVoiceTranscript);319  formData.append('gen_text', text); // Bot's response text320 321  const response = await fetch(`${API_URL}/api/tts/quick`, {322    method: 'POST',323    body: formData,324  });325 326  return await response.blob();327};328```329 330### 2. Audiobook Generator331 332```typescript333// Split book into chapters, generate each334const generateAudiobook = async (chapters: string[]) => {335  const audioChunks = [];336 337  for (const chapter of chapters) {338    const formData = new FormData();339    formData.append('ref_audio', narratorVoice);340    formData.append('ref_text', narratorSample);341    formData.append('gen_text', chapter);342    formData.append('speed', '1.1'); // Slightly faster343 344    const blob = await fetch(`${API_URL}/api/tts`, {345      method: 'POST',346      body: formData,347    }).then(r => r.blob());348 349    audioChunks.push(blob);350  }351 352  return audioChunks;353};354```355 356### 3. Multi-Language Support357 358```typescript359// Switch between models for different languages360const generateMultilingual = async (text: string, lang: 'en' | 'zh') => {361  const refAudio = lang === 'en' ? englishVoice : chineseVoice;362  const refText = lang === 'en' ? englishSample : chineseSample;363 364  // Generate speech...365};366```367 368---369 370## Next Steps371 3721. ✅ Start the backend API server3732. ✅ Test with Swagger UI (`/docs`)3743. ✅ Integrate React component3754. ✅ Test end-to-end3765. 🚀 Deploy to production377 378## Support379 380- API Documentation: `http://localhost:8000/docs`381- Project Repository: https://github.com/SWivid/F5-TTS382- Issues: Check the GitHub repository383 384---385 386**Built with F5-TTS - Zero-shot voice cloning at its finest! 🎙️**387