ThreadAbort/IndexTTS-Rust

1# CLAUDE.md2 3This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.4 5## Project Overview6 7IndexTTS-Rust is a high-performance Text-to-Speech engine, a complete Rust rewrite of the Python IndexTTS system. It uses ONNX Runtime for neural network inference and provides zero-shot voice cloning with emotion control.8 9## Build and Development Commands10 11```bash12# Build (always build release for performance testing)13cargo build --release14 15# Run linter (MANDATORY before commits - catches many issues)16cargo clippy -- -D warnings17 18# Run tests19cargo test20 21# Run specific test22cargo test test_name23 24# Run benchmarks (Criterion-based)25cargo bench26 27# Run specific benchmark28cargo bench --bench mel_spectrogram29cargo bench --bench inference30 31# Check compilation without building32cargo check33 34# Format code35cargo fmt36 37# Full pre-commit workflow (BUILD -> CLIPPY -> BUILD)38cargo build --release && cargo clippy -- -D warnings && cargo build --release39```40 41## CLI Usage42 43```bash44# Show help45./target/release/indextts --help46 47# Synthesize speech48./target/release/indextts synthesize \49  --text "Hello world" \50  --voice examples/voice_01.wav \51  --output output.wav52 53# Generate default config54./target/release/indextts init-config -o config.yaml55 56# Show system info57./target/release/indextts info58 59# Run built-in benchmarks60./target/release/indextts benchmark --iterations 10061```62 63## Architecture64 65The codebase follows a modular pipeline architecture where each stage processes data sequentially:66 67```68Text Input → Normalization → Tokenization → Model Inference → Vocoding → Audio Output69```70 71### Core Modules (src/)72 73- **audio/** - Audio DSP operations74  - `mel.rs` - Mel-spectrogram computation (STFT, filterbanks)75  - `io.rs` - WAV file I/O using hound76  - `dsp.rs` - Signal processing utilities77  - `resample.rs` - Audio resampling using rubato78 79- **text/** - Text processing pipeline80  - `normalizer.rs` - Text normalization (Chinese/English/mixed)81  - `tokenizer.rs` - BPE tokenization via HuggingFace tokenizers82  - `phoneme.rs` - Grapheme-to-phoneme conversion83 84- **model/** - Neural network inference85  - `session.rs` - ONNX Runtime wrapper (load-dynamic feature)86  - `gpt.rs` - GPT-based sequence generation87  - `embedding.rs` - Speaker and emotion encoders88 89- **vocoder/** - Neural vocoding90  - `bigvgan.rs` - BigVGAN waveform synthesis91  - `activations.rs` - Snake/SnakeBeta activation functions92 93- **pipeline/** - TTS orchestration94  - `synthesis.rs` - Main synthesis logic, coordinates all modules95 96- **config/** - Configuration management (YAML-based via serde)97 98- **error.rs** - Error types using thiserror99 100- **lib.rs** - Library entry point, exposes public API101 102- **main.rs** - CLI entry point using clap103 104### Key Constants (lib.rs)105 106```rust107pub const SAMPLE_RATE: u32 = 22050;  // Output audio sample rate108pub const N_MELS: usize = 80;        // Mel filterbank channels109pub const N_FFT: usize = 1024;       // FFT size110pub const HOP_LENGTH: usize = 256;   // STFT hop length111```112 113### Dependencies Pattern114 115- **Audio**: hound (WAV), rustfft/realfft (DSP), rubato (resampling), dasp (signal processing)116- **ML Inference**: ort (ONNX Runtime with load-dynamic), ndarray, safetensors117- **Text**: tokenizers (HuggingFace), jieba-rs (Chinese), regex, unicode-segmentation118- **Parallelism**: rayon (data parallelism), tokio (async)119- **CLI**: clap (derive), env_logger, indicatif120 121## Important Notes122 1231. **ONNX Runtime**: Uses `load-dynamic` feature - requires ONNX Runtime library installed on system1242. **Model Files**: ONNX models go in `models/` directory (not in git, download separately)1253. **Reference Implementation**: Python code in `indextts - REMOVING - REF ONLY/` is kept for reference only1264. **Performance**: Release builds use LTO and single codegen-unit for maximum optimization1275. **Audio Format**: All internal processing at 22050 Hz, 80-band mel spectrograms128 129## Testing Strategy130 131- Unit tests inline in modules132- Criterion benchmarks in `benches/` for performance regression testing133- Python regression tests in `tests/` for end-to-end validation134- Example audio files in `examples/` for testing voice cloning135 136## Missing Infrastructure (TODO)137 138- No `scripts/manage.sh` yet (should include build, test, clean, docker controls)139- No `context.md` yet for conversation continuity140- No integration tests with actual ONNX models141