CoolFace
Apppublic

iteratehack/voice-model-rl-training

sourceHugging Facemitupdated 10mo agoView on Hugging Face
0likes
App README

Voice Model RL Training

Train open-source voice models using Reinforcement Learning with PPO and REINFORCE algorithms.

Features

  • —🎯 Multiple RL Algorithms: Choose between PPO and REINFORCE
  • —🚀 GPU Acceleration: Automatic GPU detection and usage
  • —📊 Real-time Monitoring: Track training progress in real-time
  • —🎵 Model Comparison: Compare base vs trained models
  • —💾 Checkpoint Management: Automatic model saving and loading
  • —🎤 Multiple Base Models: Support for Wav2Vec2, WavLM, and more

Supported Models

  • —Facebook Wav2Vec2 (Base & Large)
  • —Microsoft WavLM Base Plus
  • —Any compatible HuggingFace speech model

How to Use

1. Training Tab

  1. 1.Select Base Model: Choose from available pretrained models
  2. 2.Configure Algorithm: Select PPO (recommended) or REINFORCE
  3. 3.Set Parameters:
  4. 4.Episodes: 10-100 (start with 20 for testing)
  5. 5.Learning Rate: 1e-5 to 1e-3 (default: 3e-4)
  6. 6.Batch Size: 4-64 (depends on GPU memory)
  7. 7.Start Training: Click "Start Training" and monitor progress

2. Compare Results Tab

  1. 1.Upload Audio: Provide a test audio sample
  2. 2.Generate Comparison: Process through both models
  3. 3.Listen: Compare base vs trained model outputs

Reward Functions

The training optimizes for three key metrics:

  • —Clarity (33%): Audio signal quality and noise reduction
  • —Naturalness (33%): Natural speech patterns and prosody
  • —Accuracy (34%): Fidelity to original content

Hardware Requirements

  • —CPU: Works but slow (5-10 min per episode)
  • —GPU: Recommended (T4 or better) (1-2 min per episode)
  • —Memory: 8GB+ RAM, 4GB+ VRAM

Technical Details

RL Algorithms

PPO (Proximal Policy Optimization)

  • —More stable training
  • —Uses value function
  • —Better for most cases
  • —Slightly slower per episode

REINFORCE

  • —Simpler algorithm
  • —Higher variance
  • —Faster per episode
  • —May need more episodes

Training Process

  1. 1.Load pretrained base model
  2. 2.Add RL policy/value heads
  3. 3.Train using custom reward function
  4. 4.Save checkpoints periodically
  5. 5.Generate comparisons

Local Development

Clone and run locally:

bash
git clone https://huggingface.co/spaces/USERNAME/voice-model-rl-training
cd voice-model-rl-training
pip install -r requirements.txt
python app.py

Repository Structure

voice-rl-training/
├── app.py                 # Main Gradio application
├── requirements.txt       # Python dependencies
├── README.md             # This file
├── voice_rl/             # Core training modules
│   ├── models/           # Model wrappers
│   ├── rl/               # RL algorithms
│   ├── training/         # Training orchestration
│   ├── data/             # Data handling
│   ├── monitoring/       # Metrics and visualization
│   └── evaluation/       # Model evaluation
└── workspace/            # Training outputs (git-ignored)