nifty-coder/stemsplit-backend

1# RateLimiter Implementation2 3## Overview4 5The RateLimiter class provides sophisticated rate limiting and quota management for the voice control optimization system. It implements sliding window rate limiting with multi-window tracking, provider-specific limit enforcement, and automatic provider switching capabilities.6 7## Features8 9### Multi-Window Tracking10- **Sliding Windows**: Requests per minute/hour with automatic cleanup of old entries11- **Fixed Windows**: Daily and monthly quotas with scheduled resets12- **Audio Duration Limits**: Track audio minutes across different time periods13- **Request Count Limits**: Track number of requests across different time periods14 15### Provider-Specific Configuration16- **Individual Limits**: Each provider can have different rate limits and quotas17- **Free Tier Management**: Separate tracking for free tier limits vs paid usage18- **Multiple Quota Types**: Support for requests/minute, requests/day, audio minutes/day, audio minutes/month19 20### Intelligent Quota Management21- **Proactive Checking**: Check if requests would exceed limits before consuming quota22- **Automatic Switching**: Integration with ProviderManager for automatic provider switching23- **Usage Statistics**: Detailed usage tracking and reporting24- **Cost Estimation**: Calculate estimated costs based on usage and provider pricing25 26### Exponential Backoff27- **Rate Limit Violations**: Implement exponential backoff for repeated quota violations28- **Provider-Specific**: Independent backoff state for each provider and quota type29- **Automatic Recovery**: Reset backoff state on successful requests30 31## Architecture32 33```34RateLimiter35├── Provider Configuration36│   ├── Rate Limits (requests/minute, requests/hour, requests/day)37│   ├── Free Tier Limits (audio minutes/day, audio minutes/month)38│   └── Cost Per Minute39├── Multi-Window Tracking40│   ├── Sliding Windows (cleanup old entries)41│   ├── Fixed Windows (scheduled resets)42│   └── Usage Statistics43├── Quota Management44│   ├── Quota Checking (before consumption)45│   ├── Quota Consumption (with validation)46│   └── Quota Reset (manual and automatic)47└── Backoff Management48    ├── Exponential Backoff49    ├── Provider-Specific State50    └── Automatic Recovery51```52 53## Usage54 55### Basic Setup56 57```python58from voice_control.rate_limiter import RateLimiter59from voice_control.models import ProviderConfig, ProviderType60 61# Initialize rate limiter62rate_limiter = RateLimiter()63await rate_limiter.initialize()64 65# Configure a provider66config = ProviderConfig(67    name="google_speech",68    provider_type=ProviderType.GOOGLE_SPEECH,69    enabled=True,70    priority=2,71    free_tier_limits={"audio_minutes_per_month": 60},72    rate_limits={"requests_per_minute": 1000},73    supported_formats=["webm", "wav"],74    supported_languages=["en-US"],75    cost_per_minute=0.006,76    api_credentials={"api_key": "your-key"}77)78 79await rate_limiter.configure_provider(config)80```81 82### Quota Management83 84```python85# Check if request would exceed quota86quota_status = await rate_limiter.check_quota("google_speech", 30.0)87if not quota_status.is_exceeded:88    # Consume quota for 30 seconds of audio89    success = await rate_limiter.consume_quota("google_speech", 30.0, 1)90```91 92### Usage Statistics93 94```python95# Get usage statistics for different time windows96minute_stats = await rate_limiter.get_usage_stats("google_speech", "minute")97daily_stats = await rate_limiter.get_usage_stats("google_speech", "day")98 99print(f"Requests: {minute_stats.requests_count}")100print(f"Audio minutes: {minute_stats.audio_minutes}")101print(f"Estimated cost: ${minute_stats.estimated_cost:.4f}")102```103 104### Provider Availability105 106```python107# Check if provider is available (not rate limited)108available = await rate_limiter.is_provider_available("google_speech", 60.0)109if available:110    # Provider can handle a 60-second request111    pass112```113 114## Integration with ProviderManager115 116The RateLimiter integrates seamlessly with the existing ProviderManager:117 118```python119# In ProviderManager.transcribe_audio()120if self._rate_limiter:121    audio_duration = len(audio_data) / (44100 * 2)  # Estimate duration122    quota_status = await self._rate_limiter.check_quota(provider_name, audio_duration)123    if quota_status.is_exceeded:124        continue  # Try next provider125    126    # After successful transcription127    await self._rate_limiter.consume_quota(provider_name, result.audio_duration)128```129 130## Configuration Examples131 132### Web Speech API (Free, No Limits)133```python134ProviderConfig(135    name="web_speech_api",136    provider_type=ProviderType.WEB_SPEECH_API,137    free_tier_limits={},  # No limits138    rate_limits={"requests_per_minute": 60},  # Basic rate limiting139    cost_per_minute=0.0  # Free140)141```142 143### Google Speech-to-Text (60 minutes/month free)144```python145ProviderConfig(146    name="google_speech",147    provider_type=ProviderType.GOOGLE_SPEECH,148    free_tier_limits={"audio_minutes_per_month": 60},149    rate_limits={150        "requests_per_minute": 1000,151        "requests_per_day": 1000000152    },153    cost_per_minute=0.006154)155```156 157### Azure Speech Services (5 hours/month free)158```python159ProviderConfig(160    name="azure_speech",161    provider_type=ProviderType.AZURE_SPEECH,162    free_tier_limits={"audio_minutes_per_month": 300},163    rate_limits={164        "requests_per_minute": 20,165        "requests_per_hour": 1000166    },167    cost_per_minute=0.004168)169```170 171## Error Handling172 173The RateLimiter raises specific exceptions for different scenarios:174 175- **QuotaExceededError**: When quota limits would be exceeded176- **RateLimitExceededError**: When rate limits are exceeded with backoff177 178```python179try:180    await rate_limiter.consume_quota("provider", 30.0, 1)181except QuotaExceededError as e:182    logger.warning(f"Quota exceeded: {e}")183    # Switch to different provider184except RateLimitExceededError as e:185    logger.warning(f"Rate limited: {e}")186    # Wait for retry_after seconds187```188 189## Testing190 191Comprehensive test suite covers:192 193- **Unit Tests**: Individual method functionality194- **Integration Tests**: Multi-provider scenarios195- **Edge Cases**: Concurrent access, large durations, zero values196- **Error Conditions**: Quota exceeded, rate limiting, backoff197 198Run tests with:199```bash200python -m pytest voice_control/tests/test_rate_limiter.py -v201```202 203## Performance Considerations204 205- **In-Memory Storage**: Default storage for development and testing206- **Redis Support**: Optional Redis backend for distributed rate limiting207- **Cleanup Tasks**: Background cleanup of old sliding window data208- **Async Operations**: All operations are async for non-blocking performance209 210## Future Enhancements211 212- **Redis Integration**: Distributed rate limiting across multiple instances213- **Metrics Export**: Integration with monitoring systems (Prometheus, etc.)214- **Dynamic Limits**: Runtime adjustment of rate limits based on provider status215- **Predictive Switching**: Machine learning-based provider selection