Sabithulla/lightweight-ai-backend

Lightweight Multi-Model AI Backend for Hugging Face Spaces

Production-Ready for FREE CPU Tier

🚀 Quick Start

This is a complete, production-ready Hugging Face Gradio Space optimized for the FREE CPU tier. It requires NO GPU and includes four AI capabilities:

✅ General chat (powered by TinyLlama) ✅ Code generation (powered by TinyLlama) ✅ Text summarization (powered by FLAN-T5-Small) ✅ Text-to-image generation (lightweight procedural)

📦 Features

Optimization for CPU Tier

• —Lazy Loading: Models loaded only when needed
• —Memory Efficient: ~1.5-2GB total RAM usage
• —Fast Responses: Token limits ensure <10s per request
• —Queue System: Handles concurrent requests safely
• —Float32 Precision: Optimized for CPU computation

Model Selection

API Endpoints

All endpoints are exposed through Gradio and accessible programmatically:

1. /generate_chat - General Chat

{
    "prompt": "Hello, how are you?",
    "max_tokens": 150,
    "temperature": 0.7
}
# Returns: Generated chat response

2. /generate_code - Code Generation

{
    "prompt": "Write a function to reverse a string",
    "max_tokens": 256,
    "temperature": 0.3
}
# Returns: Generated Python code

3. /summarize_text - Text Summarization

{
    "text": "Long article text here...",
    "max_length": 100
}
# Returns: Summarized text

4. /generate_image - Text-to-Image

{
    "prompt": "A red sunset over mountains",
    "width": 256,
    "height": 256
}
# Returns: Generated PIL Image

🔧 Configuration & Tuning

Model Parameters

Chat Generation:

• —max_tokens: 50-200 (default: 150)
• —temperature: 0.1-1.0 (default: 0.7)
• —top_p: Fixed at 0.9 for quality

Code Generation:

• —max_tokens: 100-300 (default: 256)
• —temperature: 0.1-1.0 (default: 0.3 - lower for deterministic code)

Summarization:

• —max_length: 20-150 (default: 100)
• —min_length: Fixed at 20

Image Generation:

• —width: 128-256 (default: 256)
• —height: 128-256 (default: 256)

Memory Optimization

The application includes several memory-saving techniques:

# 1. Lazy Loading
# Models only loaded when first called
model_manager.load_chat_model()  # Called only if needed

# 2. Garbage Collection
gc.collect()  # Called after each inference

# 3. CPU Optimization
torch.set_num_threads(4)  # Limits threading overhead

# 4. Token Limits
max_tokens = min(max_tokens, 200)  # Hard caps for stability

# 5. Input Truncation
if len(text) > 1000:
    text = text[:1000]  # Prevent OOM on summarization

📊 Performance Benchmarks (Rough Estimates)

On a 2-core CPU with 4GB RAM:

Total idle memory: ~1.5GB Max concurrent memory: ~2GB

🚀 Deployment to Hugging Face Spaces

Step 1: Create New Space

1. 1.Go to huggingface.co/spaces
2. 2.Click "Create new Space"
3. 3.Select "Gradio" as SDK
4. 4.Choose "Public" or "Private"

Step 2: Upload Files

Upload these files to your Space:

• —app.py
• —requirements.txt
• —.gitignore (optional)

Step 3: Configure Space Settings

• —Docker: Leave as default (builds from requirements.txt)
• —Python requirements: Auto-detected from requirements.txt
• —Persistent storage: Not needed for this project

Space will auto-restart after upload. Models will be downloaded on first use.

🔍 Advanced Usage

Using the API Programmatically

import requests
import json

# Call the chat endpoint
response = requests.post(
    "https://your-username-ai-backend.hf.space/api/predict",
    json={
        "data": [
            "Hello, what is Python?",  # prompt
            150,                        # max_tokens
            0.7                         # temperature
        ]
    }
)

result = response.json()
print(result["data"][0])  # Generated response

Using with cURL

curl -X POST https://your-username-ai-backend.hf.space/api/predict \
  -H "Content-Type: application/json" \
  -d '{
    "data": ["What is machine learning?", 150, 0.7]
  }'

Custom Model Loading

To use different models, modify the model names in app.py:

# In model_manager.load_chat_model():
model_name = "different/model-name"  # Change here

Recommended lightweight alternatives:

• —Chat: microsoft/phi-1, mosaicml/mpt-7b-instruct (7B, may be heavy)
• —Summarization: google/flan-t5-base (larger, ~250M)
• —Code: Same TinyLlama or try Salesforce/codet5-small

⚠️ Troubleshooting

Out of Memory Errors

Symptom: Space crashes with OOM Solution:

1. 1.Reduce max_tokens limits in code
2. 2.Reduce max summarization input length
3. 3.Increase queue timeout in demo.launch()

Slow Responses

Symptom: Takes >10 seconds per request Solution:

1. 1.Reduce token limits
2. 2.Disable some models in production
3. 3.Monitor CPU/RAM in Space logs

Model Download Failures

Symptom: "Cannot download model" error Solution:

1. 1.Check internet connectivity in logs
2. 2.Models auto-download on first request (may take 1-2 min)
3. 3.Wait for "Model loaded successfully" message

🎯 Production Checklist

• —✅ Models tested on CPU
• —✅ Error handling for all endpoints
• —✅ Memory cleanup between requests
• —✅ Queue system for concurrency
• —✅ Token limits for stability
• —✅ Float32 precision for CPU
• —✅ Gradio Blocks UI for testing
• —✅ API documentation
• —✅ Lazy loading implemented
• —✅ Optimized requirements.txt

📈 Scaling Beyond Free Tier

If you need more performance:

1. 1.Upgrade to paid GPU tier: Enables larger models (7B+)
2. 2.Use external APIs: ollama, vLLM for local deployment
3. 3.Implement caching: Cache popular responses
4. 4.Model distillation: Train smaller task-specific models

📝 License & Attribution

• —TinyLlama: MIT License
• —FLAN-T5: Apache 2.0
• —Transformers: Apache 2.0
• —Gradio: Apache 2.0

🤝 Contributing

To modify or improve:

1. 1.Clone this Space locally
2. 2.Modify app.py or requirements.txt
3. 3.Test locally with python app.py
4. 4.Push changes back to Space

📧 Support

For issues:

• —Check Space logs (Settings → Logs)
• —Review "Troubleshooting" section above
• —Check Hugging Face Spaces documentation
• —Review model cards on huggingface.co

Built for Hugging Face Spaces - Optimized for FREE CPU Tier 🚀

Created: 2024 Last Updated: 2024