Text-to-Document-Generation/PDF-Redaction-API
0
1# 🚀 Complete FastAPI Deployment Package2 3## 📦 What You've Got4 5A production-ready FastAPI application for PDF redaction with Named Entity Recognition, ready to deploy on HuggingFace Spaces or any cloud platform.6 7---8 9## 📁 Directory Structure10 11```12pdf-redaction-api/13│14├── 📄 main.py # FastAPI application15├── 🐳 Dockerfile # Production container16├── 🐳 docker-compose.yml # Local development17├── 📋 requirements.txt # Python dependencies18│19├── 📱 app/20│ ├── __init__.py21│ └── redaction.py # Core redaction engine22│23├── 📂 uploads/ # Temporary uploads24│ └── .gitkeep25│26├── 📂 outputs/ # Redacted PDFs27│ └── .gitkeep28│29├── 🧪 tests/30│ └── test_api.py # API tests31│32├── 📚 Documentation/33│ ├── README.md # Main docs (for HF Spaces)34│ ├── DEPLOYMENT.md # Deployment guide35│ ├── QUICKSTART.md # Quick start guide36│ └── STRUCTURE.md # Project structure37│38├── 🔧 Configuration/39│ ├── .env.example # Environment variables40│ ├── .gitignore # Git ignore41│ └── .dockerignore # Docker ignore42│43├── 🤖 .github/44│ └── workflows/45│ └── ci-cd.yml # GitHub Actions CI/CD46│47├── 📝 client_example.py # Example API client48└── 📜 LICENSE # MIT License49```50 51---52 53## ✨ Features54 55### Core Functionality56✅ PDF upload and processing57✅ OCR with pytesseract (configurable DPI)58✅ Named Entity Recognition (NER)59✅ Accurate coordinate-based redaction60✅ Multiple entity type support61✅ Downloadable redacted PDFs62 63### API Features64✅ RESTful API with FastAPI65✅ Automatic OpenAPI documentation66✅ File upload handling67✅ Background task cleanup68✅ Health checks69✅ Statistics endpoint70✅ CORS support71 72### DevOps73✅ Docker containerization74✅ Docker Compose for local dev75✅ GitHub Actions CI/CD76✅ HuggingFace Spaces ready77✅ Comprehensive testing78✅ Logging and monitoring79 80---81 82## 🎯 Quick Deployment Paths83 84### Option 1: HuggingFace Spaces (Recommended for Demo)85 86**Time: 10 minutes**87 88```bash89# 1. Create Space on HuggingFace (select Docker SDK)90# 2. Clone your space91git clone https://huggingface.co/spaces/YOUR_USERNAME/pdf-redaction-api92cd pdf-redaction-api93 94# 3. Copy all files95cp -r /path/to/pdf-redaction-api/* .96 97# 4. Deploy98git add .99git commit -m "Initial deployment"100git push101```102 103**Your API will be at:** `https://YOUR_USERNAME-pdf-redaction-api.hf.space`104 105**Cost:** FREE (with CPU Basic tier)106 107---108 109### Option 2: Docker Locally110 111**Time: 5 minutes**112 113```bash114# Build115docker build -t pdf-redaction-api .116 117# Run118docker run -p 7860:7860 pdf-redaction-api119 120# Test121curl http://localhost:7860/health122```123 124---125 126### Option 3: Direct Python127 128**Time: 3 minutes**129 130```bash131# Install dependencies132sudo apt-get install tesseract-ocr poppler-utils133pip install -r requirements.txt134 135# Run136python main.py137 138# Access at http://localhost:7860139```140 141---142 143## 🔌 API Endpoints144 145### Core Endpoints146 147| Method | Endpoint | Description |148|--------|----------|-------------|149| POST | `/redact` | Upload and redact PDF |150| GET | `/download/{job_id}` | Download redacted PDF |151| GET | `/health` | Health check |152| GET | `/stats` | API statistics |153| DELETE | `/cleanup/{job_id}` | Manual cleanup |154| GET | `/docs` | Interactive API docs |155 156### Example Usage157 158**cURL:**159```bash160curl -X POST "http://localhost:7860/redact" \161 -F "file=@document.pdf" \162 -F "dpi=300"163```164 165**Python:**166```python167import requests168 169response = requests.post(170 "http://localhost:7860/redact",171 files={"file": open("document.pdf", "rb")},172 params={"dpi": 300}173)174 175job_id = response.json()["job_id"]176redacted = requests.get(f"http://localhost:7860/download/{job_id}")177```178 179---180 181## 🎨 Architecture182 183```184┌─────────────────────────────────────────────────────────┐185│ CLIENT REQUEST │186│ (Upload PDF via POST /redact) │187└─────────────────────────────────────────────────────────┘188 ↓189┌─────────────────────────────────────────────────────────┐190│ FASTAPI (main.py) │191│ • Validate file │192│ • Generate job_id │193│ • Save to uploads/ │194└─────────────────────────────────────────────────────────┘195 ↓196┌─────────────────────────────────────────────────────────┐197│ PDFRedactor (app/redaction.py) │198│ │199│ ┌─────────────────────────────────────────┐ │200│ │ 1. OCR (pytesseract) │ │201│ │ • Convert PDF → Images (pdf2image) │ │202│ │ • Extract text + bounding boxes │ │203│ │ • Store image dimensions │ │204│ └─────────────────────────────────────────┘ │205│ ↓ │206│ ┌─────────────────────────────────────────┐ │207│ │ 2. NER (HuggingFace Transformers) │ │208│ │ • Load model │ │209│ │ • Identify entities in text │ │210│ │ • Return entity types + positions │ │211│ └─────────────────────────────────────────┘ │212│ ↓ │213│ ┌─────────────────────────────────────────┐ │214│ │ 3. Mapping │ │215│ │ • Create character span index │ │216│ │ • Match NER entities to OCR boxes │ │217│ └─────────────────────────────────────────┘ │218│ ↓ │219│ ┌─────────────────────────────────────────┐ │220│ │ 4. Redaction (pypdf) │ │221│ │ • Scale image coords → PDF coords │ │222│ │ • Create black rectangle annotations │ │223│ │ • Write redacted PDF │ │224│ └─────────────────────────────────────────┘ │225└─────────────────────────────────────────────────────────┘226 ↓227┌─────────────────────────────────────────────────────────┐228│ RESPONSE │229│ • job_id │230│ • List of entities │231│ • Download URL │232└─────────────────────────────────────────────────────────┘233```234 235---236 237## 🔐 Security Considerations238 239### Current Implementation240- ✅ File validation (PDF only)241- ✅ Temporary file cleanup242- ✅ CORS middleware243- ✅ Error handling244 245### For Production (TODO)246- ⚠️ Add API key authentication247- ⚠️ Implement rate limiting248- ⚠️ Add file size limits249- ⚠️ Use HTTPS only250- ⚠️ Implement user quotas251- ⚠️ Add input sanitization252 253**Example API Key Auth:**254```python255# Add to main.py256from fastapi import Security, HTTPException257from fastapi.security import APIKeyHeader258 259API_KEY = "your-secret-key"260api_key_header = APIKeyHeader(name="X-API-Key")261 262def verify_api_key(key: str = Security(api_key_header)):263 if key != API_KEY:264 raise HTTPException(401, "Invalid API Key")265```266 267---268 269## 📊 Performance Tuning270 271### DPI Settings272 273| DPI | Quality | Speed | Use Case |274|-----|---------|-------|----------|275| 150 | Low | Fast | Quick previews |276| 200 | Medium | Medium | General use |277| 300 | High | Slow | **Recommended** |278| 600 | Very High | Very Slow | Critical documents |279 280### Hardware Requirements281 282**Minimum (Free Tier):**283- CPU: 2 cores284- RAM: 2GB285- Storage: 1GB286 287**Recommended (Production):**288- CPU: 4+ cores289- RAM: 8GB290- Storage: 10GB291- GPU: Optional (speeds up NER)292 293---294 295## 🧪 Testing296 297```bash298# Install test dependencies299pip install pytest pytest-cov httpx300 301# Run tests302pytest tests/ -v303 304# With coverage305pytest tests/ --cov=app --cov-report=html306 307# View coverage report308open htmlcov/index.html309```310 311---312 313## 📈 Monitoring314 315### Built-in Endpoints316 317**Health Check:**318```bash319curl http://localhost:7860/health320```321 322**Statistics:**323```bash324curl http://localhost:7860/stats325```326 327### Logs328 329**Development:**330```bash331python main.py332# Logs appear in console333```334 335**Docker:**336```bash337docker logs -f container_name338```339 340**HuggingFace Spaces:**341- View in Space dashboard → Logs tab342 343---344 345## 💰 Cost Estimation346 347### HuggingFace Spaces348 349| Tier | CPU | RAM | Price | Use Case |350|------|-----|-----|-------|----------|351| Basic | 2 | 16GB | **FREE** | Demo, testing |352| CPU Upgrade | 4 | 32GB | $0.50/hr | Production |353| GPU T4 | - | - | $0.60/hr | Heavy load |354| GPU A10G | - | - | $1.50/hr | Enterprise |355 356**Monthly Costs (if always on):**357- Free: $0358- CPU Upgrade: ~$360/month359- GPU T4: ~$432/month360 361**Recommendation:** Start free, upgrade based on usage362 363### Alternatives364 365**AWS ECS Fargate:** ~$30-100/month 366**Google Cloud Run:** Pay per request (~$10-50/month) 367**DigitalOcean App:** $12-24/month 368**Self-hosted VPS:** $5-20/month369 370---371 372## 🔄 CI/CD Pipeline373 374### Automated with GitHub Actions375 376```377Push to GitHub378 ↓379 [Run Tests]380 ↓381 [Build Docker]382 ↓383 [Test Container]384 ↓385[Deploy to HuggingFace]386```387 388**Setup:**3891. Add secrets in GitHub repo settings:390 - `HF_TOKEN`: HuggingFace access token391 - `HF_SPACE`: Your space name (username/space-name)392 3932. Push to main branch → Auto-deploy! ✨394 395---396 397## 📚 Documentation Access398 399| Document | Purpose |400|----------|---------|401| `README.md` | Overview, API docs, usage examples |402| `QUICKSTART.md` | 5-minute setup guide |403| `DEPLOYMENT.md` | Production deployment |404| `STRUCTURE.md` | Code organization |405| `/docs` endpoint | Interactive API documentation |406 407---408 409## 🎓 Learning Resources410 411### FastAPI412- Docs: https://fastapi.tiangolo.com413- Tutorial: https://fastapi.tiangolo.com/tutorial414 415### HuggingFace416- Spaces: https://huggingface.co/docs/hub/spaces417- Transformers: https://huggingface.co/docs/transformers418 419### Docker420- Getting Started: https://docs.docker.com/get-started421 422---423 424## 🐛 Troubleshooting425 426### Common Issues427 428**Problem:** "Tesseract not found" 429**Solution:** `apt-get install tesseract-ocr`430 431**Problem:** "Poppler not found" 432**Solution:** `apt-get install poppler-utils`433 434**Problem:** Slow processing 435**Solution:** Lower DPI to 150-200436 437**Problem:** Out of memory 438**Solution:** Upgrade hardware or reduce DPI439 440**Problem:** Model not loading 441**Solution:** Check internet, wait for download442 443### Debug Mode444 445```python446# In main.py, add debug mode447if __name__ == "__main__":448 uvicorn.run("main:app", host="0.0.0.0", port=7860, reload=True, log_level="debug")449```450 451---452 453## ✅ Checklist for Production454 455- [ ] Test all endpoints thoroughly456- [ ] Add API key authentication457- [ ] Implement rate limiting458- [ ] Set up monitoring (Sentry, DataDog, etc.)459- [ ] Configure auto-scaling460- [ ] Set up backups461- [ ] Add usage analytics462- [ ] Create user documentation463- [ ] Set up SSL/TLS (HF provides by default)464- [ ] Test with large files465- [ ] Load testing466- [ ] Security audit467- [ ] Legal compliance (GDPR, etc.)468 469---470 471## 🎉 You're Ready!472 473Your FastAPI PDF Redaction application is complete and ready to deploy!474 475### Next Steps:4761. ✨ Deploy to HuggingFace Spaces (easiest)4772. 🧪 Test with real PDFs4783. 📊 Monitor usage4794. 🔒 Add security for production4805. 🚀 Scale as needed481 482### Support:483- 📖 Read the documentation484- 🐛 Check troubleshooting guide485- 💬 HuggingFace community forums486- 📧 Create issues on your repo487 488**Happy Deploying! 🚀**489