MCP-1st-Birthday/VideoBackgroundReplacer2

🎬 BackgroundFX Pro — Professional Video Background Replacement

BackgroundFX Pro is a GPU-accelerated app for Hugging Face Spaces (Docker) that replaces video backgrounds using:

• —SAM2 — high-quality object segmentation
• —MatAnyone — temporal video matting for stable alpha over time

Built on: CUDA 12.1.1, PyTorch 2.5.1 (cu121), torchvision 0.20.1, Streamlit 1.49.1.

✨ Features

• —Replace backgrounds with: solid color, AI-generated image (procedural), custom uploaded image, or professional backgrounds
• —Optimized for T4 GPUs on Hugging Face
• —Two-stage pipeline: SAM2 segmentation → MatAnyone refinement → compositing
• —Caching & logs stored in the repo volume:
• —HF cache → ./.hf
• —Torch cache → ./.torch
• —App data & logs → ./data (see data/run.log)
• —FFmpeg — video format conversion and frame extraction

🚀 Try It

Open the Space in your browser (GPU required): https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2

🖱️ How to Use

1. 1.Upload a video (.mp4, .mov, .avi, .mkv).
2. 2.Choose a Background Type: Image, Color, Blur, Professional Backgrounds, or AI Generated.
3. 3.If using custom background, upload your image or select from professional options.
4. 4.Click 🚀 Process Video.
5. 5.Preview and 💾 Download Result.

Tip: Start with 720p/1080p on T4; 4K can exceed memory limits.

🗂️ Project Structure (key files)

• —Dockerfile — CUDA 12.1.1 + PyTorch 2.5.1 container
• —requirements.txt — Python dependencies
• —app.py — Main Streamlit application
• —integrated_pipeline.py — Two-stage processing pipeline
• —models/sam2_loader.py — SAM2 model loader with HF Hub integration
• —models/matanyone_loader.py — MatAnyone model loader
• —utils/ — Utility functions
• —data/ — Created at runtime for logs/outputs
• —tmp/ — Created at runtime for processing jobs - video_pipeline.py — Core video processing logic (SAM2 + MatAnyone integration)
• —video_pipeline.py — Core video processing logic (SAM2 + MatAnyone integration)

⚙️ Runtime Notes

• —Binds to PORT / STREAMLIT_SERVER_PORT (defaults to 7860)
• —File upload limit: 200MB via --server.maxUploadSize=200
• —CORS disabled for Docker compatibility: --server.enableCORS=false
• —Memory management with automatic cleanup between stages
• —If processing fails, check Space logs for detailed error information

🧪 Local Development (Docker)

Requires an NVIDIA GPU with CUDA drivers.

git clone https://huggingface.co/spaces/MogensR/VideoBackgroundReplacer2
cd VideoBackgroundReplacer2

# Build (Ubuntu 22.04, CUDA 12.1.1; installs Torch 2.5.1+cu121)
docker build -t backgroundfx-pro .

# Run
docker run --gpus all -p 7860:7860 backgroundfx-pro

Access at: http://localhost:7860

🔧 Technical Details

Pipeline Architecture

1. 1.Stage 1: SAM2 generates object masks using click points
2. 2.Stage 2: MatAnyone refines masks for temporal consistency
3. 3.Stage 3: Composite foreground with new background

Model Loading

• —SAM2 models downloaded from Hugging Face Hub automatically
• —Supports small/base/large variants (small recommended for T4)
• —MatAnyone loaded from official repository

Performance Optimizations

• —T4-specific optimizations (fp16, channels_last)
• —Memory pruning during long video processing
• —Automatic model unloading between stages

📝 License

MIT License - See LICENSE file for details.