numz/SeedVR2_comfyUI
324472k
1---2license: apache-2.03pipeline_tag: video-to-video4library_name: diffusers5tags:6- art7base_model:8- ByteDance-Seed/SeedVR2-7B9- ByteDance-Seed/SeedVR2-3B10---11 12# ComfyUI-SeedVR2_VideoUpscaler13 14[](https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler)15 16Official release of [SeedVR2](https://github.com/ByteDance-Seed/SeedVR) for ComfyUI that enables high-quality video and image upscaling.17 18Can run as **Multi-GPU standalone CLI** too, see [🖥️ Run as Standalone](#️-run-as-standalone-cli) section.19 20[](https://youtu.be/MBtWYXq_r60)21 2223 2425 26## 📋 Quick Access27 28- [🆙 Future Releases](#-future-releases)29- [🚀 Updates](#-updates)30- [🎯 Features](#-features)31- [🔧 Requirements](#-requirements)32- [📦 Installation](#-installation)33- [📖 Usage](#-usage)34- [🖥️ Run as Standalone](#️-run-as-standalone-cli)35- [⚠️ Limitations](#️-limitations)36- [🤝 Contributing](#-contributing)37- [🙏 Credits](#-credits)38- [📜 License](#-license)39 40## 🆙 Future Releases41 42We're actively working on improvements and new features. To stay informed:43 44- **📌 Track Active Development**: Visit [Issues](https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler/issues) to see active development, report bugs, and request new features45- **💬 Join the Community**: Learn from others, share your workflows, and get help in the [Discussions](https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler/discussions)46- **🔮 Next Model Survey**: We're looking for community input on the next open-source super-powerful generic restoration model. Share your suggestions in [Issue #164](https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler/issues/164)47 48## 🚀 Updates49 50**2025.11.09 - Version 2.5.5**51 52- 💾 **Memory: Fixed RAM leak for long videos** - On-demand reconstruction with lightweight batch indices instead of storing full transformed videos, fixed release_tensor_memory to handle CPU/CUDA/MPS consistently, and refactored batch processing helpers53 54**2025.11.08 - Version 2.5.4**55 56- 🎨 **Fix: AdaIN color correction** - Replace `.view()` with `.reshape()` to handle non-contiguous tensors after spatial padding, resolving "view size is not compatible with input tensor's size and stride" error57- 🔴 **Fix: AMD ROCm compatibility** - Add cuDNN availability check in Conv3d workaround to prevent "ATen not compiled with cuDNN support" error on ROCm systems (AMD GPUs on Windows/Linux)58 59**2025.11.08 - Version 2.5.3**60 61- 🍎 **Fix: Apple Silicon MPS device handling** - Corrected MPS device enumeration to use `"mps"` instead of `"mps:0"`, resolving invalid device errors on M-series Macs62- 🪟 **Fix: torch.mps AttributeError on Windows** - Add defensive checks for `torch.mps.is_available()` to handle PyTorch versions where the method doesn't exist on non-Mac platforms63 64**2025.11.07 - Version 2.5.0** 🎉65 66⚠️ **BREAKING CHANGE**: This is a major update requiring workflow recreation. All nodes and CLI parameters have been redesigned for better usability and consistency. Watch the latest video from [AInVFX](https://www.youtube.com/@AInVFX) for a deep dive and check out the [usage](#-usage) section.67 68**📦 Official Release**: Now available on main branch with ComfyUI Manager support for easy installation and automatic version tracking. Updated dependencies and local imports prevent conflicts with other ComfyUI custom nodes.69 70### 🎨 ComfyUI Improvements71 72- **Four-Node Modular Architecture**: Split into dedicated nodes for DiT model, VAE model, torch.compile settings, and main upscaler for granular control73- **Global Model Cache**: Models now shared across multiple upscaler instances with automatic config updates - no more redundant loading74- **ComfyUI V3 Migration**: Full compatibility with ComfyUI V3 stateless node design75- **RGBA Support**: Native alpha channel processing with edge-guided upscaling for clean transparency76- **Improved Memory Management**: Streaming architecture prevents VRAM spikes regardless of video length77- **Flexible Resolution Support**: Upscale to any resolution divisible by 2 with lossless padding approach (replaced restrictive cropping)78- **Enhanced Parameters**: Added `uniform_batch_size`, `temporal_overlap`, `prepend_frames`, and `max_resolution` for better control79 80### 🖥️ CLI Enhancements81 82- **Batch Directory Processing**: Process entire folders of videos/images with model caching for efficiency83- **Single Image Support**: Direct image upscaling without video conversion84- **Smart Output Detection**: Auto-detects output format (MP4/PNG) based on input type85- **Enhanced Multi-GPU**: Improved workload distribution with temporal overlap blending86- **Unified Parameters**: CLI and ComfyUI now use identical parameter names for consistency87- **Better UX**: Auto-display help, validation improvements, progress tracking, and cleaner output88 89### ⚡ Performance & Optimization90 91- **torch.compile Support**: 20-40% DiT speedup and 15-25% VAE speedup with full graph compilation92- **Optimized BlockSwap**: Adaptive memory clearing (5% threshold), separate I/O component handling, reduced overhead93- **Enhanced VAE Tiling**: Tensor offload support for accumulation buffers, separate encode/decode configuration94- **Native Dtype Pipeline**: Eliminated unnecessary conversions, maintains bfloat16 precision throughout for speed and quality95- **Optimized Tensor Operations**: Replaced einops rearrange with native PyTorch ops for 2-5x faster transforms96 97### 🎯 Quality Improvements98 99- **LAB Color Correction**: New perceptual color transfer method with superior color accuracy (now default)100- **Additional Color Methods**: HSV saturation matching, wavelet adaptive, and hybrid approaches101- **Deterministic Generation**: Seed-based reproducibility with phase-specific seeding strategy102- **Better Temporal Consistency**: Hann window blending for smooth transitions between batches103 104### 💾 Memory Management105 106- **Smarter Offloading**: Independent device configuration for DiT, VAE, and tensors (CPU/GPU/none)107- **Four-Phase Pipeline**: Completes each phase (encode→upscale→decode→postprocess) for all batches before moving to next, minimizing model swaps108- **Better Cleanup**: Phase-specific resource management with proper tensor memory release109- **Peak VRAM Tracking**: Per-phase memory monitoring with summary display110 111### 🔧 Technical Improvements112 113- **GGUF Quantization Support**: Added full GGUF support for 4-bit/8-bit inference on low-VRAM systems114- **Improved GGUF Handling**: Fixed VRAM leaks, torch.compile compatibility, non-persistent buffers115- **Apple Silicon Support**: Full MPS (Metal Performance Shaders) support for Apple Silicon Macs116- **AMD ROCm Compatibility**: Conditional FSDP imports for PyTorch ROCm 7+ support117- **Conv3d Memory Workaround**: Fixes PyTorch 2.9+ cuDNN memory bug (3x usage reduction)118- **Flash Attention Optional**: Graceful fallback to SDPA when flash-attn unavailable119 120### 📚 Code Quality121 122- **Modular Architecture**: Split monolithic files into focused modules (generation_phases, model_configuration, etc.)123- **Comprehensive Documentation**: Extensive docstrings with type hints across all modules124- **Better Error Handling**: Early validation, clear error messages, installation instructions125- **Consistent Logging**: Unified indentation, better categorization, concise messages126 127**2025.08.07**128 129- 🎯 **Unified Debug System**: New structured logging with categories, timers, and memory tracking. `enable_debug` now available on main node130- ⚡ **Smart FP8 Optimization**: FP8 models now keep native FP8 storage, converting to BFloat16 only for arithmetic - faster and more memory efficient than FP16131- 📦 **Model Registry**: Multi-repo support (numz/ & AInVFX/), auto-discovery of user models, added mixed FP8 variants to fix 7B artifacts132- 💾 **Model Caching**: `cache_model` moved to main node, fixed memory leaks with proper RoPE/wrapper cleanup133- 🧹 **Code Cleanup**: New modular structure (`constants.py`, `model_registry.py`, `debug.py`), removed legacy code134- 🚀 **Performance**: Better memory management with `torch.cuda.ipc_collect()`, improved RoPE handling135 136**2025.07.17**137 138- 🛠️ Add 7B sharp Models: add 2 new 7B models with sharpen output139 140**2025.07.11**141 142- 🎬 Complete tutorial released: Adrien from [AInVFX](https://www.youtube.com/@AInVFX) created an in-depth ComfyUI SeedVR2 guide covering everything from basic setup to advanced BlockSwap techniques for running on consumer GPUs. Perfect for understanding memory optimization and upscaling of image sequences with alpha channel! [Watch the tutorial](#-usage)143 144**2025.09.07**145 146- 🛠️ Blockswap Integration: Big thanks to [Adrien Toupet](https://github.com/adrientoupet) from [AInVFX](https://www.youtube.com/@AInVFX) for this :), useful for low VRAM users (see [usage](#-usage) section)147 148**2025.07.03**149 150- 🛠️ Can run as **standalone mode** with **Multi GPU** see [🖥️ Run as Standalone](#️-run-as-standalone-cli)151 152**2025.06.30**153 154- 🚀 Speed Up the process and less VRAM used155- 🛠️ Fixed memory leak on 3B models156- ❌ Can now interrupt process if needed157- ✅ Refactored the code for better sharing with the community, feel free to propose pull requests158- 🛠️ Removed flash attention dependency (thanks to [luke2642](https://github.com/Luke2642) !!)159 160**2025.06.24**161 162- 🚀 Speed up the process until x4163 164**2025.06.22**165 166- 💪 FP8 compatibility !167- 🚀 Speed Up all Process168- 🚀 less VRAM consumption (Stay high, batch_size=1 for RTX4090 max, I'm trying to fix that)169- 🛠️ Better benchmark coming soon170 171**2025.06.20**172 173- 🛠️ Initial push174 175## 🎯 Features176 177### Core Capabilities178- **High-Quality Diffusion-Based Upscaling**: One-step diffusion model for video and image enhancement179- **Temporal Consistency**: Maintains coherence across video frames with configurable batch processing180- **Multi-Format Support**: Handles RGB and RGBA (alpha channel) for both videos and images181- **Any Video Length**: Suitable for any video length182 183### Model Support184- **Multiple Model Variants**: 3B and 7B parameter models with different precision options185- **FP16, FP8, and GGUF Quantization**: Choose between full precision (FP16), mixed precision (FP8), or heavily quantized GGUF models for different VRAM requirements186- **Automatic Model Downloads**: Models are automatically downloaded from HuggingFace on first use187 188### Memory Optimization189- **BlockSwap Technology**: Dynamically swap transformer blocks between GPU and CPU memory to run large models on limited VRAM190- **VAE Tiling**: Process large resolutions with tiled encoding/decoding to reduce VRAM usage191- **Intelligent Offloading**: Offload models and intermediate tensors to CPU or secondary GPUs between processing phases192- **GGUF Quantization Support**: Run models with 4-bit or 8-bit quantization for extreme VRAM savings193 194### Performance Features195- **torch.compile Integration**: Optional 20-40% DiT speedup and 15-25% VAE speedup with PyTorch 2.0+ compilation196- **Multi-GPU CLI**: Distribute workload across multiple GPUs with automatic temporal overlap blending197- **Model Caching**: Keep models loaded in memory for faster batch processing198- **Flexible Attention Backends**: Choose between PyTorch SDPA (stable, always available) or Flash Attention 2 (faster on supported hardware)199 200### Quality Control201- **Advanced Color Correction**: Five methods including LAB (recommended for highest fidelity), wavelet, wavelet adaptive, HSV, and AdaIN202- **Noise Injection Controls**: Fine-tune input and latent noise scales for artifact reduction at high resolutions203- **Configurable Resolution Limits**: Set target and maximum resolutions with automatic aspect ratio preservation204 205### Workflow Features206- **ComfyUI Integration**: Four dedicated nodes for complete control over the upscaling pipeline207- **Standalone CLI**: Command-line interface for batch processing and automation208- **Debug Logging**: Comprehensive debug mode with memory tracking, timing information, and processing details209- **Progress Reporting**: Real-time progress updates during processing210 211## 🔧 Requirements212 213### Hardware214 215With the current optimizations (tiling, BlockSwap, GGUF quantization), SeedVR2 can run on a wide range of hardware:216 217- **Minimal VRAM** (8GB or less): Use GGUF Q4_K_M models with BlockSwap and VAE tiling enabled218- **Moderate VRAM** (12-16GB): Use FP8 models with BlockSwap or VAE tiling as needed219- **High VRAM** (24GB+): Use FP16 models for best quality and speed without memory optimizations220 221### Software222 223- **ComfyUI**: Latest version recommended224- **Python**: 3.12+ (Python 3.12 and 3.13 tested and recommended)225- **PyTorch**: 2.0+ for torch.compile support (optional but recommended)226- **Triton**: Required for torch.compile with inductor backend (optional)227- **Flash Attention 2**: Provides faster attention computation on supported hardware (optional, falls back to PyTorch SDPA)228 229## 📦 Installation230 231### Option 1: ComfyUI Manager (Recommended)232 2331. Open ComfyUI Manager in your ComfyUI interface2342. Click "Custom Nodes Manager"2353. Search for "ComfyUI-SeedVR2_VideoUpscaler"2364. Click "Install" and restart ComfyUI237 238**Registry Link**: [ComfyUI Registry - SeedVR2 Video Upscaler](https://registry.comfy.org/nodes/seedvr2_videoupscaler)239 240### Option 2: Manual Installation241 2421. **Clone the repository** into your ComfyUI custom nodes directory:243```bash244cd ComfyUI245git clone https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler.git custom_nodes/seedvr2_videoupscaler246```247 2482. **Install dependencies using standalone Python**:249```bash250# Install requirements (from same ComfyUI directory)251# Windows:252.venv\Scripts\python.exe -m pip install -r custom_nodes\seedvr2_videoupscaler\requirements.txt253# Linux/macOS:254.venv/bin/python -m pip install -r custom_nodes/seedvr2_videoupscaler/requirements.txt255```256 2573. **Restart ComfyUI**258 259### Model Installation260 261Models will be **automatically downloaded** on first use and saved to `ComfyUI/models/SEEDVR2`.262 263You can also manually download models from:264- Main models available at [numz/SeedVR2_comfyUI](https://huggingface.co/numz/SeedVR2_comfyUI/tree/main) and [AInVFX/SeedVR2_comfyUI](https://huggingface.co/AInVFX/SeedVR2_comfyUI/tree/main)265- Additional GGUF models available at [cmeka/SeedVR2-GGUF](https://huggingface.co/cmeka/SeedVR2-GGUF/tree/main)266 267## 📖 Usage268 269### 🎬 Video Tutorials270 271#### Latest Version Deep Dive (Recommended)272 273Complete walkthrough of version 2.5 by Adrien from [AInVFX](https://www.youtube.com/@AInVFX), covering the new 4-node architecture, GGUF support, memory optimizations, and production workflows:274 275[](https://youtu.be/MBtWYXq_r60)276 277This comprehensive tutorial covers:278- Installing v2.5 through ComfyUI Manager and troubleshooting conflicts279- Understanding the new 4-node modular architecture and why we rebuilt it280- Running 7B models on 8GB VRAM with GGUF quantization281- Configuring BlockSwap, VAE tiling, and torch.compile for your hardware282- Image and video upscaling workflows with alpha channel support283- CLI for batch processing and multi-GPU rendering284- Memory optimization strategies for different VRAM levels285- Real production tips and the critical batch_size formula (4n+1)286 287#### Previous Version Tutorial288 289For reference, here's the original tutorial covering the initial release:290 291[](https://youtu.be/I0sl45GMqNg)292 293*Note: This tutorial covers the previous single-node architecture. While the UI has changed significantly in v2.5, the core concepts about BlockSwap and memory management remain valuable.*294 295### Node Setup296 297SeedVR2 uses a modular node architecture with four specialized nodes:298 299#### 1. SeedVR2 (Down)Load DiT Model300 301302 303Configure the DiT (Diffusion Transformer) model for video upscaling.304 305**Parameters:**306 307- **model**: Choose your DiT model308 - **3B Models**: Faster, lower VRAM requirements309 - `seedvr2_ema_3b_fp16.safetensors`: FP16 (best quality)310 - `seedvr2_ema_3b_fp8_e4m3fn.safetensors`: FP8 8-bit (good quality)311 - `seedvr2_ema_3b-Q4_K_M.gguf`: GGUF 4-bit quantized (acceptable quality)312 - `seedvr2_ema_3b-Q8_0.gguf`: GGUF 8-bit quantized (good quality)313 - **7B Models**: Higher quality, higher VRAM requirements314 - `seedvr2_ema_7b_fp16.safetensors`: FP16 (best quality)315 - `seedvr2_ema_7b_fp8_e4m3fn_mixed_block35_fp16.safetensors`: FP8 with last block in FP16 to reduce artifacts (good quality)316 - `seedvr2_ema_7b-Q4_K_M.gguf`: GGUF 4-bit quantized (acceptable quality)317 - `seedvr2_ema_7b_sharp_*`: Sharp variants for enhanced detail318 319- **device**: GPU device for DiT inference (e.g., `cuda:0`)320 321- **offload_device**: Device to offload DiT model when not actively processing322 - `none`: Keep model on inference device (fastest, highest VRAM)323 - `cpu`: Offload to system RAM (reduces VRAM)324 - `cuda:X`: Offload to another GPU (good balance if available)325 326- **cache_model**: Keep DiT model loaded on offload_device between workflow runs327 - Useful for batch processing to avoid repeated loading328 - Requires offload_device to be set329 330- **blocks_to_swap**: BlockSwap memory optimization331 - `0`: Disabled (default)332 - `1-32`: Number of transformer blocks to swap for 3B model333 - `1-36`: Number of transformer blocks to swap for 7B model334 - Higher values = more VRAM savings but slower processing335 - Requires offload_device to be set and different from device336 337- **swap_io_components**: Offload input/output embeddings and normalization layers338 - Additional VRAM savings when combined with blocks_to_swap339 - Requires offload_device to be set and different from device340 341- **attention_mode**: Attention computation backend342 - `sdpa`: PyTorch scaled_dot_product_attention (default, stable, always available)343 - `flash_attn`: Flash Attention 2 (faster on supported hardware, requires flash-attn package)344 345- **torch_compile_args**: Connect to SeedVR2 Torch Compile Settings node for 20-40% speedup346 347**BlockSwap Explained:**348 349BlockSwap enables running large models on GPUs with limited VRAM by dynamically swapping transformer blocks between GPU and CPU memory during inference. Here's how it works:350 351- **What it does**: Keeps only the currently-needed transformer blocks on the GPU, while storing the rest on CPU or another device352- **When to use it**: When you get OOM (Out of Memory) errors during the upscaling phase353- **How to configure**:354 1. Set `offload_device` to `cpu` or another GPU355 2. Start with `blocks_to_swap=16` (half the blocks)356 3. If still getting OOM, increase to 24 or 32 (3B) / 36 (7B)357 4. Enable `swap_io_components` for maximum VRAM savings358 5. If you have plenty of VRAM, decrease or set to 0 for faster processing359 360**Example Configuration for Low VRAM (8GB)**:361- model: `seedvr2_ema_3b-Q8_0.gguf`362- device: `cuda:0`363- offload_device: `cpu`364- blocks_to_swap: `32`365- swap_io_components: `True`366 367#### 2. SeedVR2 (Down)Load VAE Model368 369370 371Configure the VAE (Variational Autoencoder) model for encoding/decoding video frames.372 373**Parameters:**374 375- **model**: VAE model selection376 - `ema_vae_fp16.safetensors`: Default and recommended377 378- **device**: GPU device for VAE inference (e.g., `cuda:0`)379 380- **offload_device**: Device to offload VAE model when not actively processing381 - `none`: Keep model on inference device (default, fastest)382 - `cpu`: Offload to system RAM (reduces VRAM)383 - `cuda:X`: Offload to another GPU (good balance if available)384 385- **cache_model**: Keep VAE model loaded on offload_device between workflow runs386 - Requires offload_device to be set387 388- **encode_tiled**: Enable tiled encoding to reduce VRAM usage during encoding phase389 - Enable if you see OOM errors during the "Encoding" phase in debug logs390 391- **encode_tile_size**: Encoding tile size in pixels (default: 1024)392 - Applied to both height and width393 - Lower values reduce VRAM but may increase processing time394 395- **encode_tile_overlap**: Encoding tile overlap in pixels (default: 128)396 - Reduces visible seams between tiles397 398- **decode_tiled**: Enable tiled decoding to reduce VRAM usage during decoding phase399 - Enable if you see OOM errors during the "Decoding" phase in debug logs400 401- **decode_tile_size**: Decoding tile size in pixels (default: 1024)402 403- **decode_tile_overlap**: Decoding tile overlap in pixels (default: 128)404 405- **torch_compile_args**: Connect to SeedVR2 Torch Compile Settings node for 15-25% speedup406 407**VAE Tiling Explained:**408 409VAE tiling processes large resolutions in smaller tiles to reduce VRAM requirements. Here's how to use it:410 4111. **Run without tiling first** and monitor the debug logs (enable `enable_debug` on main node)4122. **If OOM during "Encoding" phase**:413 - Enable `encode_tiled`414 - If still OOM, reduce `encode_tile_size` (try 768, 512, etc.)4153. **If OOM during "Decoding" phase**:416 - Enable `decode_tiled`417 - If still OOM, reduce `decode_tile_size`4184. **Adjust overlap** (default 128) if you see visible seams in output (increase it) or processing times are too slow (decrease it).419 420**Example Configuration for High Resolution (4K)**:421- encode_tiled: `True`422- encode_tile_size: `1024`423- encode_tile_overlap: `128`424- decode_tiled: `True`425- decode_tile_size: `1024`426- decode_tile_overlap: `128`427 428#### 3. SeedVR2 Torch Compile Settings (Optional)429 430431 432Configure torch.compile optimization for 20-40% DiT speedup and 15-25% VAE speedup.433 434**Requirements:**435- PyTorch 2.0+436- Triton (for inductor backend)437 438**Parameters:**439 440- **backend**: Compilation backend441 - `inductor`: Full optimization with Triton kernel generation and fusion (recommended)442 - `cudagraphs`: Lightweight wrapper using CUDA graphs, no kernel optimization443 444- **mode**: Optimization level (compilation time vs runtime performance)445 - `default`: Fast compilation with good speedup (recommended for development)446 - `reduce-overhead`: Lower overhead, optimized for smaller models447 - `max-autotune`: Slowest compilation, best runtime performance (recommended for production)448 - `max-autotune-no-cudagraphs`: Like max-autotune but without CUDA graphs449 450- **fullgraph**: Compile entire model as single graph without breaks451 - `False`: Allow graph breaks for better compatibility (default, recommended)452 - `True`: Enforce no breaks for maximum optimization (may fail with dynamic shapes)453 454- **dynamic**: Handle varying input shapes without recompilation455 - `False`: Specialize for exact input shapes (default)456 - `True`: Create dynamic kernels that adapt to shape variations (enable when processing different resolutions or batch sizes)457 458- **dynamo_cache_size_limit**: Max cached compiled versions per function (default: 64)459 - Higher = more memory, lower = more recompilation460 461- **dynamo_recompile_limit**: Max recompilation attempts before falling back to eager mode (default: 128)462 - Safety limit to prevent compilation loops463 464**Usage:**4651. Add this node to your workflow4662. Connect its output to the `torch_compile_args` input of DiT and/or VAE loader nodes4673. First run will be slow (compilation), subsequent runs will be much faster468 469**When to use:**470- torch.compile only makes sense when processing **multiple batches, long videos, or many tiles**471- For single images or short clips, the compilation time outweighs the speed improvement472- Best suited for batch processing workflows or long videos473 474**Recommended Settings:**475- For development/testing: `mode=default`, `backend=inductor`, `fullgraph=False`476- For production: `mode=max-autotune`, `backend=inductor`, `fullgraph=False`477 478#### 4. SeedVR2 Video Upscaler (Main Node)479 480481 482Main upscaling node that processes video frames using DiT and VAE models.483 484**Required Inputs:**485 486- **image**: Input video frames as image batch (RGB or RGBA format)487- **dit**: DiT model configuration from SeedVR2 (Down)Load DiT Model node488- **vae**: VAE model configuration from SeedVR2 (Down)Load VAE Model node489 490**Parameters:**491 492- **seed**: Random seed for reproducible generation (default: 42)493 - Same seed with same inputs produces identical output494 495- **resolution**: Target resolution for shortest edge in pixels (default: 1080)496 - Maintains aspect ratio automatically497 498- **max_resolution**: Maximum resolution for any edge (default: 0 = no limit)499 - Automatically scales down if exceeded to prevent OOM500 501- **batch_size**: Frames per batch (default: 5)502 - **CRITICAL REQUIREMENT**: Must follow the **4n+1 formula** (1, 5, 9, 13, 17, 21, 25, ...)503 - **Why this matters**: The model uses these frames for temporal consistency calculations504 - **Minimum 5 for temporal consistency**: Use 1 only for single images or when temporal consistency isn't needed505 - **Match shot length ideally**: For best results, set batch_size to match your shot length (e.g., batch_size=21 for a 20-frame shot)506 - **VRAM impact**: Higher batch_size = better quality and speed but requires more VRAM507 - **If you get OOM with batch_size=5**: Try optimization techniques first (model offloading, BlockSwap, GGUF models...) before reducing batch_size or input resolution, as these directly impact quality508 509**uniform_batch_size** (default: False)510 - Pads the final batch to match `batch_size` for uniform processing511 - Prevents temporal artifacts when the last batch is significantly smaller than others512 - Example: 45 frames with `batch_size=33` creates [33, 33] instead of [33, 12]513 - Recommended when using large batch sizes and video length is not a multiple of `batch_size`514 - Increases VRAM usage slightly but ensures consistent temporal coherence across all batches515 516- **temporal_overlap**: Overlapping frames between batches (default: 0)517 - Used for blending between batches to reduce temporal artifacts518 - Range: 0-16 frames519 520- **prepend_frames**: Frames to prepend (default: 0)521 - Prepends reversed frames to reduce artifacts at video start522 - Automatically removed after processing523 - Range: 0-32 frames524 525- **color_correction**: Color correction method (default: "wavelet")526 - **`lab`**: Full perceptual color matching with detail preservation (recommended for highest fidelity to original)527 - **`wavelet`**: Frequency-based natural colors, preserves details well528 - **`wavelet_adaptive`**: Wavelet base + targeted saturation correction529 - **`hsv`**: Hue-conditional saturation matching530 - **`adain`**: Statistical style transfer531 - **`none`**: No color correction532 533- **input_noise_scale**: Input noise injection scale 0.0-1.0 (default: 0.0)534 - Adds noise to input frames to reduce artifacts at very high resolutions535 - Try 0.1-0.3 if you see artifacts with high output resolutions536 537- **latent_noise_scale**: Latent space noise scale 0.0-1.0 (default: 0.0)538 - Adds noise during diffusion process, can soften excessive detail539 - Use if input_noise doesn't help, try 0.05-0.15540 541- **offload_device**: Device for storing intermediate tensors between processing phases (default: "cpu")542 - `none`: Keep all tensors on inference device (fastest but highest VRAM)543 - `cpu`: Offload to system RAM (recommended for long videos, slower transfers)544 - `cuda:X`: Offload to another GPU (good balance if available, faster than CPU)545 546- **enable_debug**: Enable detailed debug logging (default: False)547 - Shows memory usage, timing information, and processing details548 - **Highly recommended** for troubleshooting OOM issues549 550**Output:**551- Upscaled video frames with color correction applied552- Format (RGB/RGBA) matches input553- Range [0, 1] normalized for ComfyUI compatibility554 555### Typical Workflow Setup556 557**Basic Workflow (High VRAM - 24GB+)**:558```559Load Video Frames560 ↓561SeedVR2 Load DiT Model562 ├─ model: seedvr2_ema_3b_fp16.safetensors563 └─ device: cuda:0564 ↓565SeedVR2 Load VAE Model566 ├─ model: ema_vae_fp16.safetensors567 └─ device: cuda:0568 ↓569SeedVR2 Video Upscaler570 ├─ batch_size: 21571 └─ resolution: 1080572 ↓573Save Video/Frames574```575 576**Low VRAM Workflow (8-12GB)**:577```578Load Video Frames579 ↓580SeedVR2 Load DiT Model581 ├─ model: seedvr2_ema_3b-Q8_0.gguf582 ├─ device: cuda:0583 ├─ offload_device: cpu584 ├─ blocks_to_swap: 32585 └─ swap_io_components: True586 ↓587SeedVR2 Load VAE Model588 ├─ model: ema_vae_fp16.safetensors589 ├─ device: cuda:0590 ├─ encode_tiled: True591 └─ decode_tiled: True592 ↓593SeedVR2 Video Upscaler594 ├─ batch_size: 5595 └─ resolution: 720596 ↓597Save Video/Frames598```599 600**High Performance Workflow (24GB+ with torch.compile)**:601```602Load Video Frames603 ↓604SeedVR2 Torch Compile Settings605 ├─ mode: max-autotune606 └─ backend: inductor607 ↓608SeedVR2 Load DiT Model609 ├─ model: seedvr2_ema_7b_sharp_fp16.safetensors610 ├─ device: cuda:0611 └─ torch_compile_args: connected612 ↓613SeedVR2 Load VAE Model614 ├─ model: ema_vae_fp16.safetensors615 ├─ device: cuda:0616 └─ torch_compile_args: connected617 ↓618SeedVR2 Video Upscaler619 ├─ batch_size: 81620 └─ resolution: 1080621 ↓622Save Video/Frames623```624 625## 🖥️ Run as Standalone (CLI)626 627The standalone CLI provides powerful batch processing capabilities with multi-GPU support and sophisticated optimization options.628 629### Prerequisites630 631Choose the appropriate setup based on your installation:632 633#### Option 1: Already Have ComfyUI with SeedVR2 Installed634 635If you've already installed SeedVR2 as part of ComfyUI (via [ComfyUI installation](#-installation)), you can use the CLI directly:636 637```bash638# Navigate to your ComfyUI directory639cd ComfyUI640 641# Run the CLI using standalone Python (display help message)642# Windows:643.venv\Scripts\python.exe custom_nodes\seedvr2_videoupscaler\inference_cli.py --help644# Linux/macOS:645.venv/bin/python custom_nodes/seedvr2_videoupscaler/inference_cli.py --help646```647 648**Skip to [Command Line Usage](#command-line-usage) below.**649 650#### Option 2: Standalone Installation (Without ComfyUI)651 652If you want to use the CLI without ComfyUI installation, follow these steps:653 6541. **Install [uv](https://docs.astral.sh/uv/getting-started/installation/)** (modern Python package manager):655```bash656# Windows657powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"658 659# macOS and Linux660curl -LsSf https://astral.sh/uv/install.sh | sh661```662 6632. **Clone the repository**:664```bash665git clone https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler.git seedvr2_videoupscaler666cd seedvr2_videoupscaler667```668 6693. **Create virtual environment and install dependencies**:670```bash671# Create virtual environment with Python 3.13672uv venv --python 3.13673 674# Activate virtual environment675# Windows:676.venv\Scripts\activate677# Linux/macOS:678source .venv/bin/activate679 680# Install PyTorch with CUDA support681# Check command line based on your environment: https://pytorch.org/get-started/locally/682uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu130683 684# Install SeedVR2 requirements685uv pip install -r requirements.txt686 687# Run the CLI (display help message)688# Windows:689.venv\Scripts\python.exe inference_cli.py --help690# Linux/macOS:691.venv/bin/python inference_cli.py --help692```693 694### Command Line Usage695 696The CLI provides comprehensive options for single-GPU, multi-GPU, and batch processing workflows.697 698**Basic Usage Examples:**699 700```bash701# Basic image upscaling702python inference_cli.py image.jpg703 704# Basic video video upscaling with temporal consistency705python inference_cli.py video.mp4 --resolution 720 --batch_size 33706 707# Multi-GPU processing with temporal overlap708python inference_cli.py video.mp4 \709 --cuda_device 0,1 \710 --resolution 1080 \711 --batch_size 81 \712 --uniform_batch_size \713 --temporal_overlap 3 \714 --prepend_frames 4715 716# Memory-optimized for low VRAM (8GB)717python inference_cli.py image.png \718 --dit_model seedvr2_ema_3b-Q8_0.gguf \719 --resolution 1080 \720 --blocks_to_swap 32 \721 --swap_io_components \722 --dit_offload_device cpu \723 --vae_offload_device cpu724 725# High resolution with VAE tiling726python inference_cli.py video.mp4 \727 --resolution 1440 \728 --batch_size 31 \729 --uniform_batch_size \730 --temporal_overlap 3 \731 --vae_encode_tiled \732 --vae_decode_tiled733 734# Batch directory processing with model caching735python inference_cli.py media_folder/ \736 --output processed/ \737 --cuda_device 0 \738 --cache_dit \739 --cache_vae \740 --dit_offload_device cpu \741 --vae_offload_device cpu \742 --resolution 1080 \743 --max_resolution 1920744```745 746### Command Line Arguments747 748**Input/Output:**749- `<input>`: Input file (.mp4, .avi, .png, .jpg, etc.) or directory750- `--output`: Output path (default: auto-generated in 'output/' directory)751- `--output_format`: Output format: 'mp4' (video) or 'png' (image sequence). Default: auto-detect from input type752- `--model_dir`: Model directory (default: ./models/SEEDVR2)753 754**Model Selection:**755- `--dit_model`: DiT model to use. Options: 3B/7B with fp16/fp8/GGUF variants (default: 3B FP8)756 757**Processing Parameters:**758- `--resolution`: Target short-side resolution in pixels (default: 1080)759- `--max_resolution`: Maximum resolution for any edge. Scales down if exceeded. 0 = no limit (default: 0)760- `--batch_size`: Frames per batch (must follow 4n+1: 1, 5, 9, 13, 17, 21...). Ideally matches shot length for best temporal consistency (default: 5)761- `--seed`: Random seed for reproducibility (default: 42)762- `--skip_first_frames`: Skip N initial frames (default: 0)763- `--load_cap`: Load maximum N frames from video. 0 = load all (default: 0)764- `--prepend_frames`: Prepend N reversed frames to reduce start artifacts (auto-removed) (default: 0)765- `--temporal_overlap`: Frames to overlap between batches/GPUs for smooth blending (default: 0)766 767**Quality Control:**768- `--color_correction`: Color correction method: 'lab' (perceptual, recommended), 'wavelet', 'wavelet_adaptive', 'hsv', 'adain', or 'none' (default: lab)769- `--input_noise_scale`: Input noise injection scale (0.0-1.0). Reduces artifacts at high resolutions (default: 0.0)770- `--latent_noise_scale`: Latent space noise scale (0.0-1.0). Softens details if needed (default: 0.0)771 772**Memory Management:**773- `--dit_offload_device`: Device to offload DiT model: 'none' (keep on GPU), 'cpu', or 'cuda:X' (default: none)774- `--vae_offload_device`: Device to offload VAE model: 'none', 'cpu', or 'cuda:X' (default: none)775- `--blocks_to_swap`: Number of transformer blocks to swap (0=disabled, 3B: 0-32, 7B: 0-36). Requires dit_offload_device (default: 0)776- `--swap_io_components`: Offload I/O components for additional VRAM savings. Requires dit_offload_device777- `--use_non_blocking`: Use non-blocking memory transfers for BlockSwap (recommended)778 779**VAE Tiling:**780- `--vae_encode_tiled`: Enable VAE encode tiling to reduce VRAM during encoding781- `--vae_encode_tile_size`: VAE encode tile size in pixels (default: 1024)782- `--vae_encode_tile_overlap`: VAE encode tile overlap in pixels (default: 128)783- `--vae_decode_tiled`: Enable VAE decode tiling to reduce VRAM during decoding784- `--vae_decode_tile_size`: VAE decode tile size in pixels (default: 1024)785- `--vae_decode_tile_overlap`: VAE decode tile overlap in pixels (default: 128)786- `--tile_debug`: Visualize tiles: 'false' (default), 'encode', or 'decode'787 788**Performance Optimization:**789- `--attention_mode`: Attention backend: 'sdpa' (default, stable) or 'flash_attn' (faster, requires package)790- `--compile_dit`: Enable torch.compile for DiT model (20-40% speedup, requires PyTorch 2.0+ and Triton)791- `--compile_vae`: Enable torch.compile for VAE model (15-25% speedup, requires PyTorch 2.0+ and Triton)792- `--compile_backend`: Compilation backend: 'inductor' (full optimization) or 'cudagraphs' (lightweight) (default: inductor)793- `--compile_mode`: Optimization level: 'default', 'reduce-overhead', 'max-autotune', 'max-autotune-no-cudagraphs' (default: default)794- `--compile_fullgraph`: Compile entire model as single graph (faster but less flexible) (default: False)795- `--compile_dynamic`: Handle varying input shapes without recompilation (default: False)796- `--compile_dynamo_cache_size_limit`: Max cached compiled versions per function (default: 64)797- `--compile_dynamo_recompile_limit`: Max recompilation attempts before fallback (default: 128)798 799**Model Caching (batch processing):**800- `--cache_dit`: Cache DiT model between files (single GPU only, speeds up directory processing)801- `--cache_vae`: Cache VAE model between files (single GPU only, speeds up directory processing)802 803**Multi-GPU:**804- `--cuda_device`: CUDA device id(s). Single id (e.g., '0') or comma-separated list '0,1' for multi-GPU805 806**Debugging:**807- `--debug`: Enable verbose debug logging808 809### Multi-GPU Processing Explained810 811The CLI's multi-GPU mode automatically distributes the workload across multiple GPUs with intelligent temporal overlap handling:812 813**How it works:**8141. Video is split into chunks, one per GPU8152. Each GPU processes its chunk independently8163. Chunks overlap by `--temporal_overlap` frames8174. Results are blended together seamlessly using the overlap region818 819**Example for 2 GPUs with temporal_overlap=4:**820```821GPU 0: Frames 0-50 (includes 4 overlap frames at end)822GPU 1: Frames 46-100 (includes 4 overlap frames at beginning)823Result: Frames 0-100 with smooth transition at frame 48824```825 826**Best practices:**827- Set `--temporal_overlap` to 2-8 frames for smooth blending828- Higher overlap = smoother transitions but more redundant processing829- Use `--prepend_frames` to reduce artifacts at video start830- batch_size should divide evenly into chunk sizes for best results831 832## ⚠️ Limitations833 834### Model Limitations835 836**Batch Size Constraint**: The model requires batch_size to follow the **4n+1 formula** (1, 5, 9, 13, 17, 21, 25, ...) due to temporal consistency architecture. All frames in a batch are processed together for temporal coherence, then batches can be blended using temporal_overlap. Ideally, set batch_size to match your shot length for optimal quality.837 838### Performance Considerations839 840**VAE Bottleneck**: Even with optimized DiT upscaling (BlockSwap, GGUF, torch.compile), the VAE encoding/decoding stages can be the bottleneck, especially for high resolutions. The VAE is slow. Use large batch_size to mitigate this.841 842**VRAM Usage**: While the integration now supports low VRAM systems (8GB or less with proper optimization), VRAM usage varies based on:843- Input/output resolution (larger = more VRAM)844- Batch size (higher = more VRAM but better temporal consistency and speed)845- Model choice (FP16 > FP8 > GGUF in VRAM usage)846- Optimization settings (BlockSwap, VAE tiling significantly reduce VRAM)847 848**Speed**: Processing speed depends on:849- GPU capabilities (compute performance, VRAM bandwidth, and architecture generation)850- Model size (3B faster than 7B)851- Batch size (larger batch sizes are faster per frame due to better GPU utilization)852- Optimization settings (torch.compile provides significant speedup)853- Resolution (higher resolutions are slower)854 855### Best Practices856 8571. **Start with debug enabled** to understand where VRAM is being used8582. **For OOM errors during encoding**: Enable VAE encode tiling and reduce tile size8593. **For OOM errors during upscaling**: Enable BlockSwap and increase blocks_to_swap8604. **For OOM errors during decoding**: Enable VAE decode tiling and reduce tile size861 - **If still getting OOM after trying all above**: Reduce batch_size or resolution8625. **For best quality**: Use higher batch_size matching your shot length, FP16 models, and LAB color correction8636. **For speed**: Use FP8/GGUF models, enable torch.compile, and use Flash Attention if available8647. **Test settings with a short clip first** before processing long videos865 866## 🤝 Contributing867 868Contributions are welcome! We value community input and improvements.869 870For detailed contribution guidelines, see [CONTRIBUTING.md](CONTRIBUTING.md).871 872**Quick Start:**873 8741. Fork the repository8752. Create your feature branch (`git checkout -b feature/AmazingFeature`)8763. Commit your changes (`git commit -m 'Add some AmazingFeature'`)8774. Push to the branch (`git push origin feature/AmazingFeature`)8785. Open a Pull Request to **main** branch for stable features or **nightly** branch for experimental features879 880**Get Help:**881- YouTube: [AInVFX Channel](https://www.youtube.com/@AInVFX)882- GitHub [Issues](https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler/issues): For bug reports and feature requests883- GitHub [Discussions](https://github.com/numz/ComfyUI-SeedVR2_VideoUpscaler/discussions): For questions and community support884- Discord: adrientoupet & NumZ#7184885 886## 🙏 Credits887 888This ComfyUI implementation is a collaborative project by **[NumZ](https://github.com/numz)** and **[AInVFX](https://www.youtube.com/@AInVFX)** (Adrien Toupet), based on the original [SeedVR2](https://github.com/ByteDance-Seed/SeedVR) by ByteDance Seed Team.889 890Special thanks to our community contributors including [benjaminherb](https://github.com/benjaminherb), [cmeka](https://github.com/cmeka), [FurkanGozukara](https://github.com/FurkanGozukara), [JohnAlcatraz](https://github.com/JohnAlcatraz), [lihaoyun6](https://github.com/lihaoyun6), [Luchuanzhao](https://github.com/Luchuanzhao), [Luke2642](https://github.com/Luke2642), [naxci1](https://github.com/naxci1), [q5sys](https://github.com/q5sys), and many others for their improvements, bug fixes, and testing.891 892## 📜 License893 894The code in this repository is released under the MIT license as found in the [LICENSE](LICENSE) file.