Rob1221rib/wan22-qx-encoders-gguf

<!-- README Version: v1.1 -->

WAN 2.2 QX Text Encoders (GGUF)

Quantized text encoder models for WAN (World Animated Network) 2.2 QX video generation system. These GGUF-format encoders provide efficient text-to-embedding conversion for text-to-video and image-to-video generation workflows with significantly reduced VRAM requirements.

Model Description

This repository contains 8 quantized variants of the UMT5-XXL text encoder, optimized for WAN 2.2 QX video generation pipelines. The GGUF format enables efficient inference with reduced memory footprint while maintaining high-quality text understanding for video generation prompts.

Key Features:

• —Multiple Precision Levels: Q3KS to Q8_0 quantization options
• —VRAM Optimization: 30-70% memory reduction compared to FP16
• —Quality vs Size Trade-offs: Choose optimal balance for your hardware
• —Direct Integration: Compatible with WAN 2.2 QX diffusers pipeline
• —UMT5-XXL Architecture: Advanced multilingual text understanding

Repository Contents

Text Encoders (text_encoders/)

Total Repository Size: ~30.2 GB

Hardware Requirements

Minimum Requirements

• —VRAM: 4 GB (Q3KS quantization)
• —RAM: 8 GB system memory
• —Disk Space: 3 GB (single encoder variant)
• —GPU: NVIDIA RTX 2060 or equivalent (CUDA support recommended)

Recommended Requirements

• —VRAM: 8+ GB (Q4KM or higher quantization)
• —RAM: 16 GB system memory
• —Disk Space: 10 GB (multiple variants for testing)
• —GPU: NVIDIA RTX 3060 Ti or better

Optimal Requirements

• —VRAM: 12+ GB (Q6K or Q80 quantization)
• —RAM: 32 GB system memory
• —Disk Space: 30 GB (full repository)
• —GPU: NVIDIA RTX 4070 Ti or better

Usage Examples

Basic Text Encoder Loading

from diffusers import DiffusionPipeline
import torch

# Load WAN 2.2 QX pipeline with quantized text encoder
pipe = DiffusionPipeline.from_pretrained(
    "path/to/wan-2.2-qx",
    text_encoder_path="E:/huggingface/wan22-qx-encoders-gguf/text_encoders/umt5-xxl-encoder-q4-k-m.gguf",
    torch_dtype=torch.float16,
    variant="fp16"
)

# Move pipeline to GPU
pipe = pipe.to("cuda")

# Generate video from text
prompt = "A serene mountain landscape at sunset with flowing clouds"
video_frames = pipe(
    prompt=prompt,
    num_frames=24,
    num_inference_steps=30
).frames

# Save video
save_video(video_frames, "output.mp4", fps=8)

Memory-Optimized Configuration (Low VRAM)

from diffusers import DiffusionPipeline
import torch

# Use Q3_K_S encoder for minimum VRAM usage
pipe = DiffusionPipeline.from_pretrained(
    "path/to/wan-2.2-qx",
    text_encoder_path="E:/huggingface/wan22-qx-encoders-gguf/text_encoders/umt5-xxl-encoder-q3-k-s.gguf",
    torch_dtype=torch.float16
)

# Enable memory optimizations
pipe.enable_attention_slicing()
pipe.enable_vae_slicing()
pipe = pipe.to("cuda")

# Generate with lower resolution
video_frames = pipe(
    prompt="A cat playing with a ball of yarn",
    height=512,
    width=512,
    num_frames=16,
    num_inference_steps=25
).frames

Quality-Optimized Configuration (High VRAM)

from diffusers import DiffusionPipeline
import torch

# Use Q8_0 encoder for maximum quality
pipe = DiffusionPipeline.from_pretrained(
    "path/to/wan-2.2-qx",
    text_encoder_path="E:/huggingface/wan22-qx-encoders-gguf/text_encoders/umt5-xxl-encoder-q8-0.gguf",
    torch_dtype=torch.float16
)

pipe = pipe.to("cuda")

# Generate high-quality video
video_frames = pipe(
    prompt="Cinematic shot of a futuristic cityscape with flying vehicles",
    height=1024,
    width=1024,
    num_frames=48,
    num_inference_steps=50,
    guidance_scale=7.5
).frames

Batch Processing with Different Encoders

import torch
from diffusers import DiffusionPipeline

# Test different quantization levels
encoders = {
    "q3_k_m": "E:/huggingface/wan22-qx-encoders-gguf/text_encoders/umt5-xxl-encoder-q3-k-m.gguf",
    "q4_k_m": "E:/huggingface/wan22-qx-encoders-gguf/text_encoders/umt5-xxl-encoder-q4-k-m.gguf",
    "q5_k_m": "E:/huggingface/wan22-qx-encoders-gguf/text_encoders/umt5-xxl-encoder-q5-k-m.gguf",
}

prompt = "A beautiful garden with blooming flowers in spring"

for name, encoder_path in encoders.items():
    pipe = DiffusionPipeline.from_pretrained(
        "path/to/wan-2.2-qx",
        text_encoder_path=encoder_path,
        torch_dtype=torch.float16
    ).to("cuda")

    video = pipe(prompt=prompt, num_frames=24).frames
    save_video(video, f"output_{name}.mp4", fps=8)

    # Clear VRAM
    del pipe
    torch.cuda.empty_cache()

Model Specifications

Architecture

• —Base Model: UMT5-XXL (Unified Multilingual T5)
• —Parameters: ~13 billion (unquantized)
• —Context Length: 512 tokens
• —Vocabulary Size: 250,000+ tokens
• —Language Support: Multilingual (100+ languages)

Quantization Details

Format

• —File Format: GGUF (GPT-Generated Unified Format)
• —Precision: Mixed precision quantization (K-quant variants)
• —Compression: Lossless GGUF compression
• —Compatibility: llama.cpp ecosystem, diffusers integration

Performance Tips

Quantization Selection Guide

• —8GB VRAM or less: Use Q3KM or Q4KS
• —12GB VRAM: Use Q4KM or Q5KS (recommended)
• —16GB VRAM: Use Q5KM or Q6_K
• —24GB+ VRAM: Use Q8_0 for maximum quality

Optimization Strategies

1. 1.Memory Optimization:
2. 2.Enable enable_attention_slicing() for low VRAM
3. 3.Use enable_vae_slicing() for large videos
4. 4.Reduce num_frames and resolution for faster generation

1. 1.Quality Optimization:
2. 2.Q4KM offers best quality/performance balance
3. 3.Q6K or Q80 for production-quality outputs
4. 4.Higher num_inference_steps improves coherence

1. 1.Speed Optimization:
2. 2.Lower quantization levels (Q3_K) are slightly faster
3. 3.Reduce inference steps for draft iterations
4. 4.Use torch.compile() for additional speedup (PyTorch 2.0+)

Benchmark Performance (RTX 4090, 24GB VRAM)

License

This model repository uses a custom license. Please review the WAN model license terms before use.

License Type: other (WAN License) Commercial Use: Check WAN license terms Attribution: Required for derivative works

Citation

If you use these models in your research or projects, please cite:

@misc{wan22-qx-encoders-gguf,
  title={WAN 2.2 QX Text Encoders (GGUF)},
  author={WAN Team},
  year={2024},
  howpublished={\url{https://huggingface.co/wan22-qx-encoders-gguf}},
  note={Quantized UMT5-XXL text encoders for video generation}
}

Related Resources

• —WAN Official Documentation: WAN Docs
• —Diffusers Library: https://github.com/huggingface/diffusers
• —GGUF Format: https://github.com/ggerganov/llama.cpp
• —UMT5 Paper: Unified Multilingual T5

Technical Support

For issues, questions, or feature requests:

• —GitHub Issues: Report issues
• —Hugging Face Discussions: Community support
• —Documentation: WAN User Guide

Model Card Metadata

• —Developed by: WAN Team
• —Model type: Text Encoder (Quantized)
• —Language(s): Multilingual (100+ languages)
• —License: Custom WAN License
• —Finetuned from: UMT5-XXL
• —Model Format: GGUF (quantized)
• —Precision Variants: Q3KS, Q3KM, Q4KS, Q4KM, Q5KS, Q5KM, Q6K, Q80