diffusers/Qwen-Image-Layered-modular

1---2library_name: diffusers3tags:4- modular-diffusers5- diffusers6- qwenimage-layered7- text-to-image8- modular-diffusers9- diffusers10- qwenimage-layered11- text-to-image12---13This is a modular diffusion pipeline built with 🧨 Diffusers' modular pipeline framework.14 15**Pipeline Type**: QwenImageLayeredAutoBlocks16 17**Description**: Auto Modular pipeline for layered denoising tasks using QwenImage-Layered.18 19This pipeline uses a 4-block architecture that can be customized and extended.20 21## Example Usage22 23[TODO]24 25## Pipeline Architecture26 27This modular pipeline is composed of the following blocks:28 291. **text_encoder** (`QwenImageLayeredTextEncoderStep`)30   - QwenImage-Layered Text encoder step that encode the text prompt, will generate a prompt based on image if not provided.312. **vae_encoder** (`QwenImageLayeredVaeEncoderStep`)32   - Vae encoder step that encode the image inputs into their latent representations.333. **denoise** (`QwenImageLayeredCoreDenoiseStep`)34   - Core denoising workflow for QwenImage-Layered img2img task.354. **decode** (`QwenImageLayeredDecoderStep`)36   - Decode unpacked latents (B, C, layers+1, H, W) into layer images. 37 38## Model Components39 401. image_resize_processor (`VaeImageProcessor`)412. text_encoder (`Qwen2_5_VLForConditionalGeneration`)423. processor (`Qwen2VLProcessor`)434. tokenizer (`Qwen2Tokenizer`): The tokenizer to use445. guider (`ClassifierFreeGuidance`)456. image_processor (`VaeImageProcessor`)467. vae (`AutoencoderKLQwenImage`)478. pachifier (`QwenImageLayeredPachifier`)489. scheduler (`FlowMatchEulerDiscreteScheduler`)4910. transformer (`QwenImageTransformer2DModel`) 50 51## Input/Output Specification52 53**Inputs:**54 55- `image` (`Image | list`): Reference image(s) for denoising. Can be a single image or list of images.56- `resolution` (`int`, *optional*, defaults to `640`): The target area to resize the image to, can be 1024 or 64057- `prompt` (`str`, *optional*): The prompt or prompts to guide image generation.58- `use_en_prompt` (`bool`, *optional*, defaults to `False`): Whether to use English prompt template59- `negative_prompt` (`str`, *optional*): The prompt or prompts not to guide the image generation.60- `max_sequence_length` (`int`, *optional*, defaults to `1024`): Maximum sequence length for prompt encoding.61- `generator` (`Generator`, *optional*): Torch generator for deterministic generation.62- `num_images_per_prompt` (`int`, *optional*, defaults to `1`): The number of images to generate per prompt.63- `latents` (`Tensor`, *optional*): Pre-generated noisy latents for image generation.64- `layers` (`int`, *optional*, defaults to `4`): Number of layers to extract from the image65- `num_inference_steps` (`int`, *optional*, defaults to `50`): The number of denoising steps.66- `sigmas` (`list`, *optional*): Custom sigmas for the denoising process.67- `attention_kwargs` (`dict`, *optional*): Additional kwargs for attention processors.68- `**denoiser_input_fields` (`None`, *optional*): conditional model inputs for the denoiser: e.g. prompt_embeds, negative_prompt_embeds, etc.69- `output_type` (`str`, *optional*, defaults to `pil`): Output format: 'pil', 'np', 'pt'.70 71**Outputs:**72 73- `images` (`list`): Generated images.74