diffusers/Qwen-Image-Layered-modular

This is a modular diffusion pipeline built with 🧨 Diffusers' modular pipeline framework.

Pipeline Type: QwenImageLayeredAutoBlocks

Description: Auto Modular pipeline for layered denoising tasks using QwenImage-Layered.

This pipeline uses a 4-block architecture that can be customized and extended.

Example Usage

[TODO]

Pipeline Architecture

This modular pipeline is composed of the following blocks:

1. 1.text_encoder (QwenImageLayeredTextEncoderStep)
2. 2.QwenImage-Layered Text encoder step that encode the text prompt, will generate a prompt based on image if not provided.
3. 3.vae_encoder (QwenImageLayeredVaeEncoderStep)
4. 4.Vae encoder step that encode the image inputs into their latent representations.
5. 5.denoise (QwenImageLayeredCoreDenoiseStep)
6. 6.Core denoising workflow for QwenImage-Layered img2img task.
7. 7.decode (QwenImageLayeredDecoderStep)
8. 8.Decode unpacked latents (B, C, layers+1, H, W) into layer images.

Model Components

1. 1.imageresizeprocessor (VaeImageProcessor)
2. 2.textencoder (`Qwen25_VLForConditionalGeneration`)
3. 3.processor (Qwen2VLProcessor)
4. 4.tokenizer (Qwen2Tokenizer): The tokenizer to use
5. 5.guider (ClassifierFreeGuidance)
6. 6.image_processor (VaeImageProcessor)
7. 7.vae (AutoencoderKLQwenImage)
8. 8.pachifier (QwenImageLayeredPachifier)
9. 9.scheduler (FlowMatchEulerDiscreteScheduler)
10. 10.transformer (QwenImageTransformer2DModel)

Input/Output Specification

Inputs:

• —image (Image | list): Reference image(s) for denoising. Can be a single image or list of images.
• —resolution (int, optional, defaults to 640): The target area to resize the image to, can be 1024 or 640
• —prompt (str, optional): The prompt or prompts to guide image generation.
• —use_en_prompt (bool, optional, defaults to False): Whether to use English prompt template
• —negative_prompt (str, optional): The prompt or prompts not to guide the image generation.
• —max_sequence_length (int, optional, defaults to 1024): Maximum sequence length for prompt encoding.
• —generator (Generator, optional): Torch generator for deterministic generation.
• —num_images_per_prompt (int, optional, defaults to 1): The number of images to generate per prompt.
• —latents (Tensor, optional): Pre-generated noisy latents for image generation.
• —layers (int, optional, defaults to 4): Number of layers to extract from the image
• —num_inference_steps (int, optional, defaults to 50): The number of denoising steps.
• —sigmas (list, optional): Custom sigmas for the denoising process.
• —attention_kwargs (dict, optional): Additional kwargs for attention processors.
• —**denoiser_input_fields (None, optional): conditional model inputs for the denoiser: e.g. promptembeds, negativeprompt_embeds, etc.
• —output_type (str, optional, defaults to pil): Output format: 'pil', 'np', 'pt'.

Outputs:

• —images (list): Generated images.