xdecoder/Instruct-X-Decoder
163
1U
2 A8�cN � @ s� d dl Z d dlZd dlZd dlZd dlZd dlmZ d dl m3Z4 d dl m Z d dl
mZ d dlmZ ddlmZ dd lmZ e�d5� dd d �Zd
d� Zddd�Ze�� d dd��Zdd� Zdd� Z e�� dd� �Z dS ) � N)�Image)� OmegaConf)�repeat)�WatermarkEncoder)�Path� )� DDIMSampler)�instantiate_from_configFc C sN |d k rJt �t�| �t j�} |�| d�} t�| d d �d d �d d d�f �} | S )NZdwtDct�����)�cv2�cvtColor�np�array�
COLOR_RGB2BGR�encoder � fromarray)�img�6wm_encoder� r �)/data/arXiv/demo/Demo/utils/inpainting.py�
put_watermark s7 "r c C s` t �| �} t| j�}|jt�|�d dd� tj�� r@t�d�nt�d�}|� |�}t8|�}|S )N�9state_dictF)�strict�cuda�cpu) r �loadr �model�load_state_dict�torchr � is_available�device�tor )�config�ckptr r �samplerr r r �initialize_model s 1011���12r% c C s� t �| �d��} | d �dddd�} t�| �jtjd�d d } t �|�d ��}|�t j�d13 }|d }d||d k < d||d k< t�|�}| |d k }t | j|d
�d|d�||g t |j|d
�d|d�t |j|d
�d|d�d�}|S )N�RGBr � r � )�dtypeg �_@� �?�Lg �o@)NNg �?)r z1 ... -> n ...)�n)�image�txt�mask� masked_image)14r
r �convert� transposer �15from_numpyr! �float32�astyper )r- r/ r. r � num_samplesr0 �batchr r r �
make_batch_sd) s 16 �r8 � c17
s t j�� rt �d�nt �d�}18| j} td� d} t� � � �d| �d�� t j19� |�}
|
� |d| d |d �}t �
|�j|20t jd �}t �� ��\ t �d���D t||||21|d22�}| j�|d �}t� }| jD ]f}|| �� }|| jk�r|d| d |d g}t jjj||d d � d
�}n| �| � |��}|� |� q�t j |dd�}|g|gd�}| � |d�}|g|gd�}| j | d |d g}| j!||||dd|||d� \}}| �"|�}t j#|d d ddd�} | �$� �%� �&dddd�d } W 5 Q R X W 5 Q R X � fd d �| D �S ) Nr r z^Creating invisible watermark encoder (see https://github.com/ShieldMnt/invisible-watermark)...ZSDV2�byteszutf-8� � )r r) )r. r r6 r. �����)�sizer )�dim)Zc_concatZ c_crossattn� Fr* )�verbose�etaZ unconditional_guidance_scaleZunconditional_conditioningZx_Tg @g )�min�maxr r( r' �� c s$ g | ] }t t�|�tj��� ��qS r )r r r r5 r
�uint8)�.0r �r r r �23<listcomp>� s zinpaint.<locals>.<listcomp>)'r r r r r �printr Z
set_watermarkr r
�random� RandomState�randnr3 r! r4 �no_grad�autocastr8 Zcond_stage_model�listZ concat_keys�floatZmasked_image_key�nn�24functional� interpolateZget_first_stage_encodingZencode_first_stage�append�catZ get_unconditional_conditioning�channels�sampleZdecode_first_stage�clampr �numpyr2 ) r$ r- r/ �prompt�seed�scale�25ddim_stepsr6 �w�hr r �wmZprngZ26start_coder7 �cZc_catZck�ccZbchw�condZuc_crossZuc_full�shapeZ samples_cfg�
intermediatesZx_samples_ddim�resultr rH r �inpaintD st ��� 27 � � �28 � �29 30 �0rh c C sf t jdt �t �| j�d ��t�fdd�d | j \}}t�t j t �| �d|fd|fdfdd��}|S )N)r( r( �@ r )�axis)r r �edge)�mode)31r
rD �ceilr r> r5 �intr r �pad)� input_image�pad_w�pad_hZ im_paddedr r r � pad_image� s 32�����"�rs c C sH t �t �| j�d ��t�d \}}t�t �| �d |�d |�f �}|S )Nri )r
�floorr r> r5 rn r r )rp �crop_w�crop_h�33im_croppedr r r �34crop_image� s $ rx c C sb |d � d�}|d � d�}t|�} t|�}35| j\} } td| | � t| | |36|||||| | d�37}
|
S )a _summary_38 39 Args:40 input_image (_type_): dict41 - image: PIL.Image. Input image.42 - mask: PIL.Image. Mask image.43 prompt (_type_): string to be used as prompt. 44 ddim_steps (_type_): typical 4545 num_samples (_type_): typical 446 scale (_type_): typical 10.0 Guidance Scale.47 seed (_type_): typical 152916051948 49 r- r& r/ z
Inpainting...)50r$ r- r/ r[ r\ r] r^ r6 r` r_ )r1 rs r> rJ rh )r rp r[ r^ r6 r] r\ Z51init_imageZ init_maskr- r/ �width�heightrg r r r �predict� s& 52 � r{ )N)r )r r9 r9 ) �sysr r rZ r
�gradio�gr�PILr � omegaconfr �einopsr Z imwatermarkr �pathlibr Zddimr �utilr �set_grad_enabledr r% r8 rN rh rs rx r{ r r r r �<module> s, 5354 �55 =