Apppublic
declare-lab/tango2

sourceHugging Faceupdated 2y agoView on Hugging Face
92likes
rootdiffuserssrcdiffusersmodels__pycache__resnet.cpython-39.pyc
resnet.cpython-39.pyc318 linesDownload Raw Back to __pycache__
1a

2��'d��@s`ddlmZddlmZddlZddlmZddlmmZ	ddl3mZGdd�dej�Z
Gdd	�d	ej�ZGd4d�dej�ZGdd
�d
ej�ZGdd�dej�ZGdd�dej�ZGdd�dej�ZGdd�dej�ZGdd�dej�ZGdd�dejj�Zdd�ZGdd�dej�ZGdd�dej�Zd*d!d"�Zd+d#d$�Zd,d&d'�ZGd(d)�d)ej�ZdS)-�)�partial)�OptionalN�)�AdaGroupNormcs*eZdZdZd	�fdd�	Zdd�Z�ZS)5�6Upsample1Da7    An upsampling layer with an optional convolution.8 9    Parameters:10            channels: channels in the inputs and outputs.11            use_conv: a bool determining if a convolution is applied.12            use_conv_transpose:13            out_channels:14    FN�convcsnt���||_|p||_||_||_||_d|_|rNt�	||jddd�|_n|rjtj15|j|jddd�|_dS)N��r���padding)�super�__init__�channels�out_channels�use_conv�use_conv_transpose�namer�nn�ConvTranspose1d�Conv1d)�selfrrrrr��	__class__��L/home/deep/Projects/audio_diffusion/diffusers/src/diffusers/models/resnet.pyr%s1617zUpsample1D.__init__cCsH|jd|jksJ�|jr$|�|�Stj|ddd�}|jrD|�|�}|S)Nr�@�nearest��scale_factor�mode)�shaperrr�F�interpolater�r�xrrr�forward3s1819zUpsample1D.forward)FFNr��__name__�20__module__�__qualname__�__doc__rr&�
__classcell__rrrrrs21rcs*eZdZdZd22�fdd�	Zdd	�Z�ZS)�Downsample1D��23    A downsampling layer with an optional convolution.24 25    Parameters:26        channels: channels in the inputs and outputs.27        use_conv: a bool determining if a convolution is applied.28        out_channels:29        padding:30    FNrrcstt���||_|p||_||_||_d}||_|rPtj|j|jd||d�|_	n |j|jks`J�tj31||d�|_	dS)Nr	r32��strider��kernel_sizer0)r
rrrrrrrrr�	AvgPool1d)rrrrrrr0rrrrKs3334zDownsample1D.__init__cCs|jd|jksJ�|�|�S�Nr)r!rrr$rrrr&ZszDownsample1D.forward)FNrrr'rrrrr-@s35r-cs,eZdZdZd	�fdd�	Zd36dd�Z�ZS)�37Upsample2Dz�38    An upsampling layer with an optional convolution.39 40    Parameters:41        channels: channels in the inputs and outputs.42        use_conv: a bool determining if a convolution is applied.43        use_conv_transpose:44        out_channels:45    FNrcs~t���||_|p||_||_||_||_d}|rJt�||jddd�}n|rdtj	|j|jddd�}|dkrt||_46n||_dS)Nrr	rr47rr)r
rrrrrrr�ConvTranspose2d�Conv2dr�Conv2d_0)rrrrrrrrrrrjs4849zUpsample2D.__init__cCs�|jd|jksJ�|jr$|�|�S|j}|tjkr@|�tj�}|jddkrV|�	�}|durpt50j|ddd�}nt51j||dd�}|tjkr�|�|�}|jr�|j
dkr�|�|�}n52|�|�}|S)	Nrr�@rrr)�sizer r)r!rrr�dtype�torch�bfloat16�to�float32�53contiguousr"r#rrr8)r�
hidden_states�output_sizer;rrrr&~s$545556575859zUpsample2D.forward)FFNr)Nr'rrrrr5_s60r5cs*eZdZdZd61�fdd�	Zdd	�Z�ZS)�Downsample2Dr.FNrrcs�t���||_|p||_||_||_d}||_|rNtj|j|jd||d�}n|j|jks^J�tj	||d�}|dkr�||_62||_n|dkr�||_n||_dS)Nr	r63r/r1rr8)r
rrrrrrrr7�	AvgPool2dr8r)rrrrrrr0rrrrr�s"6465zDownsample2D.__init__cCs\|jd|jksJ�|jr:|jdkr:d}tj||ddd�}|jd|jksNJ�|�|�}|S)Nrr)rrrr�constant)r �value)r!rrrr"�padr)rrArGrrrr&�s66zDownsample2D.forward)FNrrr'rrrrrC�s67rCcs0eZdZd�fdd�	Zd
dd	�Zd68d�Z�ZS)�
FirUpsample2DNF�rr69r70rcsFt���|r|n|}|r0tj||dddd�|_||_||_||_dS�Nr71r�r2r0r)r
rrr7r8r�72fir_kernelr�rrrrrLrrrr�s73zFirUpsample2D.__init__r	rcCs.t|t�r|dksJ�|dur(dg|}tj|tjd�}|jdkrNt�||�}|t�|�}|||d}|j�r�|j	d}|j	d}|j	d}|j	d||d}	||f}74|j	dd|||j	dd||f}|d|j	dd|75d||d|j	dd|76d|f}|ddk�r<|ddk�s@J�|j	d|}
t�77||
d|||f�}tj|ddgd	��ddddd�}t�78||
|d||f�}t
j|||79|dd80�}t|tj||jd�|	dd|d|	ddfd�}nB|j	d|}	t|tj||jd�||	dd|d|	dfd
�}|S)aCFused `upsample_2d()` followed by `Conv2d()`.81 82        Padding is performed only once at the beginning, not between the operations. The fused op is considerably more83        efficient than performing the same calculation using standard TensorFlow ops. It supports gradients of84        arbitrary order.85 86        Args:87            hidden_states: Input tensor of the shape `[N, C, H, W]` or `[N, H, W, C]`.88            weight: Weight tensor of the shape `[filterH, filterW, inChannels,89                outChannels]`. Grouped convolution can be performed by `inChannels = x.shape[0] // numGroups`.90            kernel: FIR filter of the shape `[firH, firW]` or `[firN]`91                (separable). The default is `[1] * factor`, which corresponds to nearest-neighbor upsampling.92            factor: Integer upsampling factor (default: 2).93            gain: Scaling factor for signal magnitude (default: 1.0).94 95        Returns:96            output: Tensor of the shape `[N, C, H * factor, W * factor]` or `[N, H * factor, W * factor, C]`, and same97            datatype as `hidden_states`.98        rN�r;r	r99r�����r)�dims)r0�output_paddingr��device�rG��uprG)�100isinstance�intr<�tensorr?�ndim�outer�sumrr!�reshape�flip�permuter"�conv_transpose2d�upfirdn2d_nativerS)rrA�weight�kernel�factor�gain�convH�convWZinC�	pad_valuer0�output_shaperQ�101num_groupsZinverse_conv�outputrrr�_upsample_2d�sR102103104105106�  �  107���zFirUpsample2D._upsample_2dcCsL|jr6|j||jj|jd�}||jj�dddd�}n|j||jdd�}|S)N�rcrrOr	�rcrd)rrlr8rbrL�biasr])rrA�heightrrrr&-s108zFirUpsample2D.forward)NNFrI)NNr	r)r(r)r*rrlr&r,rrrrrH�s	109PrHcs0eZdZd�fdd�	Zd
dd	�Zd110d�Z�ZS)�FirDownsample2DNFrIcsFt���|r|n|}|r0tj||dddd�|_||_||_||_dSrJ)r
rrr7r8rLrrrMrrrr8s111zFirDownsample2D.__init__r	rc
Cst|t�r|dksJ�|dur(dg|}tj|tjd�}|jdkrNt�||�}|t�|�}||}|jr�|j	\}}}}|j	d||d}	||g}112t113|tj||jd�|	dd|	dfd�}tj
|||114dd�}n:|j	d|}	t115|tj||jd�||	dd|	dfd	�}|S)116a>Fused `Conv2d()` followed by `downsample_2d()`.117        Padding is performed only once at the beginning, not between the operations. The fused op is considerably more118        efficient than performing the same calculation using standard TensorFlow ops. It supports gradients of119        arbitrary order.120 121        Args:122            hidden_states: Input tensor of the shape `[N, C, H, W]` or `[N, H, W, C]`.123            weight:124                Weight tensor of the shape `[filterH, filterW, inChannels, outChannels]`. Grouped convolution can be125                performed by `inChannels = x.shape[0] // numGroups`.126            kernel: FIR filter of the shape `[firH, firW]` or `[firN]` (separable). The default is `[1] *127            factor`, which corresponds to average pooling.128            factor: Integer downsampling factor (default: 2).129            gain: Scaling factor for signal magnitude (default: 1.0).130 131        Returns:132            output: Tensor of the shape `[N, C, H // factor, W // factor]` or `[N, H // factor, W // factor, C]`, and133            same datatype as `x`.134        rNrNrrRr	rTr/��downrG)rWrXr<rYr?rZr[r\rr!rarSr"�conv2d)
rrArbrcrdre�_rfrgrhZstride_valueZ
upfirdn_inputrkrrr�_downsample_2dAs4135136��zFirDownsample2D._downsample_2dcCsL|jr6|j||jj|jd�}||jj�dddd�}n|j||jdd�}|S)N)rbrcrrOr	rn)rrvr8rbrLror])rrAZdownsample_inputrrrr&ws137zFirDownsample2D.forward)NNFrI)NNr	r)r(r)r*rrvr&r,rrrrrq7s	1386rqcs&eZdZd�fdd�	Zdd�Z�ZS)�
KDownsample2D�reflectcsNt���||_t�gd�g�}|jddd|_|jd|j|dd�dS)N���?��?r{rzrr	rcF��139persistent�	r
r�pad_moder<rYr!rG�register_buffer�T�rrZ	kernel_1drrrr�s140141zKDownsample2D.__init__cCs�t�||jfd|j�}|�|jd|jd|jjd|jjdg�}tj|jd|jd�}|j�	|�|||f<tj142||dd�S)NrrrrRr	)r0)r"rGr�	new_zerosr!rcr<�arangerSr>rt�rr%rb�indicesrrrr&�s143.zKDownsample2D.forward)rx�r(r)r*rr&r,rrrrrw�srwcs&eZdZd�fdd�	Zdd�Z�ZS)�KUpsample2DrxcsRt���||_t�gd�g�d}|jddd|_|jd|j|dd�dS)Nryr	rrcFr|r~r�rrrr�s144145zKUpsample2D.__init__cCs�t�||jddfd|j�}|�|jd|jd|jjd|jjdg�}tj|jd|jd�}|j�	|�|||f<tj146||d|jddd�S)Nrr	rrrRr/)r"rGrr�r!rcr<r�rSr>r`r�rrrr&�s147 .zKUpsample2D.forward)rxr�rrrrr��sr�csZeZdZdZddddddddd	d148dddddddd�eeed
��fdd�Zdd�Z�Z	S)�
ResnetBlock2DaM149    A Resnet block.150 151    Parameters:152        in_channels (`int`): The number of channels in the input.153        out_channels (`int`, *optional*, default to be `None`):154            The number of output channels for the first conv2d layer. If None, same as `in_channels`.155        dropout (`float`, *optional*, defaults to `0.0`): The dropout probability to use.156        temb_channels (`int`, *optional*, default to `512`): the number of channels in timestep embedding.157        groups (`int`, *optional*, default to `32`): The number of groups to use for the first normalization layer.158        groups_out (`int`, *optional*, default to None):159            The number of groups to use for the second normalization layer. if set to None, same as `groups`.160        eps (`float`, *optional*, defaults to `1e-6`): The epsilon to use for the normalization.161        non_linearity (`str`, *optional*, default to `"swish"`): the activation function to use.162        time_embedding_norm (`str`, *optional*, default to `"default"` ): Time scale shift config.163            By default, apply timestep embedding conditioning with a simple shift mechanism. Choose "scale_shift" or164            "ada_group" for a stronger conditioning with scale and shift.165        kernel (`torch.FloatTensor`, optional, default to None): FIR filter, see166            [`~models.resnet.FirUpsample2D`] and [`~models.resnet.FirDownsample2D`].167        output_scale_factor (`float`, *optional*, default to be `1.0`): the scale factor to use for the output.168        use_in_shortcut (`bool`, *optional*, default to `True`):169            If `True`, add a 1x1 nn.conv2d layer for skip-connection.170        up (`bool`, *optional*, default to `False`): If `True`, add an upsample layer.171        down (`bool`, *optional*, default to `False`): If `True`, add a downsample layer.172        conv_shortcut_bias (`bool`, *optional*, default to `True`):  If `True`, adds a learnable bias to the173            `conv_shortcut` output.174        conv_2d_out_channels (`int`, *optional*, default to `None`): the number of channels in the output.175            If None, same as `out_channels`.176    NF�i� Tg�����ư>�swish�defaultg�?)r�
conv_shortcut�dropout�
temb_channels�groups�177groups_out�pre_norm�eps�
non_linearity�time_embedding_normrc�output_scale_factor�use_in_shortcutrVrs�conv_shortcut_bias�conv_2d_out_channels)r�r�cs�t���||_d|_||_|dur(|n|}||_||_||_||_|
|_||_	|dur\|}|j	dkrzt178||||	d�|_ntj
j|||	dd�|_tj
j||dddd�|_|du�r|j	dkr�tj
�||�|_nF|j	d	kr�tj
�|d179|�|_n&|j	dk�rd|_ntd|j	�d���nd|_|j	dk�r<t180||||	d�|_ntj
j|||	dd�|_tj
�|�|_|�ph|}tj
j||dddd�|_|181d
k�r�dd�|_n@|182dk�r�t
��|_n*|183dk�r�t
��|_n|184dk�r�t
��|_d|_|_|j�r8|dk�r185d��fdd�|_n,|dk�r(ttj ddd�|_nt!|dd�|_nV|j�r�|dk�r^d��fdd�|_n0|dk�r|ttj"d186d187d�|_nt#|dddd�|_|du�r�|j|kn||_$d|_%|j$�r�tj
j||ddd |d!�|_%dS)"NT�	ada_group)r�)rj�num_channelsr��affiner188rrKr��scale_shiftr	zunknown time_embedding_norm : � r�cSs189t�|�S�N)r"�silu�r%rrr�<lambda>�z(ResnetBlock2D.__init__.<locals>.<lambda>�mishr��gelu�firrIcst|�d�S�Nrm)�upsample_2dr��rLrrr�r�Zsde_vprrrF)rcst|�d�Sr�)�
downsample_2dr�r�rrr�r�r1�op)rrrr)r2r0rro)&r
rr��in_channelsrZuse_conv_shortcutrVrsr�r�r�norm1r<r�	GroupNormr7�conv1�Linear�
time_emb_proj�190ValueError�norm2�Dropoutr��conv2�nonlinearity�Mish�SiLU�GELU�upsample�191downsamplerr"r#r5�192avg_pool2drCr�r�)rr�rr�r�r�r�r�r�r�r�r�rcr�r�rVrsr�r�rr�rr�sz193194195196197198199200201202203204205206207�zResnetBlock2D.__init__cCs�|}|jdkr|�||�}n208|�|�}|�|�}|jdurn|jddkrX|��}|��}|�|�}|�|�}n|jdur�|�|�}|�|�}|�|�}|jdur�|�|�|��dd�dd�ddf}|dur�|jdkr�||}|jdkr�|�	||�}n209|�	|�}|du�r:|jdk�r:t210j|ddd�\}}|d||}|�|�}|�|�}|�
|�}|jdu�rn|�|�}|||j}|S)	Nr�rr9r�r�r	r)�dim)r�r�r�r�r!r@r�r�r�r�r<�chunkr�r�r�r�)r�input_tensor�tembrA�scale�shift�
output_tensorrrrr&%s@211212213214215216217218219220$221222223224225226zResnetBlock2D.forward)227r(r)r*r+�boolrrXrr&r,rrrrr��s."��dr�c@seZdZdd�ZdS)r�cCs|t�tjj�|��Sr�)r<�tanhr�228functional�softplus)rrArrrr&YszMish.forwardN)r(r)r*r&rrrrr�Xsr�cCs�t|j�dkr$|dd�dd�dfSt|j�dkrN|dd�dd�ddd�fSt|j�dkrx|dd�dd�ddd�fStdt|��d���dS)Nr	r229rrz`len(tensor)`: z has to be 2, 3 or 4.)�lenr!r�)rYrrr�rearrange_dims^sr�cs*eZdZdZd�fdd�	Zdd�Z�ZS)�Conv1dBlockz'230    Conv1d --> GroupNorm --> Mish231    �cs>t���tj||||dd�|_t�||�|_t��|_dS)Nr	r)	r
rrr�conv1dr��232group_normr�r�)r�inp_channelsrr2Zn_groupsrrrrns233zConv1dBlock.__init__cCs2|�|�}t|�}|�|�}t|�}|�|�}|Sr�)r�r�r�r�r$rrrr&us234235236zConv1dBlock.forward)r�r'rrrrr�isr�cs&eZdZd�fdd�	Zdd�Z�ZS)�ResidualTemporalBlock1D�csbt���t|||�|_t|||�|_t��|_t�||�|_	||krTt�237||d�nt��|_dSr4)
r
rr��conv_in�conv_outrr��time_emb_actr��time_embr�Identity�
residual_conv)rr�r�	embed_dimr2rrrr�s238239�z ResidualTemporalBlock1D.__init__cCs>|�|�}|�|�}|�|�t|�}|�|�}||�|�S)z�240        Args:241            x : [ batch_size x inp_channels x horizon ]242            t : [ batch_size x embed_dim ]243 244        returns:245            out : [ batch_size x out_channels x horizon ]246        )r�r�r�r�r�r�)rr%�t�outrrrr&�s247	248249250zResidualTemporalBlock1D.forward)r�r�rrrrr�sr�r	cCs�t|t�r|dksJ�|dur(dg|}tj|tjd�}|jdkrNt�||�}|t�|�}|||d}|jd|}t	||j251|jd�||dd|d|dfd�}|S)a�Upsample2D a batch of 2D images with the given filter.252    Accepts a batch of 2D images of the shape `[N, C, H, W]` or `[N, H, W, C]` and upsamples each image with the given253    filter. The filter is normalized so that if the input pixels are constant, they will be scaled by the specified254    `gain`. Pixels outside the image are assumed to be zero, and the filter is padded with zeros so that its shape is255    a: multiple of the upsampling factor.256 257    Args:258        hidden_states: Input tensor of the shape `[N, C, H, W]` or `[N, H, W, C]`.259        kernel: FIR filter of the shape `[firH, firW]` or `[firN]`260          (separable). The default is `[1] * factor`, which corresponds to nearest-neighbor upsampling.261        factor: Integer upsampling factor (default: 2).262        gain: Scaling factor for signal magnitude (default: 1.0).263 264    Returns:265        output: Tensor of the shape `[N, C, H * factor, W * factor]`266    rNrNr	rrRrU�rWrXr<rYr?rZr[r\r!rar>rS�rArcrdrerhrkrrrr��s 267268�r�cCs�t|t�r|dksJ�|dur(dg|}tj|tjd�}|jdkrNt�||�}|t�|�}||}|jd|}t	||j269|jd�||dd|dfd�}|S)a�Downsample2D a batch of 2D images with the given filter.270    Accepts a batch of 2D images of the shape `[N, C, H, W]` or `[N, H, W, C]` and downsamples each image with the271    given filter. The filter is normalized so that if the input pixels are constant, they will be scaled by the272    specified `gain`. Pixels outside the image are assumed to be zero, and the filter is padded with zeros so that its273    shape is a multiple of the downsampling factor.274 275    Args:276        hidden_states: Input tensor of the shape `[N, C, H, W]` or `[N, H, W, C]`.277        kernel: FIR filter of the shape `[firH, firW]` or `[firN]`278          (separable). The default is `[1] * factor`, which corresponds to average pooling.279        factor: Integer downsampling factor (default: 2).280        gain: Scaling factor for signal magnitude (default: 1.0).281 282    Returns:283        output: Tensor of the shape `[N, C, H // factor, W // factor]`284    rNrNrrRr	rrr�r�rrrr��s285286"�r��rrcCsT|}}|}}|d}	}287|d}}|j\}
}}}|�d||d�}|j\}
}}}|j\}}|�d|d|d|�}t�|ddd|dddd|dg�}|�d|||||�}t�|ddt|	d�t|d�t|288d�t|d�g�}|�|j�}|dd�t|289d�|jdt|d��t|	d�|jdt|d��dd�f}|�dddd�}|�dd|||290||||	|g�}t	�291|ddg��dd||�}t�||�}|�d||||292||d|||	||d�}|�dddd�}|dd�dd|�dd|�dd�f}|||293|||d}|||	|||d}|�d|||�S)NrrrOr	r294)r!r]�viewr"rG�maxr>rSr_r<r^rt)rYrcrVrsrGZup_xZup_yZdown_xZdown_yZpad_x0Zpad_y0Zpad_x1Zpad_y1ru�channel�in_h�in_w�minor�kernel_h�kernel_wr��w�out_h�out_wrrrra�sF295$0  ��*�$racs,eZdZdZd	�fdd�	Zd296dd�Z�ZS)�TemporalConvLayera297    Temporal convolutional layer that can be used for video (sequence of images) input Code mostly copied from:298    https://github.com/modelscope/modelscope/blob/1509fdb973e5871f37148a4b5e5964cafd43e64d/modelscope/models/multi_modal/video_synthesis/unet_sd.py#L1016299    Nr�cs300t���|p|}||_||_t�t�d|�t��tj||ddd��|_	t�t�d|�t��t�301|�tj||ddd��|_t�t�d|�t��t�302|�tj||ddd��|_t�t�d|�t��t�303|�tj||ddd��|_
tj�|j
dj�tj�|j
dj�dS)Nr�)r304rr)rrrrrO)r
r�in_dim�out_dimr�305Sequentialr�r��Conv3dr�r�r��conv3�conv4�init�zeros_rbro)rr�r�r�rrrrs6306 �307�308�309�zTemporalConvLayer.__init__rcCs�|ddd�f�d|f|jdd���ddddd�}|}|�|�}|�|�}|�|�}|�|�}||}|�ddddd��|jd|jddf|jdd��}|S)NrOrrr	r310r)r]r!r_r�r�r�r�)rrAZ311num_frames�identityrrrr&7s4�312313314315$�zTemporalConvLayer.forward)Nr�)rr'rrrrr�s!r�)Nr	r)Nr	r)rrr�)�	functoolsr�typingrr<�torch.nnrZtorch.nn.functionalr�r"�	attentionr�Modulerr-r5rCrHrqrwr�r�r�r�r�r�r�r�rar�rrrr�<module>s.&E/dK7316%317#318,