declare-lab/tango2
92
1a
2 ��'d�\ � @ s� d dl Z d dl mZ ddlmZmZmZmZ ddlmZ ddl m3Z4 dd5d �Z dd d
�Z G dd� dej
�ZG dd� dej
�ZG dd� dej
�ZG dd� dej
�ZG dd� dej
�ZdS )� N)�nn� )� Downsample2D�
ResnetBlock2D�TemporalConvLayer�6Upsample2D)�Transformer2DModel)�TransformerTemporalModelFT�defaultc C sv | dkr$t |||||||| | |d�7S | dkrd|8d u r<td��t|||||||| | |9|| |
|||d�S t| � d���d S )N� DownBlock3D)10�11num_layers� in_channels� out_channels�
temb_channels�add_downsample�12resnet_eps�
resnet_act_fn�
resnet_groups�downsample_padding�resnet_time_scale_shift�CrossAttnDownBlock3Dz>cross_attention_dim must be specified for CrossAttnDownBlock3D)r r
r r r r r r r �cross_attention_dim�attn_num_head_channels�dual_cross_attention�use_linear_projection�only_cross_attention�upcast_attentionr � does not exist.)r �13ValueErrorr )�down_block_typer r
r r r r r r r r r r r r r r � r �T/home/deep/Projects/audio_diffusion/diffusers/src/diffusers/models/unet_3d_blocks.py�get_down_block sF � �r" c C sv | dkr$t |||||||||14|d�15S | dkrd| d u r<td��t|||||||||16| | | |
|||d�S t| � d���d S )N� UpBlock3D)17r r
r �prev_output_channelr � add_upsampler r r r �CrossAttnUpBlock3Dz<cross_attention_dim must be specified for CrossAttnUpBlock3D)r r
r r$ r r% r r r r r r r r r r r )r# r r& )�
up_block_typer r
r r$ r r% r r r r r r r r r r r r r! � get_up_blockO sF � �r( c
s> e Zd Zdeeeeeeeeed � � fd d
�
Zddd�Z� Z S )�UNetMidBlock3DCrossAttn� r �����ư>r18 �swish� T� �?� F) r
r �dropoutr r r r r �resnet_pre_normc s t � �� d| _|19| _|d ur"|n t|d d�}t||||||||| | d�20g}t||dd�g}g }g }t|�D ]z}|�t ||21 |22|d| |||d�� |�t23||24 |25|d| |d �� |�t||||||||| | d�26� |�t||dd�� qnt � |�| _
t � |�| _t � |�| _t � |�| _d S )27NT� r- �28r
r r �eps�groupsr0 �time_embedding_norm�
non_linearity�output_scale_factor�pre_norm皙�����?�r0 r )r
r r �norm_num_groupsr r �r
r r r< )�super�__init__�has_cross_attentionr �minr r �range�appendr r r �29ModuleList�resnets�30temp_convs�31attentions�temp_attentions)�selfr
r r0 r r r r r r1 r r8 r r r r rE rF rG rH �_�� __class__r r! r? � s� 32���� �� ��33���� z UNetMidBlock3DCrossAttn.__init__Nc C s� | j d ||�}| jd ||d�}t| j| j| j dd � | jdd � �D ]@\}}} }34||||d�j}|||d�j}| ||�}|35||d�}qH|S )Nr ��36num_framesr ��encoder_hidden_states�cross_attention_kwargs)rE rF �ziprG rH �sample) rI �
hidden_states�tembrP �attention_maskrN rQ �attn� temp_attn�resnet� temp_convr r r! �forward� s ��37z UNetMidBlock3DCrossAttn.forward)
r* r r+ r38 r, r- Tr r. r/ FTF)NNNr N�39�__name__�40__module__� __qualname__�int�float�str�boolr? r[ �
__classcell__r r rK r! r) � s< ��f �r) c s@ e Zd Zdeeeeeeeeeed �41� fd d
�
Zddd�Z� Z S )r r* r r+ r42 r, r- Tr/ r. F�43r
r r r0 r r r r r r1 c s t � �� g }g }g }g }d| _| | _t|�D ]�}|dkr>|n|}|�t||||| ||||
|44d�45� |�t||dd�� |�t|| | |d| | |||d� � |�t || | |d| | d�� q.t46� |�| _ t47� |�| _
t48� |�| _t49� |�| _|�rt50� t|d||d d51�g�| _nd | _d | _d S ) NTr r3 r: r; r �r
r r r< r r r r= �op��use_convr �padding�nameF)r> r? r@ r rB rC r r r r r rD rE rF rG rH r � downsamplers�gradient_checkpointing)rI r
r r r0 r r r r r r1 r r r8 r r r r r r rE rG rH rF �irK r r! r? s� 52 ������
��53 54���z CrossAttnDownBlock3D.__init__Nc
C s� d}t | j| j| j| j�D ]J\}} }55} |||�}| ||d�}|56|||d�j}| ||d�j}||f7 }q| jd ur�| jD ] } | |�}qv||f7 }||fS )Nr rM rO )rR rE rF rG rH rS rl )
rI rT rU rP rV rN rQ �
output_statesrY rZ rW rX � downsamplerr r r! r[ e s$ 57�58 � 59606162z CrossAttnDownBlock3D.forward)r* r r+ r63 r, r- Tr r/ r. r TFFFF)NNNr Nr\ r r rK r! r sD ��b �r c s@ e Zd Zdeeeeeeeeeed �64� fd65d �
Zdd
d�Z� Z S )r r* r r+ r66 r, r- Tr. re c s� t � �� g }g }t|�D ]J}|dkr*|n|}|�t||||| |||| |67d�68� |�t||dd�� qt�|�| _t�|�| _ | r�t�t69|d||
dd�g�| _ nd | _ d| _ d S ) Nr r3 r: r; Trg rh F)
r> r? rB rC r r r rD rE rF r rl rm )rI r
r r r0 r r r r r r1 r8 r r rE rF rn rK r r! r? � sJ 70 ���� 71���zDownBlock3D.__init__Nc C sl d}t | j| j�D ](\}}|||�}|||d�}||f7 }q| jd urd| jD ] }||�}qL||f7 }||fS )Nr rM )rR rE rF rl )rI rT rU rN ro rY rZ rp r r r! r[ � s 72 73747576zDownBlock3D.forward)77r* r r+ r78 r, r- Tr. Tr )Nr r\ r r rK r! r � s. ��<r c sB e Zd Zdeeeeeeeeeeed � � fd d
�
Zddd�Z� Z S )r& r* r r+ r79 r, r- Tr/ r. F) r
r r$ r r0 r r r r r r1 c s4 t � �� g }g }g }g }d| _| | _t|�D ]�}||d krB|n|}|dkrR|n|}|�t|| ||||80||| || d�81� |�t||dd�� |�t|| | |d|
|82|||d� � |�t || | |d|
|83d�� q.t84� |�| _ t85� |�| _
t86� |�| _t87� |�| _|�r$t88� t|d|d �g�| _nd | _d89| _d S ) NTr r r3 r: r; rf r= �ri r F)r> r? r@ r rB rC r r r r r rD rE rF rG rH r �90upsamplersrm )rI r
r r$ r r0 r r r r r r1 r r r8 r% r r r r rE rF rG rH rn �res_skip_channels�resnet_in_channelsrK r r! r? � s| 91 ������
��92 zCrossAttnUpBlock3D.__init__Nc C s� t | j| j| j| j�D ]f\} }93} } |d }
|d d� }tj||
gdd�}| ||�}|94||d�}| |||d�j}| ||d�j}q| jd ur�| jD ]}|||�}q�|S )N�����r ��dimrM rO ) rR rE rF rG rH �torch�catrS rr )rI rT �res_hidden_states_tuplerU rP �
upsample_sizerV rN rQ rY rZ rW rX �res_hidden_states� upsamplerr r r! r[ 3 s$ � 95 �9697 zCrossAttnUpBlock3D.forward)r* r r+ r98 r, r- Tr r/ r. TFFFF)NNNNr Nr\ r r rK r! r& � sF ��_ �r& c
sB e Zd Zdeeeeeeeeeeed � � fd99d �
Zdd
d�Z� Z S )r# r* r r+ r100 r, r- Tr. ) r
r$ r r r0 r r r r r r1 c s� t � �� g }g }t|�D ]b}||d kr.|n|}|dkr>|n|}|�t|| ||||101||| | | d�102� |�t||dd�� qt�|�| _t�|�| _ |
r�t�t103|d|d�g�| _ nd | _ d| _ d S ) Nr r r3 r: r; Trq F)
r> r? rB rC r r r rD rE rF r rr rm )rI r
r$ r r r0 r r r r r r1 r8 r% rE rF rn rs rt rK r r! r? X sB 104 ���� zUpBlock3D.__init__Nc 105 C sx t | j| j�D ]D\}}|d }|d d� }tj||gdd�}|||�}|||d�}q| jd urt| jD ]} | ||�}qd|S )Nru r rv rM )rR rE rF rx ry rr )106rI rT rz rU r{ rN rY rZ r| r} r r r! r[ � s 107108109 zUpBlock3D.forward) r* r r+ r110 r, r- Tr. T)NNr r\ r r rK r! r# W s. ��8r# )NNNFTFFr111 )NNFTFFr112 )rx r rY r r r r �transformer_2dr �transformer_temporalr r" r( �Moduler) r r r& r# r r r r! �<module> s8
�113C �1148~ O