Apppublic
declare-lab/tango2

sourceHugging Faceupdated 2y agoView on Hugging Face
92likes
rootdiffuserssrcdiffusersmodels__pycache__embeddings.cpython-39.pyc
embeddings.cpython-39.pyc114 linesDownload Raw Back to __pycache__
1a

2�0dY2�@s�ddlZddlmZddlZddlZddlmZdejee	e3e4ed�dd	�Zdd5d�Zdd
�Z
dd�ZGdd�dej�ZGdd�dej�ZGdd�dej�ZGdd�dej�ZGdd�dej�ZGdd�dej�ZGdd�dej�ZdS) �N)�Optional)�nnF��')�	timesteps�
embedding_dim�flip_sin_to_cos�downscale_freq_shift�scale�6max_periodc	Cs�t|j�dksJd��|d}t�|�tjd|tj|jd�}|||}t�|�}|dd�df�	�|ddd�f}||}tj7t�|�t�|�gdd�}|r�tj8|dd�|d�f|dd�d|�fgdd�}|ddkr�tj
j�|d	�}|S)9a�10    This matches the implementation in Denoising Diffusion Probabilistic Models: Create sinusoidal timestep embeddings.11 12    :param timesteps: a 1-D Tensor of N indices, one per batch element.13                      These may be fractional.14    :param embedding_dim: the dimension of the output. :param max_period: controls the minimum frequency of the15    embeddings. :return: an [N x dim] Tensor of positional embeddings.16    rzTimesteps should be a 1d-array�r)�start�end�dtype�deviceN�������dim)rrrr)�len�shape�math�log�torch�arange�float32r�exp�float�cat�sin�cosr�17functional�pad)	rrrr	r18rZhalf_dim�exponent�emb�r$�P/home/deep/Projects/audio_diffusion/diffusers/src/diffusers/models/embeddings.py�get_timestep_embeddings�19$2r&cCs�tj|tjd�}tj|tjd�}t�||�}tj|dd�}|�dd||g�}t||�}|r~|dkr~tjt�||g�|gdd�}|S)z�20    grid_size: int of the grid height and width return: pos_embed: [grid_size*grid_size, embed_dim] or21    [1+grid_size*grid_size, embed_dim] (w/ or w/o cls_token)22    �rr��axisrr)	�nprr�meshgrid�stack�reshape�!get_2d_sincos_pos_embed_from_grid�concatenate�zeros)�	embed_dimZ	grid_sizeZ	cls_token�extra_tokensZgrid_hZgrid_w�grid�	pos_embedr$r$r%�get_2d_sincos_pos_embedAs23r5cCsN|ddkrtd��t|d|d�}t|d|d�}tj||gdd�}|S)Nrr� embed_dim must be divisible by 2rr()�24ValueError�!get_1d_sincos_pos_embed_from_gridr*r/)r1r3Zemb_hZemb_wr#r$r$r%r.Rsr.cCs�|ddkrtd��tj|dtjd�}||d}dd|}|�d�}t�d	||�}t�|�}t�|�}tj||gd25d�}|S)zu26    embed_dim: output dimension for each position pos: a list of positions to be encoded: size (M,) out: (M, D)27    rrr6r'g@��?rrzm,d->mdrr()	r7r*r�float64r-�einsumrrr/)r1�pos�omega�outZemb_sinZemb_cosr#r$r$r%r8^s282930r8cs*eZdZdZd�fdd	�	Zd31d�Z�ZS)
�32PatchEmbedz2D Image to Patch Embedding�����FTc	s�t���||||}	||_||_tj||||f||d�|_|rXtj|ddd�|_nd|_t	|t33|	d��}34|jdt�
|35����d�dd�dS)	N)�kernel_size�stride�biasFg�����ư>)�elementwise_affine�epsg�?r4r)�36persistent)�super�__init__�flatten�37layer_normr�Conv2d�proj�	LayerNorm�normr5�int�register_bufferr�38from_numpyr�	unsqueeze)�self�height�widthZ39patch_size�in_channelsr1rMrLrFZnum_patchesr4��	__class__r$r%rKvs40�zPatchEmbed.__init__cCs<|�|�}|jr"|�d��dd�}|jr2|�|�}||jS)Nrr)rOrL�	transposerMrQr4)rVZlatentr$r$r%�forward�s4142zPatchEmbed.forward)r@r@rArBrCFTT)�__name__�43__module__�__qualname__�__doc__rKr]�
__classcell__r$r$rZr%r?ss�r?cs:eZdZdeeeeeed��fdd�
Zd	dd�Z�ZS)44�TimestepEmbedding�siluN)rY�time_embed_dim�act_fn�out_dim�post_act_fncst���t�||�|_|dur4tj||dd�|_nd|_|dkrNt��|_n6|dkrbt��|_n"|dkrvt�	�|_nt45|�d���|dur�|}n|}t�||�|_|dur�d|_nJ|dkr�t��|_n6|dkr�t��|_n"|dkr�t�	�|_nt46|�d���dS)NF)rFrd�mish�geluzE does not exist. Make sure to define one of 'silu', 'mish', or 'gelu')
rJrKr�Linear�linear_1�	cond_proj�SiLU�act�Mish�GELUr7�linear_2�post_act)rVrYrerfrgrhZ
cond_proj_dimZtime_embed_dim_outrZr$r%rK�s2	47zTimestepEmbedding.__init__cCsV|dur||�|�}|�|�}|jdur4|�|�}|�|�}|jdurR|�|�}|S�N)rmrlrorrrs)rV�sample�	conditionr$r$r%r]�s484950515253zTimestepEmbedding.forward)rdNNN)N)	r^r_r`rR�strrrKr]rbr$r$rZr%rc�s��,rccs.eZdZeeed��fdd�Zdd�Z�ZS)�	Timesteps��num_channelsrr	cs t���||_||_||_dSrt)rJrKrzrr	)rVrzrr	rZr$r%rK�s54zTimesteps.__init__cCst||j|j|jd�}|S)N)rr	)r&rzrr	)rVrZt_embr$r$r%r]�s�zTimesteps.forward)	r^r_r`rR�boolrrKr]rbr$r$rZr%rx�srxcs2eZdZdZdeed��fdd�
Zd	d55�Z�ZS)�GaussianFourierProjectionz-Gaussian Fourier embeddings for noise levels.�r9TF)�embedding_sizer56csZt���tjt�|�|dd�|_||_||_|rVtjt�|�|dd�|_	|j	|_dS)NF)�
requires_grad)57rJrKr�	Parameterr�randn�weightrr�W)rVr~r58Zset_W_to_weightrrrZr$r%rK�s59z"GaussianFourierProjection.__init__cCs�|jrt�|�}|dd�df|jddd�fdtj}|jrbtjt�|�t�|�gdd�}ntjt�|�t�|�gdd�}|S)Nrrr)	rrr�r*�pirrrr)rV�xZx_projr>r$r$r%r]�s60, z!GaussianFourierProjection.forward)r}r9TTF)	r^r_r`rarRrrKr]rbr$r$rZr%r|�s��r|cs4eZdZdZeeeed��fdd�Zdd�Z�ZS)�ImagePositionalEmbeddingsa�61    Converts latent image classes into vector embeddings. Sums the vector embeddings with positional embeddings for the62    height and width of the latent space.63 64    For more details, see figure 10 of the dall-e paper: https://arxiv.org/abs/2102.1209265 66    For VQ-diffusion:67 68    Output vector embeddings are used as input for the transformer.69 70    Note that the vector embeddings for the transformer are different than the vector embeddings from the VQVAE.71 72    Args:73        num_embed (`int`):74            Number of embeddings for the latent pixels embeddings.75        height (`int`):76            Height of the latent image i.e. the number of height embeddings.77        width (`int`):78            Width of the latent image i.e. the number of width embeddings.79        embed_dim (`int`):80            Dimension of the produced vector embeddings. Used for the latent pixel, height, and width embeddings.81    )�	num_embedrWrXr1csVt���||_||_||_||_t�|j|�|_t�|j|�|_	t�|j|�|_82dSrt)rJrKrWrXr�r1r�	Embeddingr#�83height_emb�	width_emb)rVr�rWrXr1rZr$r%rKs84z"ImagePositionalEmbeddings.__init__cCs�|�|�}|�tj|j|jd��d|j��}|�d�}|�tj|j	|jd��d|j	��}|�d�}||}|�d|j|j	d�}||dd�d|j85d�dd�f}|S)N�rrrr)r#r�rrrWr�viewrUr�rXr)rV�indexr#r�r�Zpos_embr$r$r%r]0s86"87"88$z!ImagePositionalEmbeddings.forward)r^r_r`rarRrKr]rbr$r$rZr%r�s�r�cs<eZdZdZ�fdd�Zd89dd�Zdejd�dd	�Z�Z	S)�LabelEmbeddinga790    Embeds class labels into vector representations. Also handles label dropout for classifier-free guidance.91 92    Args:93        num_classes (`int`): The number of classes.94        hidden_size (`int`): The size of the vector embeddings.95        dropout_prob (`float`): The probability of dropping a label.96    cs4t���|dk}t�|||�|_||_||_dS�Nr)rJrKrr��embedding_table�num_classes�dropout_prob)rVr��hidden_sizer�Zuse_cfg_embeddingrZr$r%rKQs9798zLabelEmbedding.__init__NcCsH|dur&tj|jd|jd�|jk}nt�|dk�}t�||j|�}|S)zB99        Drops labels to enable classifier-free guidance.100        Nrr�r)r�randrrr��tensor�wherer�)rV�labels�force_drop_idsZdrop_idsr$r$r%�101token_dropXs102zLabelEmbedding.token_drop)r�cCs6|jdk}|jr|s|dur(|�||�}|�|�}|Sr�)r��trainingr�r�)rVr�r�Zuse_dropout�103embeddingsr$r$r%r]cs104105106zLabelEmbedding.forward)N)N)107r^r_r`rarKr�r�108LongTensorr]rbr$r$rZr%r�Gs	109r�cs(eZdZd�fdd�	Zddd�Z�ZS)	�CombinedTimestepLabelEmbeddings皙�����?cs:t���tdddd�|_td|d�|_t|||�|_dS)Nr}Trry)rYre)rJrKrx�	time_projrc�timestep_embedderr��class_embedder)rVr�rZclass_dropout_probrZr$r%rKls110z(CombinedTimestepLabelEmbeddings.__init__NcCs2|�|�}|�|j|d��}|�|�}||}|S)Nr')r�r��tor�)rV�timestep�class_labels�hidden_dtypeZtimesteps_projZ
timesteps_embZconditioningr$r$r%r]ss111112113z'CombinedTimestepLabelEmbeddings.forward)r�)N)r^r_r`rKr]rbr$r$rZr%r�ksr�)Frrr)Fr)r�typingr�numpyr*rr�TensorrRr{rr&r5r.r8�Moduler?rcrxr|r�r�r�r$r$r$r%�<module>s4��+114(<A$