antoniomae1234/Seed-VC-characters
6
1o
2 ��yf!0 � @ s� d Z ddlZddlZddlZddlZddlmZ ddlmZ ddl3m Z m Z ddlZddl
Z
G dd� dej�Zeg d��Zd2d4ejd ed ejfd
d�Zd3d5ejded ed ejfdd�Z d4dejdededed ef6dd�Zd4dejdededefdd�Zd5dejd ejeef d ed e fd d!�Z dejd ejeef fd"d#�Z G d$d%� d%ej�Z G d&d'� d'ej�Z G d(d)� d)ej�Z!G d*d+� d+ej�Z"G d,d-� d-ej�Z#G d.d/� d/ej�Z$G d0d1� d1ej�Z%dS )6z,Convolutional layers wrappers and utilities.� N)�nn)�7functional)�
spectral_norm� weight_normc sF e Zd ZdZdejeeje ej f f� fdd� Z8� fdd�Z � Z S )�
ConvLayerNormz�9 Convolution-friendly LayerNorm that moves channels to last dimensions10 before running the normalization and moves them back to original position right after.11 �normalized_shapec s t � j|fi |�� d S �N)�super�__init__)�selfr �kwargs�� __class__� � D:\seed-vc\modules\encodec.pyr12 s zConvLayerNorm.__init__c s( t �|d�}t� �|�}t �|d�}d S )Nzb ... t -> b t ...zb t ... -> b ... t)�einopsZ rearranger �forward�r �xr
r r r s zConvLayerNorm.forward)
�__name__�13__module__� __qualname__�__doc__�tp�Union�int�List�torch�Sizer14 r �
__classcell__r r r
r r s &r )�noner r Ztime_layer_norm�15layer_norm�time_group_normr �module�norm�returnc C s0 |t v sJ �|dkrt| �S |dkrt| �S | S )Nr r )�CONV_NORMALIZATIONSr r )r# r$ r r r �apply_parametrization_norm* s r' F�causalc K s� |t v sJ �|dkr t| tjjj�sJ �t| jfi |��S |dkr<|r'td��t| tjjj�s1J �tj d| jfi |��S t�16� S )z�Return the proper normalization module. If causal is True, this will ensure the returned17 module is causal, or return an error if the normalization doesn't support causal evaluation.18 r! r" z,GroupNorm doesn't support causal evaluation.� ) r& �19isinstancer �modules�conv�_ConvNdr � out_channels�20ValueError� GroupNorm�Identity)r# r( r$ � norm_kwargsr r r �get_norm_module6 s r3 r � kernel_size�stride�
padding_totalc C s@ | j d }|| | | d }t�|�d | || }|| S )zSee `pad_for_conv1d`.21 �����r) )�shape�math�ceil)r r4 r5 r6 �length�n_framesZ ideal_lengthr r r � get_extra_padding_for_conv1dG s 22r= c C s t | |||�}t�| d|f�S )a� Pad for a convolution to make sure that the last window is full.23 Extra padding is added at the end. This is required to ensure that we can rebuild24 an output of the same length, as otherwise, even with padding, some time steps25 might get removed.26 For instance, with total padding = 4, kernel size = 4, stride = 2:27 0 0 1 2 3 4 5 0 0 # (0s are padding)28 1 2 3 # (output frames of a convolution, last 0 is never used)29 0 0 1 2 3 4 5 0 # (output of tr. conv., but pos. 5 is going to get removed as padding)30 1 2 3 4 # once you removed padding, we are missing one time step !31 r )r= �F�pad)r r4 r5 r6 �
extra_paddingr r r �pad_for_conv1dQ s rA �zero� �paddings�mode�valuec C s� | j d }|\}}|dkr|dksJ ||f��|dkrKt||�}d}||kr4|| d }t�| d|f�} t�| |||�} | j d | }32| dd|33�f S t�| |||�S )z�Tiny wrapper around F.pad, just to allow for reflect padding on small input.34 If this is the case, we insert extra 0 padding to the right before the reflection happen.35 r7 r �reflectr) .N)r8 �maxr> r? ) r rD rE rF r; � padding_left�
padding_right�max_padZ extra_pad�padded�endr r r �pad1d` s 36 37 rN c C sX |\}}|dkr |dksJ ||f��|| | j d ks J �| j d | }| d||�f S )zCRemove padding from x, handling properly zero padding. Only for 1d!r r7 .)r8 )r rD rI rJ rM r r r �unpad1dt s38 rO c �L e Zd ZdZddi d�dededejeejf f� fdd �Z d39d � Z40� Z S ) �41NormConv1dz�Wrapper around Conv1d and normalization applied to this conv42 to provide a uniform interface across normalization approaches.43 Fr �r( r$ r2 r( r$ r2 c �D t � �� ttj|i |��|�| _t| j||fi |��| _|| _d S r ) r r44 r' r �Conv1dr, r3 r$ � norm_type�r r( r$ r2 �argsr r
r r r45 � � 4647zNormConv1d.__init__c C � | � |�}| �|�}|S r �r, r$ r r r r r � � 4849zNormConv1d.forward� r r r r �bool�strr �Dict�Anyr50 r r r r r
r rQ } � ��rQ c �F e Zd ZdZdi d�dedejeejf f� fdd�Zdd � Z � Z51S )52�53NormConv2dz�Wrapper around Conv2d and normalization applied to this conv54 to provide a uniform interface across normalization approaches.55 r �r$ r2 r$ r2 c sF t � �� ttj|i |��|�| _t| jfd|d�|��| _|| _d S �NF)r( r$ ) r r56 r' r �Conv2dr, r3 r$ rU �r r$ r2 rW r r
r r r57 � s 5859zNormConv2d.__init__c C rY r rZ r r r r r � r[ zNormConv2d.forward� r r r r r^ r r_ r` r60 r r r r r
r rc � s 61��rc c rP ) �NormConvTranspose1dz�Wrapper around ConvTranspose1d and normalization applied to this conv62 to provide a uniform interface across normalization approaches.63 Fr rR r( r$ r2 c rS r ) r r64 r' r �ConvTranspose1d�convtrr3 r$ rU rV r
r r r65 � rX z NormConvTranspose1d.__init__c C rY r �rk r$ r r r r r � r[ zNormConvTranspose1d.forwardr\ r r r
r ri � ra ri c rb )66�NormConvTranspose2dz�Wrapper around ConvTranspose2d and normalization applied to this conv67 to provide a uniform interface across normalization approaches.68 r rd r$ r2 c s@ t � �� ttj|i |��|�| _t| jfd|d�|��| _d S re )r r69 r' r �ConvTranspose2drk r3 r$ rg r
r r r70 � s 71 z NormConvTranspose2d.__init__c C rY r rl r r r r r � r[ zNormConvTranspose2d.forwardrh r r r
r rm � s 72��rm c st e Zd ZdZddddddi dfdeded ed73ed ed ed
edededejeej f def� fdd�
Z74dd� Z � Z S )�SConv1dz]Conv1d with some builtin handling of asymmetric or causal padding75 and normalization.76 r) TFr rG � in_channelsr. r4 r5 �dilation�groups�biasr( r$ r2 �pad_modec
sf t � �� |dkr |dkr t�d|� d|� d|� d�� t||||||||| |77d�78| _|| _| | _d S )Nr) zKSConv1d has been initialized with stride > 1 and dilation > 1 (kernel_size=z stride=z , dilation=z).)rq rr rs r( r$ r2 )r r79 �warnings�warnrQ r, r( rt )
r rp r. r4 r5 rq rr rs r( r$ r2 rt r r
r r r80 � s 81��82�83�84zSConv1d.__init__c C s� |j \}}}| jjjd }| jjjd }| jjjd }|d | d }|| }t||||�} | jr<t||| f| jd�}n|d }85||86 } t|| |87| f| jd�}| �|�S )Nr r) )rE � ) r8 r, r4 r5 rq r= r( rN rt ) r r �B�C�Tr4 r5 rq r6 r@ rJ rI r r r r � s 88zSConv1d.forward)
r r r r r r] r^ r r_ r` r89 r r r r r
r ro � s4 ����������ro c sb e Zd ZdZddddi fdededed ed90ed ed ed
ej eej91f f� fdd�
Z dd� Z � Z
S )�SConvTranspose1dzfConvTranspose1d with some builtin handling of asymmetric or causal padding92 and normalization.93 r) Fr � �?rp r. r4 r5 r( r$ �trim_right_ratior2 c 94 sb t � �� t|||||||d�| _|| _|| _| js#| jdks#J d��| jdkr-| jdks/J �d S )NrR r| zB`trim_right_ratio` != 1.0 only makes sense for causal convolutionsrC )r r95 ri rk r( r} )96r rp r. r4 r5 r( r$ r} r2 r r
r r r97 � s 9899�� zSConvTranspose1d.__init__c C s� | j j jd }| j j jd }|| }| � |�}| jr/t�|| j �}|| }t|||f�}|S |d }|| }t|||f�}|S )Nr rw )rk r4 r5 r( r9 r: r} rO )r r r4 r5 r6 �yrJ rI r r r r � s 100�zSConvTranspose1d.forward)r r r r r r] r^ �floatr r_ r` r101 r r r r r
r r{ � s& �������
r{ c s8 e Zd ZdZd dededef� fdd�
Zd d102� Z� ZS ) �SLSTMz~103 LSTM without worrying about the hidden state, nor the layout of the data.104 Expects input as convolutional layout.105 rw T� dimension�106num_layers�skipc s* t � �� || _t�|||�| _d | _d S r )r r107 r� r �LSTM�lstm�hidden)r r� r� r� r
r r r108 s 109110zSLSTM.__init__c C sX |� ddd�}| jr| �|�\}}n111| �|| j�\}| _| jr#|| }|� ddd�}|S )Nrw r r) )�permute�trainingr� r� r� )r r r~ �_r r r r s z
SLSTM.forward)rw T) r r r r r r] r112 r r r r r
r r� s r� )r )Fr )r )rB rC )&r r9 �typingr ru r r �torch.nnr r> �torch.nn.utilsr r r � LayerNormr � frozensetr& �Moduler^ r' r] r3 �Tensorr r= rA �Tupler rN rO rQ rc ri rm ro r{ r� r r r r �<module> s< ��113� 114( ')