Montey/python-edge
0
1o
2 �e!g9 � @ sL d Z dd lZddlmZ G dd� dejj�ZG dd� d�ZG dd� d�Zd S ) N� )�tqdmc s$ e Zd Z� fdd�Zdd� Z� ZS )�LossSchedulerModelc s� t t| ��� t|j�dkrt|j�dksJ �|jd }||jd kr*||jd ks,J �| �dtj�|�� | �dtj�|�� d S )N� � r �wx�we) �superr �__init__�len�shape�register_parameter�torch�nn� Parameter)�Ar r �B�� __class__� �/sandbox/src/loss.pyr s � zLossSchedulerModel.__init__c C s\ |}|t |� d dksJ �|| j| }t|| j| �D ]3\}}||| 7 }q |�|j�S )Nr r )r4 r �zipr �to�dtype)r �t�xT�e_prevr �C�D�Er r r �forward s *& zLossSchedulerModel.forward)�__name__�5__module__� __qualname__r r �
__classcell__r r r r r s r c @ sJ e Zd Zdd� Zedd� �Zdd� Zddd �Zd6d � Ze �7� d d
� �Z dS )�
LossSchedulerc C s || _ || _d| _d| _d S )Ng �?r )� timesteps�model�init_noise_sigma�order)r r% r& r r r r s zLossScheduler.__init__c C s( t j| dd�\}}}t||�}t||�S �N�cpu)� map_location)r
�loadr r$ )�pathr r r r r r r r, s (zLossScheduler.loadc C s0 | j | jj| jj}}}t�|||f|� d S �N)r% r& r r r
�save)r r- r r r r r r r/ s 0 zLossScheduler.save�cudac C s6 |}t | _g | _d| _| j�|�| _| j�|�| _d S )N�����)�_Ar r �t_prevr& r r% )r �num_inference_steps�devicer r r r �
set_timesteps s 6 zLossScheduler.set_timestepsc O � |S r. r )r �sampler r r r r �scale_model_input � z LossScheduler.scale_model_inputc O s� | }|j �� �|�}|jdks||jd ksJ �|jdkr ||_|j�|� |�||j|j�}|d t|j �krDt |_g |_d|_|fS ||_|fS )Nr1 r )8r% �tolist�indexr3 r r �appendr& r9 r2 ) �self� model_output�timestepr8 r r r r r r r r �step s 0 $�zLossScheduler.stepN)r0 ) r r! r" r � staticmethodr, r/ r6 r9 r
�no_gradrA r r r r r$ 10 s 1112r$ c @ sP e Zd Zddd�Zdd� Zdd� Zdd � Zd13d � Zd d
� Zdd� Z ddd�Z14dS )�SchedulerWrapper�loss_params.pthc C s, || _ i i i | _| _| _t| _|| _d S r. )� scheduler�catch_x�catch_e�catch_x_r2 �loss_scheduler�loss_params_path)r rF rK r r r r s , zSchedulerWrapper.__init__c K sz |}| j tu r"| jj|fi |��}| jj| _| jj| _| jj| _|S | j j|fi |��}| j j| _| jj| _| jj| _|S r. )rJ r2 rF r6 r% r' r( )r r4 r r r r r r r6 s @6z SchedulerWrapper.set_timestepsc K s� |}|}|}| j tu r]| jj|||fi |��}|�� }|| jvr/g | j|< g | j|< g | j|< | j| �|� � �15� � � � | j| �|� � �16� � � � | j| �|d � � �17� � � � |S | j j|||fi |��}|S )Nr ) rJ r2 rF rA r; rG rH rI r= �clone�detachr* ) r r? r@ r8 �Fr r r r r r r rA s 18 (\ zSchedulerWrapper.stepc C r7 r. r )r r8 r@ r r r r9 & r: z"SchedulerWrapper.scale_model_inputc C s | j �|||�}|S r. )rF � add_noise)r �original_samples�noiser% r r r r rO ' s zSchedulerWrapper.add_noisec C s� t dd� | jD �dd�}g g }}|D ] }tj| j| dd�}|�|� tj| j| dd�}|�|� q|d }tj| j| dd�}|�|� tj|tjd�}t� |�}t� |�}|||fS ) Nc S s g | ]}|�qS r r )�.0r r r r �19<listcomp>) s z-SchedulerWrapper.get_path.<locals>.<listcomp>T)�reverser )�dimr1 )r )20�sortedrG r
�catr= rH rI �tensor�int32�stack) r r r r r rN �G�H�Ir r r �get_path( s "FTzSchedulerWrapper.get_pathc C s4 t j| jdd�\}}}t||�| _t|| j�| _d S r) )r
r, rK r �21loss_modelr$ rJ )r r r r r r r �load_loss_params, s 4 z!SchedulerWrapper.load_loss_params� c C s | � � d S r. )r` )r �num_accelerate_stepsr r r � prepare_loss- s z SchedulerWrapper.prepare_lossN)rE )ra ) r r! r" r r6 rA r9 rO r^ r` rc r r r r rD s 22rD )r2 r
r r �Moduler r$ rD r r r r �<module> s