procgne/Plonk
0
1o
2 >��f � @ sD d dl Z d dlmZ d dlZG dd� dej�ZG dd� dej�ZdS )� Nc s* e Zd ZdZd� fdd� Zdd� Z� ZS ) �PositionalEmbedding�23 Taken from https://github.com/NVlabs/edm4 �' Fc s` t � �� || _|| _|| _tjd| jd tjd�}d| | j }d| j | }| �d|� d S )Nr � )�start�end�dtype� �freqs) �super�__init__� num_channels�
max_positions�endpoint�torch�arange�float32�register_buffer)�selfr
r r r5 �� __class__� �A/home/dufour/Documents/diff_plonk/models/positional_embeddings.pyr s 6z PositionalEmbedding.__init__c C s4 t �|| j�}t j|�� |�� gdd�}|�|j�S )Nr ��dim)r �outerr7 �cat�cos�sin�tor )r �x�outr r r �forward s zPositionalEmbedding.forward)r F��__name__�8__module__� __qualname__�__doc__r r" �
__classcell__r r r r r s 9r c s* e Zd ZdZd� fdd� Zdd� Z� ZS )�FourierEmbeddingr � c s( t � �� | �dt�|d �| � d S )Nr10 r )r r r r �randn)r r
�scaler r r r " s 11 zFourierEmbedding.__init__c C s< |� dtj | j �|j��}tj|�� |� � gdd�}|S )Nr r r )12�ger�np�pir13 r r r r r r )r r r r r r" & s zFourierEmbedding.forward)r* r# r r r r r) s r) )r �torch.nn�nn�numpyr. �Moduler r) r r r r �<module> s14