CLYang617/RemoteSensingChangeDetection-RSCD.HA2F
0
1a
2 8��g� � @ sL d dl Z d dlmZ d dlmZ d dlmZ G dd� dej�Zd3dd �Z dS ) � N)�
trunc_normal_)� weight_normc s. e Zd Zd � fdd� Zdd � Zd4d � Z� ZS )
�DINOHeadF� � � Tc s` t � �� t|d�}t||||||d�| _| �| j� ttj ||dd��| _5| j6j j �
d� d S )N� )�7hidden_dim�use_bn�biasF�r )�super�__init__�max�8_build_mlp�mlp�apply�
_init_weightsr �nn�Linear�9last_layer�weight_g�data�fill_)�self�in_dim�out_dimr10 �nlayersr �bottleneck_dimZmlp_bias�� __class__� �FC:\Users\82607\Desktop\code\ChangeViT-master\model\layers\dino_head.pyr
s 11 1213 zDINOHead.__init__c C sD t |tj�r@t|jdd� t |tj�r@|jd ur@tj�|jd� d S )Ng{�G�z�?)�stdr )�14isinstancer r r �weightr �init� constant_)r �mr! r! r" r s zDINOHead._init_weightsc C s@ | � |�}|jtjkrdnd}tjj|dd|d�}| �|�}|S )Ng�����ư>g�-���q=������ )�dim�p�eps)r �dtype�torch�float16r �15functional� normalizer )r �xr- r! r! r" �forward$ s16 1718zDINOHead.forward)Fr r r T)�__name__�19__module__� __qualname__r r r4 �
__classcell__r! r! r r" r s �r FTc C s� | dkrt j|||d�S t j|||d�g}|r>|�t �|�� |�t �� � t| d �D ]<}|�t j|||d�� |r�|�t �|�� |�t �� � qX|�t j|||d�� t j|� S d S )Nr r r* )r r �append� BatchNorm1d�GELU�range�20Sequential)r r r r r21 r �layers�_r! r! r" r , s r )NFT)22r/ �torch.nnr �
torch.nn.initr Ztorch.nn.utilsr �Moduler r r! r! r! r" �<module> s23