Aluode/PerceptionLabPortable
0
1o
2 .�Yiqo � @ s d Z ddlZddlZddlmZ ddlZddlZddlmZm Z ddl3m Z ddl m
Z
mZmZmZ ddlmZ dd lmZmZ dd4lmZ d d lmZ e�e�Z d=de e e f fdd�Z G dd� de j �Z!G dd� de j"�Z#G dd� de j$�Z%G dd� de j&�Z'G dd� de j$�Z(d>d ejd e)d e dejfd d!�Z*G d"d#� d#e j$�Z+d?d%d&�Z,G d'd(� d(e j$�Z-G d)d*� d*e j$�Z.G d+d,� d,e j$�Z/G d-d.� d.e j$�Z0G d/d0� d0e j$�Z1eG d1d2� d2e��Z2eG d3d4� d4e2��Z3ed5d6�G d7d8� d8e2��Z4ed9d6�G d:d;� d;e2e��Z5g d<�Z6dS )@z9PyTorch BiT model. Also supports backbone for ViT hybrid.� N)�Optional)�Tensor�nn� )�ACT2FN)�BackboneOutput� BaseModelOutputWithNoAttention�(BaseModelOutputWithPoolingAndNoAttention�$ImageClassifierOutputWithNoAttention)�PreTrainedModel)�auto_docstring�logging)�
BackboneMixin� )� BitConfig� �returnc C s� d}| du r|d ||d d } | |fS t | t�r_| �� } | dkrI|dkrA||d d dkrA|d ||d d } | |fS d} d}| |fS | dkrSd} | |fS |d ||d d } | |fS ) al 5 Utility function to get the tuple padding value given the kernel_size and padding.6 7 Args:8 padding (Union[`str`, `int`], *optional*):9 Padding value, can be either `"same"`, `"valid"`. If a different value is provided the default padding from10 PyTorch is used.11 kernel_size (`int`, *optional*, defaults to 7):12 Kernel size of the convolution layers.13 stride (`int`, *optional*, defaults to 1):14 Stride value of the convolution layers.15 dilation (`int`, *optional*, defaults to 1):16 Dilation value of the convolution layers.17 FNr � Zsamer T�valid)�18isinstance�str�lower)�padding� kernel_size�stride�dilationZdynamic� r ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\transformers/models/bit/modeling_bit.py�get_padding_value) s$ 19 ���r c s6 e Zd ZdZ d20� fdd� Zdd � Z� ZS ) �WeightStandardizedConv2dz�Conv2d with Weight Standardization. Includes TensorFlow compatible SAME padding. Used for ViT Hybrid model.21 22 Paper: [Micro-Batch Training with Batch-Channel Normalization and Weight23 Standardization](https://huggingface.co/papers/1903.10520v2)24 r �SAMEF�����ư>c25 26 sT t ||||d�\}}27t� j||||||||d� |28r"t|||�| _nd | _| | _d S )N)r r )r r r �groups�bias)r �super�__init__� DynamicPad2d�pad�eps) �selfZ29in_channel� out_channelsr r r r r" r# r( Z30is_dynamic�� __class__r r r% Y s �3132z!WeightStandardizedConv2d.__init__c C sj | j d ur33| � |�}tjj| j�d| jd�d d dd| jd��| j�}tj� ||| j34| j | j | j
| j�}|S )Nr �����T� )�trainingZmomentumr( )r' r �35functionalZ36batch_norm�weightZreshaper* r( Z37reshape_asZconv2dr# r r r r" )r) � hidden_stater1 r r r �forwardv s 3839 ���z WeightStandardizedConv2d.forward)r r r r Fr! ��__name__�40__module__� __qualname__�__doc__r% r3 �
__classcell__r r r+ r r R s � r c s* e Zd ZdZd� fdd� Zdd� Z� ZS ) �BitGroupNormActivationzQ41 A module that combines group normalization with an activation function.42 ��h㈵��>Tc s8 t � j|j|||d� |rt|j | _d S t�� | _d S )N)r( �affine)r$ r% �43num_groupsr �44hidden_act�45activationr �Identity)r) �config� num_channelsr( r< �apply_activationr+ r r r% � s z BitGroupNormActivation.__init__c C s* t j�|| j| j| j| j�}| �|�}|S �N)r r0 Z46group_normr= r1 r# r( r? )r) r2 r r r r3 � s 47z BitGroupNormActivation.forward)r; TTr4 r r r+ r r: � s r: c s* e Zd ZdZd� fdd� Zdd� Z� ZS )r& z�48 A module that wraps dynamic padding of any input, given the parameters of the convolutional layer and the input49 hidden states.50 r c sj t � �� t|t�r||f}t|t�r||f}t|t�r ||f}|| _|| _|| _|| _dd� }|| _d S )Nc S s0 t t�| | �d | |d | d | d�S )Nr r )�max�math�ceil)�xr r r r r r �compute_padding� s 0z.DynamicPad2d.__init__.<locals>.compute_padding) r$ r% r �intr r r �valuerI )r) r r r rK rI r+ r r r% � s 5152535455zDynamicPad2d.__init__c C s� |� � dd � \}}| �|| jd | jd | jd �}| �|| jd | jd | jd �}|dks4|dkrNtjj||d ||d |d ||d g| jd�}|S )N�����r r r )rK ) �sizerI r r r r r0 r' rK )r) �inputZ input_heightZ input_width�padding_height�
padding_widthr r r r3 � s ""5657��58zDynamicPad2d.forward)r r4 r r r+ r r& � s r& c s< e Zd ZdZ d
def� fd d59�
Zd d � Z� ZS )� BitMaxPool2dz1Tensorflow like 'SAME' wrapper for 2D max poolingNr F�r r r Tr c s� t |tjj�r |n||f}t |tjj�r|n||f}t |tjj�r#|n||f}t� �|||||� |r=t||||�| _d S t� � | _d S rD )60r � collections�abc�Iterabler$ r% r& r' r r@ )r) r r r � ceil_moder Z
padding_value�use_dynamic_paddingr+ r r r% � s 61zBitMaxPool2d.__init__c C s* | � |�}tj�|| j| j| j| j| j�S rD ) r' r r0 Z62max_pool2dr r r r rV �r) �
hidden_statesr r r r3 � s 63�zBitMaxPool2d.forward)Nr FrR r T)r5 r6 r7 r8 rJ r% r3 r9 r r r+ r rQ � s ��rQ c s8 e Zd ZdZdef� fdd� Zdedefdd�Z� ZS ) �
BitEmbeddingszL64 BiT Embeddings (stem) composed of a single aggressive convolution.65 rA c s� t � �� t|j|jddd|jd�| _tdd|jd�| _ |jd ur.|j�66� dkr.t � � | _
nt jdd d67�| _
|jd krDt||jd �| _nt � � | _|j| _d S )
Nr r �:�0�yE>)r r r( r r )r r rW r )r r r r r. )r rK �
preactivation�rB )r$ r% r rB �embedding_size�global_padding� convolutionrQ Zembedding_dynamic_padding�pooler�upperr r@ r' Z
ConstantPad2d�68layer_typer: �norm�r) rA r+ r r r% � s" 69� 7071 zBitEmbeddings.__init__� pixel_valuesr c C sH |j d }|| jkrtd��| �|�}| �|�}| �|�}| �|�}|S )Nr zeMake sure that the channel dimension of the pixel values match with the one set in the configuration.)�shaperB �72ValueErrorr` r' rd ra )r) rf rB Z embeddingr r r r3 s 7374�75767778zBitEmbeddings.forward) r5 r6 r7 r8 r r% r r3 r9 r r r+ r rZ � s rZ r. FrN � drop_probr/ c C sd |dks|s| S d| }| j d fd| jd }|tj|| j| jd� }|�� | �|�| }|S )aF 79 Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).80 81 Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks,82 however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper...83 See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the84 layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the85 argument.86 r. r r )r )�dtype�device)rg �ndim�torchZrandrj rk Zfloor_�div)rN ri r/ Z keep_probrg Z
random_tensor�outputr r r � drop_path s 87rp c sT e Zd ZdZd dee ddf� fdd�
Zdejdejfdd �Z de88fd89d �Z � Z S )
� BitDropPathzXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).Nri r c s t � �� || _d S rD )r$ r% ri )r) ri r+ r r r% / s 9091zBitDropPath.__init__rY c C s t || j| j�S rD )rp ri r/ rX r r r r3 3 � zBitDropPath.forwardc C s d| j � �S )Nzp=)ri )r) r r r �92extra_repr6 s zBitDropPath.extra_reprrD )
r5 r6 r7 r8 r �floatr% rm r r3 r rs r9 r r r+ r rq , s93 rq � c C s: |}t |t| |d �| | �}|d| k r||7 }|S )Nr g�������?)rE rJ )rK ZdivisorZ min_value� new_valuer r r �make_div: s94 rw c �: e Zd ZdZ d � fdd� Zd d95� Z� ZS ) � BitPreActivationBottleneckLayera Pre-activation (v2) bottleneck block.96 Follows the implementation of "Identity Mappings in Deep Residual Networks":97 https://github.com/KaimingHe/resnet-1k-layers/blob/master/resnet-pre-act.lua98 99 Except it puts the stride on 3x3 conv when available.100 N� �?r r. Fc s� t � �� |p|}|p |}t|| �} |101r t||||dd�| _nd | _t||�| _t|| dd|jd�| _ t|| d�| _102t| | d||d|jd�| _ t|| �| _ t| |dd|jd�| _
| d krdt| �| _d S t�� | _d S )103NT�r �preactr r[ �r( r r] r )r r" r( r r )r$ r% rw �BitDownsampleConv�104downsampler: �norm1r r_ �conv1�norm2�conv2�norm3�conv3rq r r@ rp ) r) rA � in_channelsr* � bottle_ratior r �first_dilationr" �drop_path_rate�is_first_layerZ mid_channelsr+ r r r% J s, 105
106� � $z(BitPreActivationBottleneckLayer.__init__c C s^ | � |�}|}| jd ur| �|�}| �|�}| �| �|��}| �| �|��}| �|�}|| S rD )r� r r� r� r� r� r� rp )r) rY Zhidden_states_preact�shortcutr r r r3 v s 107108109110111z'BitPreActivationBottleneckLayer.forward�Nrz r r Nr r. Fr4 r r r+ r ry B s �,ry c rx ) �BitBottleneckLayerz\Non Pre-activation bottleneck block, equivalent to V1.5/V1b bottleneck. Used for ViT Hybrid.Nrz r r. Fc 112 s� t � �� |p|}|p |}t|| �} |113r t||||dd�| _nd | _t|| dd|jd�| _t|| d�| _ t| | d|||d|jd�| _114t|| d�| _ t| |dd|jd�| _ t||dd �| _
| d115kret| �nt�� | _t|j | _d S ) NFr{ r r[ r} r] r )r r r" r( r �rB rC r )r$ r% rw r~ r r r_ r� r: r� r� r� r� r� rq r r@ rp r r> r? ) r) rA r� r* r� r r r� r" r� r� Zmid_chsr+ r r r% � s< 116
117��118zBitBottleneckLayer.__init__c C sp |}| j d ur | � |�}| �|�}| �|�}| �|�}| �|�}| �|�}| �|�}| �|�}| �|| �}|S rD ) r r� r� r� r� r� r� rp r? )r) rY r� r r r r3 � s 119120121122123124125126127zBitBottleneckLayer.forwardr� r4 r r r+ r r� � s �1r� c s* e Zd Z d� fdd� Zdd� Z� ZS )r~ r Tc sH t � �� t||d|d|jd�| _|rt�� | _d S t||dd�| _d S )Nr r[ )r r( r Fr� ) r$ r% r r_ �convr r@ r: rd )r) rA r� r* r r| r+ r r r% � s 128��� �zBitDownsampleConv.__init__c C s | � | �|��S rD )rd r� )r) rH r r r r3 � rr zBitDownsampleConv.forward)r T)r5 r6 r7 r% r3 r9 r r r+ r r~ � s129 �r~ c s@ e Zd ZdZ d � fdd� Zdd� Zded efd130d �Z� ZS )
�BitStagez7131 A ResNet v2 stage composed by stacked layers.132 rz Nc s� t � �� |dv r dnd} |jdkrt}133nt}134|} t�� | _t|�D ]$} | � | ||�\}}
}| j�135t | �|136|| ||||| |
|d� � |} |} q"d S )N)r r r r Z137bottleneck)r r r� r� r� r� ) r$ r% rc r� ry r �138Sequential�layers�range� _get_updated_hyperparameters�139add_moduler )r) rA r� r* r r �depthr� �
layer_dropoutr� Z layer_cls�prev_chs� layer_idxr� r� r+ r r r% � s8 140 141142 143����zBitStage.__init__c C s0 |r|| }nd}|dkrd}|dk}|||fS )zt144 Get the new hyper-parameters with respect to the previous ones and the index of the current layer.145 r. r r r )r) r� r r� r� r� r r r r� s 146147z%BitStage._get_updated_hyperparametersrN r c C s$ |}t | j�D ]\}}||�}q|S rD )� enumerater� )r) rN r2 �_�layerr r r r3 ) s 148zBitStage.forward)rz N) r5 r6 r7 r8 r% r� r r3 r9 r r r+ r r� � s �.r� c sH e Zd Zdef� fdd� Zdd� Z dded ed149ed efd d
�Z � Z150S )�151BitEncoderrA c
152 s� t � �� t�g �| _|j}d}d}dd� t�t� d|j153t |j ����
|j �D �}tt|j |j|��D ]-\}\}}} | �|||||�\}154} }t|||155| ||| d�} |156}|| 9 }| j�t|�| � q3d S )N� r c S s g | ]}|� � �qS r )�tolist)�.0rH r r r �157<listcomp>; s ��z'BitEncoder.__init__.<locals>.<listcomp>r )r r r� r� )r$ r% r Z158ModuleList�stagesr^ rm r �npZlinspacer� �sumZdepths�splitr� �zip� hidden_sizesr� r� r� r )
r) rA r� �current_strider Zlayer_dropouts� stage_idxZ
current_depth�current_hidden_sizer� r* r �stager+ r r r% 1 s6 159 "��160161��162�zBitEncoder.__init__c C s> t ||j �}|dkr
dnd}||jkr||9 }d}|||fS )Nr r r )rw Z width_factorZ
output_stride)r) r� r� r� r rA r* r r r r r� W s 163164z'BitEncoder._get_updated_hyperparametersFTr2 �output_hidden_states� return_dictr c C sb |rdnd }| j D ]
}|r||f }||�}q |r ||f }|s+tdd� ||fD ��S t||d�S )Nr c s s � | ] }|d ur|V qd S rD r )r� �vr r r � <genexpr>n s � z%BitEncoder.forward.<locals>.<genexpr>)�last_hidden_staterY )r� �tupler )r) r2 r� r� rY Z stage_moduler r r r3 _ s 165166167168�zBitEncoder.forward)FT) r5 r6 r7 r r% r� r �boolr r3 r9 r r r+ r r� 0 s & �����r� c @ s, e Zd ZU eed< dZdZdgZdd� ZdS )�BitPreTrainedModelrA �bitrf rZ c C s� t |tj�rtjj|jddd� d S t |tj�rMtjj|jt� d�d� |j169d urKtj� |j�\}}|dkr=dt� |� nd}tj� |j170| |� d S d S t |tj
tjf�rhtj�|jd� tj�|j171d� d S d S )NZfan_outZrelu)�modeZ nonlinearity� )�ar r )r r �Conv2d�initZkaiming_normal_r1 �LinearZkaiming_uniform_rF �sqrtr# Z _calculate_fan_in_and_fan_outZuniform_Z BatchNorm2d� GroupNormZ constant_)r) �moduleZfan_inr� �boundr r r �
_init_weights} s 172��z BitPreTrainedModel._init_weightsN) r5 r6 r7 r �__annotations__Zbase_model_prefixZmain_input_nameZ_no_split_modulesr� r r r r r� v s 173 r� c 174 sF e Zd Z� fdd�Ze d175dedee dee defdd ��Z � Z176S ) �BitModelc sd t � �|� || _t|�| _t|�| _|jdkr!t||j d d�nt177� � | _ t178�
d�| _| �� d S )Nr\ r- r] )r r )r$ r% rA rZ �embedderr� �encoderrc r: r� r r@ rd ZAdaptiveAvgPool2dra � post_initre r+ r r r% � s 179180181�� zBitModel.__init__Nrf r� r� r c C s� |d ur|n| j j}|d ur|n| j j}| �|�}| j|||d�}|d }| �|�}| �|�}|s;||f|dd � S t|||jd�S )N�r� r� r r )r� �
pooler_outputrY ) rA r� �use_return_dictr� r� rd ra r rY )r) rf r� r� Zembedding_outputZencoder_outputsr� �
pooled_outputr r r r3 � s" �182�183184�zBitModel.forward�NN) r5 r6 r7 r% r r r r� r r3 r9 r r r+ r r� � s �����r� z�185 BiT Model with an image classification head on top (a linear layer on top of the pooled features), e.g. for186 ImageNet.187 )Z custom_introc s\ e Zd Z� fdd�Ze d deej deej dee dee de188f189d d190��Z � Z S ) �BitForImageClassificationc s^ t � �|� |j| _t|�| _t�t�� |jdkr#t�|j d |j�nt�191� �| _ | � � d S )Nr r- )
r$ r% Z192num_labelsr� r� r r� ZFlattenr� r� r@ �193classifierr� re r+ r r r% � s 194$� z"BitForImageClassification.__init__Nrf �labelsr� r� r c 195 C s� |dur|n| j j}| j|||d�}|r|jn|d }| �|�}d}|dur.| �||| j �}|sD|f|dd� } |durB|f| S | S t|||jd�S )a0 196 labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*):197 Labels for computing the image classification/regression loss. Indices should be in `[0, ...,198 config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy).199 Nr� r r )�loss�logitsrY )rA r� r� r� r� Z
loss_functionr200 rY )201r) rf r� r� r� �outputsr� r� r� ro r r r r3 � s
202z!BitForImageClassification.forward)NNNN)
r5 r6 r7 r% r r rm Z FloatTensorZ203LongTensorr� r204 r3 r9 r r r+ r r� � s$ ������r� zL205 BiT backbone, to be used with frameworks like DETR and MaskFormer.206 c 207 sJ e Zd ZdZ� fdd�Ze d dedee dee de fd d208��Z209� Z S ) � BitBackboneFc s> t � �|� t � �|� t|�| _|jg|j | _| �� d S rD ) r$ r% Z_init_backboner� r� r^ r� Z num_featuresr� re r+ r r r% � s210 211 zBitBackbone.__init__Nrf r� r� r c 212 C s� |dur|n| j j}|dur|n| j j}| j|ddd�}|j}d}t| j�D ]\}}|| jv r6||| f7 }q&|sF|f} |rD| |jf7 } | S t||rP|jdd�S ddd�S )aN 213 Examples:214 215 ```python216 >>> from transformers import AutoImageProcessor, AutoBackbone217 >>> import torch218 >>> from PIL import Image219 >>> import requests220 221 >>> url = "http://images.cocodataset.org/val2017/000000039769.jpg"222 >>> image = Image.open(requests.get(url, stream=True).raw)223 224 >>> processor = AutoImageProcessor.from_pretrained("google/bit-50")225 >>> model = AutoBackbone.from_pretrained("google/bit-50")226 227 >>> inputs = processor(image, return_tensors="pt")228 >>> outputs = model(**inputs)229 ```NTr� r )� feature_mapsrY Z230attentions) rA r� r� r� rY r� Z stage_namesZ out_featuresr )231r) rf r� r� r� rY r� �idxr� ro r r r r3 s. �232� ��zBitBackbone.forwardr� ) r5 r6 r7 Zhas_attentionsr% r r r r� r r3 r9 r r r+ r r� � s 233�����r� )r� r� r� r� )Nr r r )r. F)ru )7r8 rS rF �typingr �numpyr� rm r r Z activationsr Zmodeling_outputsr r r r234 Zmodeling_utilsr �utilsr r
Zutils.backbone_utilsr Zconfiguration_bitr Z235get_loggerr5 �loggerr� r� r r� r r� r: �Moduler& Z MaxPool2drQ rZ rt rp rq rw ry r� r~ r� r� r� r� r� r� �__all__r r r r �<module> sR 236)03 3237DIJF1�.� >