Mjolnir65/FasterRCNN

1B

2��hoV�3@sX
dd
lZddl
mZ
ddlmZmZmZmZ
dd
lZddlm	Z	m4Z5
ddlmZ

ddlmZmZmZmZ
Gdd�d�Zejje6e7e8e9d	�d10d��
ZGdd
�d
�ZGdd�d�ZGdd�d�ZGdd�de�Ze	jed
d�dd�Ze	jeeefeed�dd�Zejj e11ed�dd��
Z!e12eeed�dd�Z"d#e#ee13e14e15eee#efe16d �d!d"�Z$d
S)$�N)
�OrderedDict)�Dict�List�Optional�Tuple)�nn�Tensor)
�17functional)�complete_box_iou_loss�distance_box_iou_loss�FrozenBatchNorm2d�generalized_box_iou_lossc@sFeZ
dZd
Zeedd�dd�Zeee	eeeefd�dd�Z18dS)	�BalancedPositiveNegativeSamplerz`19    This class samples batches, ensuring that they contain a fixed proportion of positives20    N)�batch_size_per_image�positive_fraction�returncCs|
|_||_
d
S)z�21        Args:22            batch_size_per_image (int): number of elements to be selected per image23            positive_fraction (float): percentage of positive elements per batch24        N)rr)�selfrr�r�4/home/rohit/Desktop/Sem6/CV/q1.2/detection/_utils.py�__init__s
z(BalancedPositiveNegativeSampler.__init__)�matched_idxsrcCs
g}g}x�|
D]�}t�
|d
k�
d}t�
|dk�
d}t|j|j�
}t|��|�}|j|}t|��|�}tj|��|jd�d|�}	tj|��|jd�d|�}25||	}||26}tj	|tj27d�}
tj	|tj28d�}d
|
|<d
||<|�|
�

|�|�

qW||fS)a29        Args:30            matched_idxs: list of tensors containing -1, 0 or positive values.31                Each tensor corresponds to a specific image.32                -1 values are ignored, 0 are considered as negatives and > 0 as33                positives.34 35        Returns:36            pos_idx (list[tensor])37            neg_idx (list[tensor])38 39        Returns two lists of binary masks for each image.40        The first list contains the positive elements that were selected,41        and the second list the negative example.42        �
r
)
�deviceN)
�dtype)�torch�where�intrr�min�numel�randpermr�43zeros_like�uint8�append)rr�pos_idx�neg_idx�matched_idxs_per_image�positive�negative�num_pos�num_neg�perm1�perm2�pos_idx_per_image�neg_idx_per_image�pos_idx_per_image_mask�neg_idx_per_image_maskrrr�__call__s(

44


45



46
z(BalancedPositiveNegativeSampler.__call__)�__name__�47__module__�__qualname__�__doc__r�floatrrrrr0rrrrrs	r)�reference_boxes�	proposals�weightsrcCs~
|d
}|d}|d}|d}|
dd�d
f�d�
}|
dd�df�d�
}|
dd�df�d�
}	|
dd�df�d�
}48|dd�d
f�d�
}|dd�df�d�
}|dd�df�d�
}
|dd�df�d�
}|	|}|49|}|d|}|d|}|
|}||}|d|}|d|}||||}||||}|t
�||�
}|t
�||�
}t
j||||fdd�}|S)z�50    Encode a set of proposals with respect to some51    reference boxes52 53    Args:54        reference_boxes (Tensor): reference boxes55        proposals (Tensor): boxes to be encoded56        weights (Tensor[4]): the weights for ``(x, y, w, h)``57    r
r��Ng�?)
�dim)�	unsqueezer�log�cat)r6r7r8�wx�wy�ww�wh�proposals_x1�proposals_y1�proposals_x2�proposals_y2�reference_boxes_x1�reference_boxes_y1�reference_boxes_x2�reference_boxes_y2�	ex_widths�58ex_heights�ex_ctr_x�ex_ctr_y�	gt_widths�59gt_heights�gt_ctr_x�gt_ctr_y�60targets_dx�61targets_dy�62targets_dw�63targets_dh�targetsrrr�encode_boxesJs4



















rXc@s�eZ
dZd
Ze�d�
f
eeeeefedd�dd�Ze	e64e	e65e	e66d�dd	�Ze67e68e69d�d70d�Ze71e	e72e73d�d
d�Z
e74e75e76d�dd�ZdS)�BoxCoderz~77    This class encodes and decodes a set of bounding boxes into78    the representation used for training the regressors.79    g@O@N)r8�bbox_xform_cliprcCs|
|_||_
d
S)za80        Args:81            weights (4-element tuple)82            bbox_xform_clip (float)83        N)r8rZ)rr8rZrrrr�s
zBoxCoder.__init__)r6r7rcCsBd
d�|
D�
}tj
|
dd�}
tj
|dd�}|�|
|�}|�|d�S)Nc
Ssg|]}
t|
�
�qSr)
�len)�.0�
brrr�84<listcomp>�sz#BoxCoder.encode.<locals>.<listcomp>r
)
r;)rr>�
encode_single�split)rr6r7�boxes_per_imagerWrrr�encode�s85




zBoxCoder.encodecCs.|
j}|
j
}tj|j||d
�}t|
||�}|S)z�86        Encode a set of proposals with respect to some87        reference boxes88 89        Args:90            reference_boxes (Tensor): reference boxes91            proposals (Tensor): boxes to be encoded92        )rr)rrr�	as_tensorr8rX)rr6r7rrr8rWrrrr_�s93	


zBoxCoder.encode_single)�	rel_codes�boxesrcCs�t�
t|ttf�d
�
t�
t|
tj�d�
dd�|D�
}tj|dd�}d}x|D]}||7}qPW|dkrt|
�|d�}
|�|
|�}|dkr�|�|dd�}|S)	Nz2This function expects boxes of type list or tuple.z5This function expects rel_codes of type torch.Tensor.c
Ssg|]}
|
�d�
�qS)
r
)
�size)r\r]rrrr^�sz#BoxCoder.decode.<locals>.<listcomp>r
)
r;������)	r�_assert�94isinstance�list�tuplerr>�reshape�
decode_single)rrdrera�concat_boxes�box_sum�val�95pred_boxesrrr�decode�s"



96



97






zBoxCoder.decodecCs�
|�|
j
�
}|d
d
�df|d
d
�df}|d
d
�df|d
d
�df}|d
d
�dfd|}|d
d
�dfd|}|j\}}}	}98|
d
d
�dd
d�f|}|
d
d
�dd
d�f|}|
d
d
�dd
d�f|	}
|
d
d
�dd
d�f|99}tj|
|jd�}
tj||jd�}||d
d
�d
f|d
d
�d
f}||d
d
�d
f|d
d
�d
f}t�|
�
|d
d
�d
f}t�|�
|d
d
�d
f}tjd|j
|jd	�|}tjd|j
|jd	�|}||}||}||}||}tj	||||fdd100��101d�
}|S)z�102        From a set of original boxes and encoded relative box offsets,103        get the decoded boxes.104 105        Args:106            rel_codes (Tensor): encoded boxes107            boxes (Tensor): reference boxes.108        Nr9r
r:rg�?rh)
�max)rr)
r;)�torr8r�clamprZ�exp�tensorr�stack�flatten)rrdre�widths�heights�ctr_x�ctr_yr?r@rArB�dx�dy�dw�dh�109pred_ctr_x�110pred_ctr_y�pred_w�pred_h�c_to_c_h�c_to_c_w�pred_boxes1�pred_boxes2�pred_boxes3�pred_boxes4rrrrrrn�s0111 
 






$
$







zBoxCoder.decode_single)r1r2r3r4�mathr=rr5rrrrbr_rsrnrrrrrYzs
	rYc@sFeZ
dZd
Zd
edd�dd�Zeeed�dd	�Zeeed112�dd�ZdS)�BoxLinearCoderz�113    The linear box-to-box transform defined in FCOS. The transformation is parameterized114    by the distance from the center of (square) src box to 4 edges of the target box.115    TN)�normalize_by_sizercCs116|
|_d
S)zq117        Args:118            normalize_by_size (bool): normalize deltas by the size of src (anchor) boxes.119        N)
r�)rr�rrrr�szBoxLinearCoder.__init__)r6r7rc
Cs�d
|
d|
d}d
|
d|
d}||d}||d}|d|}|d|}tj
||||fdd�}	|jr�|
d|
d}120|
d|
d}tj
|121||122|fdd�}|	|}	|	S)aC
123        Encode a set of proposals with respect to some reference boxes124 125        Args:126            reference_boxes (Tensor): reference boxes127            proposals (Tensor): boxes to be encoded128 129        Returns:130            Tensor: the encoded relative box offsets that can be used to131            decode the boxes.132 133        g�?).r
).r9).r).r:rg)
r;)rryr�)
rr6r7�reference_boxes_ctr_x�reference_boxes_ctr_y�target_l�target_t�target_r�target_brW�reference_boxes_w�reference_boxes_h�reference_boxes_sizerrrrb�s








zBoxLinearCoder.encode)rdrerc
Cs�|j|
j
d
�
}d|d|d}d|d|d}|jrz|d|d}|d|d}tj||||fdd�}|
|}
||
d}||
d}	||
d}134||
d}tj||	|135|fdd�}|S)	a�
136        From a set of original boxes and encoded relative box offsets,137        get the decoded boxes.138 139        Args:140            rel_codes (Tensor): encoded boxes141            boxes (Tensor): reference boxes.142 143        Returns:144            Tensor: the predicted boxes with the encoded relative box offsets.145 146        .. note::147            This method assumes that ``rel_codes`` and ``boxes`` have same size for 0th dimension. i.e. ``len(rel_codes) == len(boxes)``.148 149        )
rg�?).r
).r9).r).r:rg)
r;)rurr�rry)
rrdrer}r~�boxes_w�boxes_h�
list_box_sizer�r�r�r�rrrrrrs
s







zBoxLinearCoder.decode)
T)	r1r2r3r4�boolrrrbrsrrrrr��s#r�c@s\eZ
dZd
ZdZdZeed�Zdeee	dd�dd	�Z150eed151�dd�Zeeedd
�dd�Z
dS)�Matchera�152    This class assigns to each predicted "element" (e.g., a box) a ground-truth153    element. Each predicted element will have exactly zero or one matches; each154    ground-truth element may be assigned to zero or more predicted elements.155 156    Matching is based on the MxN match_quality_matrix, that characterizes how well157    each (ground-truth, predicted)-pair match. For example, if the elements are158    boxes, the matrix may contain box IoU overlap values.159 160    The matcher returns a tensor of size N containing the index of the ground-truth161    element m that matches to prediction n. If there is no match, a negative value162    is returned.163    rg�����)�BELOW_LOW_THRESHOLD�BETWEEN_THRESHOLDSFN)�high_threshold�
low_threshold�allow_low_quality_matchesrcCs2d
|_d|_
t�||
k
d�
|
|_||_||_dS)a�164        Args:165            high_threshold (float): quality values greater than or equal to166                this value are candidate matches.167            low_threshold (float): a lower quality threshold used to stratify168                matches into three levels:169                1) matches >= high_threshold170                2) BETWEEN_THRESHOLDS matches in [low_threshold, high_threshold)171                3) BELOW_LOW_THRESHOLD matches in [0, low_threshold)172            allow_low_quality_matches (bool): if True, produce additional matches173                for predictions that have only low-quality match candidates. See174                set_low_quality_matches_ for more details.175        rgr�z)low_threshold should be <= high_thresholdN)r�r�rrir�r�r�)rr�r�r�rrrrQ
s




zMatcher.__init__)�match_quality_matrixrcCs�|
��d
kr,|
j
d
d
kr$td�
�
ntd�
�
|
jd
d�
\}}|jrL|��}nd}||jk}||jk||jk@}|j||<|j	||<|jr�|dkr�t176�dd�
n|�|||
�
|S)a�
177        Args:178            match_quality_matrix (Tensor[float]): an MxN tensor, containing the179            pairwise quality between M ground-truth elements and N predicted elements.180 181        Returns:182            matches (Tensor[int64]): an N tensor where N[i] is a matched gt in183            [0, M - 1] or a negative value indicating that prediction i could not184            be matched.185        r
zENo ground-truth boxes available for one of the images during trainingzANo proposal boxes available for one of the images during training)
r;NFzall_matches should not be None)
r�shape�186ValueErrorrtr��cloner�r�r�r�rri�set_low_quality_matches_)rr��matched_vals�matches�all_matches�below_low_threshold�between_thresholdsrrrr0f
s"
187

188189

190
191

zMatcher.__call__)r�r�r�rcCsB|jd
d�
\}}t
�||dd�dfk�
}|d
}|||
|<dS)ax
192        Produce additional matches for predictions that have only low-quality matches.193        Specifically, for each ground-truth find the set of predictions that have194        maximum overlap with it (including ties); for each prediction in that set, if195        it is unmatched, then match it to the ground-truth with which it has the highest196        quality value.197        r)
r;N)rtrr)rr�r�r��highest_quality_foreach_gt�
_� gt_pred_pairs_of_highest_quality�pred_inds_to_updaterrrr��
s	
z Matcher.set_low_quality_matches_)
F)r1r2r3r4r�r�r�__annotations__r5r�rrr0r�rrrrr�:
s


(r�cs8eZ
dZed
d��f
dd�Zeed��f
dd�Z�ZS)�198SSDMatcherN)�	thresholdrcst�j
|
|
d
d�
dS)NF)
r�)�superr)rr�)
�	__class__rrr�
s
zSSDMatcher.__init__)r�rcs>t��
|
�
}|
jd
d�
\}}tj|�d�
tj|jd�||<|S)Nr)
r;r
)rr)r�r0rtr�arangerf�int64r)rr�r�r��highest_quality_pred_foreach_gt)
r�rrr0�
s199


zSSDMatcher.__call__)r1r2r3r5rrr0�
__classcell__rr)
r�rr��
s
r�)�model�epsrcCs&x |��D]}t
|t�r200|
|_q201Wd
S)a�
202    This method overwrites the default eps values of all the203    FrozenBatchNorm2d layers of the model with the provided value.204    This is necessary to address the BC-breaking change introduced205    by the bug-fix at pytorch/vision#2933. The overwrite is applied206    only when the pretrained weights are loaded to maintain compatibility207    with previous versions.208 209    Args:210        model (nn.Module): The model on which we perform the overwrite.211        eps (float): The new value of eps.212    N)�modulesrjrr�)r�r��modulerrr�
overwrite_eps�
s

213
r�)r�rfrc	Cs�|j}|�
�
t���f
t|���
j}tjd
d|
d
|
df|d�}||�
}t|tj	�rft214d|fg
�
}dd�|��D�
}WdQRX|r�|��
|S)	a�
215    This method retrieves the number of output channels of a specific model.216 217    Args:218        model (nn.Module): The model for which we estimate the out_channels.219            It should return a single Tensor or an OrderedDict[Tensor].220        size (Tuple[int, int]): The size (wxh) of the input.221 222    Returns:223        out_channels (List[int]): A list of the output channels of the model.224    rr:r
)
r�
0c
Ssg|]}
|
�d�
�qS)
r)
rf)r\�
xrrrr^�
sz)retrieve_out_channels.<locals>.<listcomp>N)
�training�evalr�no_grad�next�225parametersr�zerosrjrr�values�train)r�rf�in_trainingr�tmp_img�features�out_channelsrrr�retrieve_out_channels�
s
226





r�)�
vrc


Cs|S)
Nr)
r�rrr�_fake_cast_onnx�
sr�)�input�	orig_kval�axisrcCsZtj
��st|
|�|�
�St�|�
|�d
�
}t�t�tj|
g
|j	d�|fd
��
}t227|�
S)ay228    ONNX spec requires the k-value to be less than or equal to the number of inputs along229    provided dim. Certain models use the number of elements along a particular axis instead of K230    if K exceeds the number of elements along that axis. Previously, python's min() function was231    used to determine whether to use the provided k-value or the specified dim axis value.232 233    However, in cases where the model is being exported in tracing mode, python min() is234    static causing the model to be traced incorrectly and eventually fail at the topk node.235    In order to avoid this situation, in tracing mode, torch.min() is used instead.236 237    Args:238        input (Tensor): The original input tensor.239        orig_kval (int): The provided k-value.240        axis(int): Axis along which we retrieve the input size.241 242    Returns:243        min_kval (int): Appropriately selected k-value.244    r
)
r)r�jit�245is_tracingrrf�_shape_as_tensorr<r>rxrr�)r�r�r��axis_dim_val�min_kvalrrr�	_topk_min�
s246247


$
r�)�type�	box_coder�anchors_per_image�matched_gt_boxes_per_image�bbox_regression_per_image�cnfrc248Cs�t�
|d
kd|���
|dkr:|
�||�}tj||dd�S|dkr||
�||�}|dk	rfd|krf|dnd}tj||d|d	�S|
�||�}|dk	r�d249|kr�|d250nd}	|dkr�t||d|	d
�S|dkr�t||d|	d
�St	||d|	d
�SdS)N)�l1�	smooth_l1�ciou�diou�giouzUnsupported loss: r��sum)
�	reductionr��betag�?)r�r�r�gH�����z>r�)r�r�r�)251rrir_�
F�l1_loss�smooth_l1_lossrnr252rr
)253r�r�r�r�r�r��target_regressionr��bbox_per_imager�rrr�	_box_losss










r�)
N)%r��collectionsr�typingrrrrrrr�torch.nnr	r��torchvision.opsr254rrr
rr��_script_if_tracingrXrYr�r�r��Moduler5r�rr��unusedr�r��strr�rrrr�<module>
s(




?
/iWj