Mjolnir65/FasterRCNN

1B

2��h�@�3@s�
dd
lm
Z
mZmZmZmZmZ
ddlZddlm	
m4Z
ddlm	Z	
ddlm
Z

ddlmZ
ddlmZ
ddlmZmZmZ
ddlmZ
dd	lmZmZ
dd5lmZmZ
ddlmZ
dd
l m!Z!
ddl"m#Z#m$Z$
ddl%m&Z&
ddl'm(Z(
ddl)m*Z*m+Z+
ddl,m-Z-
dddgZ.dd�Z/Gdd�de&�Z0Gdd�de	j1�Z2Gdd�de	j3�Z4Gdd�de	j1�Z5edd �Z6Gd!d�de�Z7ed"e7j8fd#ej9fd$�dd%dej9dd&�ee7e:ee;eeee;e
e0d'�d(d��
Z<dS))�)�Any�Callable�List�Optional�Tuple�UnionN)
�nn)
�MultiScaleRoIAlign)
�misc)
�ObjectDetection)�register_model�Weights�WeightsEnum)
�_COCO_CATEGORIES)�_ovewrite_value_param�handle_legacy_interface)�resnet50�ResNet50_Weights�
)
�
overwrite_eps)
�AnchorGenerator)�_resnet_fpn_extractor�_validate_trainable_layers)
�GeneralizedRCNN)
�RoIHeads)�RegionProposalNetwork�RPNHead)
�GeneralizedRCNNTransform�6FasterRCNN�FasterRCNN_ResNet50_FPN_Weights�fasterrcnn_resnet50_fpncCsd
}dt|�
}
t
||
�S)N))
� )
�@)
�)
�
)
i)
)g�?g�?g@)�lenr)�anchor_sizes�
aspect_ratios�r(�9/home/rohit/Desktop/Sem6/CV/q1.2/detection/faster_rcnn.py�_default_anchorgens


r*cs"eZ
dZd
Zd�f
dd�	Z�ZS)ra�7    Implements Faster R-CNN.8 9    The input to the model is expected to be a list of tensors, each of shape [C, H, W], one for each10    image, and should be in 0-1 range. Different images can have different sizes.11 12    The behavior of the model changes depending on if it is in training or evaluation mode.13 14    During training, the model expects both the input tensors and targets (list of dictionary),15    containing:16        - boxes (``FloatTensor[N, 4]``): the ground-truth boxes in ``[x1, y1, x2, y2]`` format, with17          ``0 <= x1 < x2 <= W`` and ``0 <= y1 < y2 <= H``.18        - labels (Int64Tensor[N]): the class label for each ground-truth box19 20    The model returns a Dict[Tensor] during training, containing the classification and regression21    losses for both the RPN and the R-CNN.22 23    During inference, the model requires only the input tensors, and returns the post-processed24    predictions as a List[Dict[Tensor]], one for each input image. The fields of the Dict are as25    follows:26        - boxes (``FloatTensor[N, 4]``): the predicted boxes in ``[x1, y1, x2, y2]`` format, with27          ``0 <= x1 < x2 <= W`` and ``0 <= y1 < y2 <= H``.28        - labels (Int64Tensor[N]): the predicted labels for each image29        - scores (Tensor[N]): the scores or each prediction30 31    Args:32        backbone (nn.Module): the network used to compute the features for the model.33            It should contain an out_channels attribute, which indicates the number of output34            channels that each feature map has (and it should be the same for all feature maps).35            The backbone should return a single Tensor or and OrderedDict[Tensor].36        num_classes (int): number of output classes of the model (including the background).37            If box_predictor is specified, num_classes should be None.38        min_size (int): minimum size of the image to be rescaled before feeding it to the backbone39        max_size (int): maximum size of the image to be rescaled before feeding it to the backbone40        image_mean (Tuple[float, float, float]): mean values used for input normalization.41            They are generally the mean values of the dataset on which the backbone has been trained42            on43        image_std (Tuple[float, float, float]): std values used for input normalization.44            They are generally the std values of the dataset on which the backbone has been trained on45        rpn_anchor_generator (AnchorGenerator): module that generates the anchors for a set of feature46            maps.47        rpn_head (nn.Module): module that computes the objectness and regression deltas from the RPN48        rpn_pre_nms_top_n_train (int): number of proposals to keep before applying NMS during training49        rpn_pre_nms_top_n_test (int): number of proposals to keep before applying NMS during testing50        rpn_post_nms_top_n_train (int): number of proposals to keep after applying NMS during training51        rpn_post_nms_top_n_test (int): number of proposals to keep after applying NMS during testing52        rpn_nms_thresh (float): NMS threshold used for postprocessing the RPN proposals53        rpn_fg_iou_thresh (float): minimum IoU between the anchor and the GT box so that they can be54            considered as positive during training of the RPN.55        rpn_bg_iou_thresh (float): maximum IoU between the anchor and the GT box so that they can be56            considered as negative during training of the RPN.57        rpn_batch_size_per_image (int): number of anchors that are sampled during training of the RPN58            for computing the loss59        rpn_positive_fraction (float): proportion of positive anchors in a mini-batch during training60            of the RPN61        rpn_score_thresh (float): only return proposals with an objectness score greater than rpn_score_thresh62        box_roi_pool (MultiScaleRoIAlign): the module which crops and resizes the feature maps in63            the locations indicated by the bounding boxes64        box_head (nn.Module): module that takes the cropped feature maps as input65        box_predictor (nn.Module): module that takes the output of box_head and returns the66            classification logits and box regression deltas.67        box_score_thresh (float): during inference, only return proposals with a classification score68            greater than box_score_thresh69        box_nms_thresh (float): NMS threshold for the prediction head. Used during inference70        box_detections_per_img (int): maximum number of detections per image, for all classes.71        box_fg_iou_thresh (float): minimum IoU between the proposals and the GT box so that they can be72            considered as positive during training of the classification head73        box_bg_iou_thresh (float): maximum IoU between the proposals and the GT box so that they can be74            considered as negative during training of the classification head75        box_batch_size_per_image (int): number of proposals that are sampled during training of the76            classification head77        box_positive_fraction (float): proportion of positive proposals in a mini-batch during training78            of the classification head79        bbox_reg_weights (Tuple[float, float, float, float]): weights for the encoding/decoding of the80            bounding boxes81    N� �5�����ffffff�?�333333�?r$��?�皙�����?�d���?c's�
t|
d
�st
d�
�
t|ttd�
f�s6tdt|�
���
�
t|ttd�
f�sZtdt|�
���
�
|dk	rt|dk	r�t
d�
�
n|dkr�t
d�
�
|
j}|dkr�t�}|dkr�t	||�82�d�}t|	|83d�} t||d�}!t||||||| |!|
|d	�84}"|dk�
rtd85ddd
gddd�}|dk�
r0|j
d}#d}$t||#d|$�}|dk�
rHd}$t|$|�}t|||||||||||�}%|dk�
rxdddg}|dk�
r�dddg}t||||f|�
}&t��|
|"|%|&�
dS)N�out_channelsz�backbone should contain an attribute out_channels specifying the number of output channels (assumed to be the same for all the levels)zJrpn_anchor_generator should be of type AnchorGenerator or None instead of zEbox_roi_pool should be of type MultiScaleRoIAlign or None instead of z:num_classes should be None when box_predictor is specifiedzBnum_classes should not be None when box_predictor is not specifiedr
)�training�testing)
�score_thresh�
0�
1�
2�
3��)�
featmap_names�output_size�sampling_ratioig86ףp=87�?g�v��/�?g�C�l���?gZd;�O�?gy�&1��?g�������?)�hasattr�88ValueError�89isinstancer�type�	TypeErrorr	r7r*r�num_anchors_per_location�dictrrB�90TwoMLPHead�FastRCNNPredictorrr�super�__init__)'�self�backbone�num_classes�min_size�max_size�91image_mean�	image_std�rpn_anchor_generator�rpn_head�rpn_pre_nms_top_n_train�rpn_pre_nms_top_n_test�rpn_post_nms_top_n_train�rpn_post_nms_top_n_test�rpn_nms_thresh�rpn_fg_iou_thresh�rpn_bg_iou_thresh�rpn_batch_size_per_image�rpn_positive_fraction�rpn_score_thresh�box_roi_pool�box_head�
box_predictor�box_score_thresh�box_nms_thresh�box_detections_per_img�box_fg_iou_thresh�box_bg_iou_thresh�box_batch_size_per_image�box_positive_fraction�bbox_reg_weights�kwargsr7�rpn_pre_nms_top_n�rpn_post_nms_top_n�rpn�92resolution�representation_size�	roi_heads�	transform)
�	__class__r(r)rNssv%93







94














95
96
97

98

99









100
101
102
103
zFasterRCNN.__init__)Nr+r,NNNNr-r.r-r.r/r/r0r$r1r2NNNr3r1r4r1r1r5r6N)�__name__�104__module__�__qualname__�__doc__rN�
__classcell__r(r()
rur)r%s:L























cs(eZ
dZd
Z�f
dd�Zdd�Z�ZS)rKz�105    Standard heads for FPN-based models106 107    Args:108        in_channels (int): number of input channels109        representation_size (int): size of the intermediate representation110    cs*t��
�
t�|
|�|_t�||�|_dS)
N)rMrNr�Linear�fc6�fc7)rO�in_channelsrr)
rur(r)rN�s
111
zTwoMLPHead.__init__cCs0|
jd
d�
}
t
�|�|
�
�
}
t
�|�|
�
�
}
|
S)Nr)
�	start_dim)�flatten�
F�relur|r})rO�
xr(r(r)�forward�s

zTwoMLPHead.forward)rvrwrxryrNr�rzr(r()
rur)rK�srKcsJeZ
dZdeeeefeeeeeedej	fd��f
dd�
Z112�ZS)�FastRCNNConvFCHeadN.)�113input_size�conv_layers�	fc_layers�114norm_layercs�|
\}}}g}|}	x&|D]}115|�t
j|	|116|d
��

|117}	qW|�t���

|	||}	x4|D],}118|�t�|	|119��

|�tjdd�
�

|120}	qZWt�j|�
xH|�	�D]<}t121|tj�r�tjj
|jddd�
|jdk	r�tj�|j�

q�WdS)aa
122        Args:123            input_size (Tuple[int, int, int]): the input size in CHW format.124            conv_layers (list): feature dimensions of each Convolution layer125            fc_layers (list): feature dimensions of each FCN layer126            norm_layer (callable, optional): Module specifying the normalization layer to use. Default: None127        )
r�T)
�inplace�fan_outr�)�mode�nonlinearityN)�append�misc_nn_ops�Conv2dNormActivationr�Flattenr{�ReLUrMrN�modulesrF�Conv2d�init�kaiming_normal_�weight�bias�zeros_)rOr�r�r�r�r~�	in_height�in_width�blocks�previous_channels�current_channels�layer)
rur(r)rN
s$128

129




130






131
zFastRCNNConvFCHead.__init__)
N)rvrwrxr�intrrrr�ModulerNrzr(r()
rur)r�
sr�cs*eZ
dZd
Zd�f
dd�	Zdd�Z�ZS)rLz�132    Standard classification + bounding box regression layers + theta133    for Fast R-CNN.134 135    Args:136        in_channels (int): number of input channels137        num_classes (int): number of output classes (including background)138    rcs@t��
�
t�|
|�|_t�|
|d
�|_t�|
d|�|_dS)N�r)rMrNrr{�	cls_score�	bbox_pred�139theta_pred)rOr~rQ�num_theta_bins)
rur(r)rN1
s
140


zFastRCNNPredictor.__init__cCsv|
��d
krBt
�t|
jdd��
ddgkdt|
jdd��
���
|
jdd�
}
|�|
�
}|�|
�
}|�|
�
}|||fS)Nr�r@rzPx has the wrong shape, expecting the last two dimensions to be [1,1] instead of )
r)	�dim�torch�_assert�list�shaper�r�r�r�)rOr��scores�bbox_deltasZtheta_predsr(r(r)r�7
s




141
142
143zFastRCNNPredictor.forward)
r)rvrwrxryrNr�rzr(r()
rur)rL'
srL)rr)�144categoriesrRc@s8eZ
dZed
eedddddi
i
ddd	d145��d�ZeZdS)
rzMhttps://download.pytorch.org/models/fasterrcnn_resnet50_fpn_coco-258fb6c6.pthi�"}z[https://github.com/pytorch/vision/tree/main/references/detection#faster-r-cnn-resnet-50-fpnzCOCO-val2017�box_mapg�B@g\���(�`@g�����c@zSThese weights were produced by following a similar training recipe as on the paper.)�146num_params�recipe�_metrics�_ops�147_file_size�_docs)�url�148transforms�metaN)rvrwrxr
r�_COMMON_META�COCO_V1�DEFAULTr(r(r(r)rJ
s







�149pretrained�pretrained_backbone)�weights�weights_backboneT)r��progressrQr��trainable_backbone_layers)r�r�rQr�r�rm�returnc150Ks
t�
|�
}t�
|�
}|dk	r8d}td
|t|jd�
�}n|dkrDd}|dk	pR|dk	}t||dd�}|rltjnt	j151}t||
|d�}t||�}t
|f
d
|i
|��
}	|dk	�
r|	j|j|
d�
dd	�
tj	jj|	jjjjd152dd�
tj	j�|	jjjjd
�
|tjk�
rt|	d�
|	S)NrQr��[��)r�r�r�)
r�F)
�strictr�r�)r�r�r
g)r�verifyrrr%r�rr��FrozenBatchNorm2dr�BatchNorm2drrr�load_state_dict�get_state_dictr�r�r�rsrdr�r��	constant_r�r�r)153r�r�rQr�r�rm�154is_trainedr�rP�modelr(r(r)r `
s(155
156






157
158




159)=�typingrrrrrrr�Ztorch.nn.functionalr�160functionalr�Ztorchvision.opsr	r161r�Ztorchvision.transforms._presetsrZtorchvision.models._apirr
rZtorchvision.models._metarZtorchvision.models._utilsrrZtorchvision.models.resnetrr�_utilsr�anchor_utilsr�backbone_utilsrr�generalized_rcnnrrsrrprrrtr�__all__r*rr�rK�162Sequentialr�rLr�rr��
IMAGENET1K_V1�boolr�r r(r(r(r)�<module>
sN 















F%