Apppublic
CraigDroke/pgt_toy_problem

sourceHugging Faceupdated 2y agoView on Hugging Face
1likes
root__pycache__plaus_functs.cpython-310.pyc
plaus_functs.cpython-310.pyc280 linesDownload Raw Back to __pycache__
1o

2e7�g@��@sFddlZddlZddlTddlmZmZmZddlZddlZddl	m3Zddl	mZddl
mZmZmZd7dd	�Zd8d4d�Zd9d
d�Zddl
mZd9dd�Zdd�Zdd�Zdd�Zdd�Zdd�Zdd�Zdd�Zd d!�Zd"d#�Zdd$lm Z d%d&�Z!d:d(d)�Z"d*d+�Z#d;d,d-�Z$d<d.d/�Z%		0		2d=d3d4�Z&Gd5d6�d6e'j(�Z)dS)>�N)�*)�normalize_tensor�overlay_mask�imshow)�Path)�non_max_suppression�	xyxy2xywh�scale_coordsFTcCs�|jt�dg�kr|jt�g�krt�|�����}nd}tjj|||dd�d}|r0t�|�}|r:tj	|ddd�}|ra|rMt�5|�}t�|�}	t�|�}6t
|�}|ra||�7�8}|||8|	7}|S)a�9    Compute the gradient of an image with respect to a given tensor.10 11    Args:12        img (torch.Tensor): The input image tensor.13        grad_wrt (torch.Tensor): The tensor with respect to which the gradient is computed.14        norm (bool, optional): Whether to normalize the gradient. Defaults to True.15        absolute (bool, optional): Whether to take the absolute values of the gradients. Defaults to True.16        grayscale (bool, optional): Whether to convert the gradient to grayscale. Defaults to True.17        keepmean (bool, optional): Whether to keep the mean value of the attribution map. Defaults to False.18 19    Returns:20        torch.Tensor: The computed attribution map.21 22    �NT��grad_outputs�create_graphr��keepdim)�shape�torch�Size�	ones_like�clone�detach�autograd�grad�abs�sum�mean�min�max�normalize_batch)�img�grad_wrt�norm�absolute�	grayscale�keepmean�grad_wrt_outputs�attribution_map�attmean�attmin�attmax�r)�./home/drokec87/pgt_toy_problem/plaus_functs.py�get_gradients."23��24252627r+c28Cs~t�|�}|rt�|�}|rtj|ddd�}|r=|r)t�|�}t�|�}t�|�}	t|�}|r=||��8}|||	|7}|S)am29    Generate Gaussian noise based on the input image.30 31    Args:32        img (torch.Tensor): Input image.33        grad_wrt: Gradient with respect to the input image.34        norm (bool, optional): Whether to normalize the generated noise. Defaults to True.35        absolute (bool, optional): Whether to take the absolute values of the gradients. Defaults to True.36        grayscale (bool, optional): Whether to convert the noise to grayscale. Defaults to True.37        keepmean (bool, optional): Whether to keep the mean of the noise. Defaults to False.38 39    Returns:40        torch.Tensor: Generated Gaussian noise.41    r42Tr)r�43randn_likerrrrrr)44rrr r!r"r#�gaussian_noiser&r'r(r)r)r*�get_gaussian3s4546474849r.�H�����z>cCs�t�|���rtj|dd�}t||�}t�||�t�|�}|rttt|��D]L}tj||dd�||dd�||dd�gdd�}	t�	||�}50|||	|51|	<t52|53dd�|durct�	|�}t54||d	d�t55||d56d�q'|S)�x57    Calculates the plausibility score based on the given inputs.58 59    Args:60        imgs (torch.Tensor): The input images.61        targets_out (torch.Tensor): The output targets.62        attr (torch.Tensor): The attribute tensor.63        debug (bool, optional): Whether to enable debug mode. Defaults to False.64 65    Returns:66        torch.Tensor: The plausibility score.67    ���nanNr68r��dim�figs/test_bbox��	save_path�figs/im0�	figs/attr)r�isnan�any�69nan_to_num�get_bbox_mapr�range�len�cat�70zeros_liker)�targets_out�attr�debug�corners�imgs�eps�71coords_map�plaus_score�i�
coords_map3ch�	test_bboxr)r)r*�get_plaus_scoreVs!72873rN)�bbox_iouc%74Cs|dd�df��}|dd�dd�f}t�tj|jd|jd|jd|jdg|jd�|jddf�}t|�|��}	|	}75|rM|dd�dd�f��}76tj|tjd�}|77dd�df|78dd�df}}
|79dd�df|80dd�df}}t	|81jd�D]}d	|||dd�|||
|�||||�f<q~t�82|���r�tj|d83d�}|r�t	t
|��D]C}tj||dd�||dd�||dd�gdd�}t�||�}|||||<t|d
d�t||dd�t||dd�q�||��}84|85dd�dfd}|86dd�dfd}tj|dd	d�}d}t	|87jd�D]�}|||dd�d||�d||�f}|||dd�d||�||d�f}|||dd�||d�d||�f}|||dd�||d�||d�f}t|d�\}}t|d�\}}t|d�\}}t|d�\} }!|||7}|||7}| ||7} |!||7}!t�t�||��d�|jdt�||��d�|jdt�|| ��d�|jdt�||!��d�|jdg�}"|du�r�|"}�qtj||"gdd�}�q|�dd�}t|j|ddd�}#|#��}$|$S)r0Nr�����devicer88��dtypeTr1r2r4r6r7r9r:r������F�CIoU)�x1y1x2y2�metric)�intr�tile�tensorrrT�corners_coords_batchrB�boolr?r;r<r=r@rArr�max_indices_2dr�	unsqueezer�viewrO�Tr)%rCrDrErFrGrH�target_inds�89xyxy_batch�90num_pixels�xyxy_corners�corI�x1�x2�y1�y2�icrKrLrM�attr_�corners_attr�attr0�attr1�attr2�attr3�x_0�y_0�x_1�y_1�x_2�y_2�x_3�y_3�max_corners�IoU_rJr)r)r*�get_attr_corners�shB""28	((((�91rcCsJt�|�t�|�}||jd}||jd}t�|�d�|�d�g�S)Nr92r)rr�argmaxrrArb)�x_inp�index�x�yr)r)r*ra�s939495rac	Cs�|jd}|d}tj|dtjd�}t|�D]`}||df|dk||df|dk@}||df|dk||df|dk@}|d||df||df||df|d||df||df||dfk}|||B|@A}|}q|S)Nrr96�.rrU�.r97)rrrBr`r?)	�poly�grid�98num_points�j�oddNodesrK�cond1�cond2�cond3r)r)r*�point_in_polygon�s99((Xr�c
Cs�|jd}t�|�}|d|}|�d��d��dd|jd|jd�}||df|dk||df|dk@}||df|dk||df|dk@}|d||df||df||df|d||df||df||dfk}tj|dtjd�}	||B|@}100g}t|101�dkr�t|102�ddkr�|�|103d�|104dd�}105t�	|106dt107t|108�d��|109t110t|111�d�d��}112t|113�dks�|D]}t�	|114|�}115q�|116}	|	S)Nrr117rWr�r�rUrP)rr�arangerb�expandrBr`r@�append�bitwise_xorr\)
r�r�r�rKr��
poly_expandedr�r�r�r��cond�c�c_r)r)r*�point_in_polygon_gpu
s(118119(((X4�r�c	Csbt�|��|j���}t�|��|j���}t�||�\}}tj||fdd�}t||�}|�d�S)NrWr4r)	rr��torT�float�meshgrid�stackr�rb)	r��h�wr�r��grid_y�grid_xr��bitmapr)r)r*�bitmap_for_polygon/s120121r�cCs>|\}}}}||d}||d}t�||||||g�S)NrP)rr^��center_xywh�center_x�center_yr�r�r�r�r)r)r*�corners_coords8sr�cCsz|dd�df|dd�df}}|dd�df|dd�df}}||d}||d}tj||||||gdd�S)Nrr122rPrRr4)rr�r�r)r)r*r_>s123""r_cCs�tj|jdgdt|jdd���R|jd�}tj|jdgdt|jdd���R|jd�}t|jd�D]}||��||<||��||<q7||||}|S)z�124    Normalize a batch of tensors along each channel.125    126    Args:127        x (torch.Tensor): Input tensor of shape (batch_size, channels, height, width).128        129    Returns:130        torch.Tensor: Normalized tensor of the same shape as the input.131    r)r132r133NrS)r�zerosrr@rTr?rr)r��mins�maxsrK�x_r)r)r*rEs01340rcCsF|��t���||�\}}Wd�n1swY~||fS)a135    Get detections from a model given an input image and targets.136 137    Args:138        model (nn.Module): The model to use for detection.139        img (torch.Tensor): The input image tensor.140 141    Returns:142        torch.Tensor: The detected bounding boxes.143    N)�evalr�no_grad)�model_cloner�det_out�outr)r)r*�get_detectionsXs144�r�cs�|j\}}}}|����dd�dd�ft�||||g��|j��dd�dd�f<|jr8�fdd�t|�D�ng}t|dd|dd�}	g}145t	|	�D]�\}}��dd�df|kd	d�f}
t146|
�}|��}tj|dd�d147fddd�}||}tt�
|dd�d148fdk��d	}|d|�}tj|jdd	f|jd
�|}tj||dd�gd�ffd	d�}t|dd�dd�f�|dd�dd�f<t�||g�gd�}|dd�dd�f|�|j�<|149�|�qIt�|150d��|j�}151|152S)NrPcs,g|]}��dd�df|kdd�f�qS)Nrr153r))�.0rK��targets_r)r*�154<listcomp>rs,zget_labels.<locals>.<listcomp>g����MbP?g333333�?T)�155conf_thres�	iou_thres�labels�multi_labelrr156rX)r5�157descendingg�������?rS)�rr158rPrRr4)r159rr160r)rrr�Tensorr�rT�save_hybridr?r�	enumerater@�argsortr\r�onesrArr^r�)r�rG�targets�opt�nb�_�height�width�lb�o�pred_labels�si�predr��nl�predn�sort_indices�sorted_pred�n_conf�new_col�preds�gnr)r�r*�161get_labelsms,>  ""($r�)�	make_gridcCs`||��}d}||k}t�|�\}}|������}|������}td|�d|�d��dS)Ngffffff�?z The central bright point is at (z, �))rr�wherer�r�item�print)rD�162img_tensor�	threshold�mask�y_coords�x_coords�163centroid_x�164centroid_yr)r)r*�get_center_coords�sr���?cCs�|jd|jd}}t�t�t�|�t�|����|j�\}}tj||fdd���}	|	�|jdddd�}165gg|jd}t	|jd�D]�}||dd�df|k}
t166|
�dkr�|
dd�dd�f}|dd�df||dd�df||dd�df<|dd�df<|�d��d�}tj|167|�t168|�ddd�|dd�}tj
|dd�d�d�}|jdd	kr�tj|||gdd�n|}nt�||�}|||<qEtt�|��|}t�|���r�tj|d169d�}|�rRt	t170|��D]f}|ddk�rQtj||dd�||dd�||dd�gdd�}t|d
d�|du�r t�|�}t||dd�t||||ddd�}t|dd�||||}t|dd�t||dd�q�|S)a�171    Compute the distance grids from each pixel to the target coordinates.172 173    Args:174        attr (torch.Tensor): Attribution maps.175        targets (torch.Tensor): Target coordinates.176        focus_coeff (float, optional): Focus coefficient, smaller means more focused. Defaults to 0.5.177        debug (bool, optional): Whether to visualize debug information. Defaults to False.178 179    Returns:180        torch.Tensor: Distance grids.181    rW�����r4rNrPrXr182rRr1r2�zfigs/dist_gridsr7r9g�?)�alphazfigs/dist_grid_overlayzfigs/weighted_attrr:)rrr�r�r�r�rTr�r�r?r@rbr rrArBrr;r<r=rr)rDr�rG�focus_coeffrEr�r��xx�yy�idx_grid�idx_batch_grid�dist_grids_r��rows�xy�	xy_center�dists�183dist_grid_�	dist_grid�184dist_gridsrK�	grid_show�img_overlay�
weighted_attrr)r)r*�get_distance_grids�sD*B&(1858186187�r�cCst�||�}|S�N)rr)r%�distance_map�	dist_attrr)r)r*�attr_reg�sr�c	CsB|dd�df��}|dd�dd�f}t�tj|jd|jd|jd|jdg|jd�|jddf�}t|�|��}|}|rM|dd�dd�f��}tj|tjd�}|dd�df|dd�df}	}188|dd�df|dd�df}}t	|jd�D]}
d|||
dd�|	|
|189|
�||
||
�f<q~|�190tj�}|S)	NrrPrQrRrSr191rUT)r\rr]r^rrTr_rBr`r?r��float32)rCrDrFrerfrgrhrirIrjrkrlrmrn�bbox_mapr)r)r*r>�sB""2r>cCsb|st||�����d�|d�}nt�d�}t||||j�}|jr.t	||��192tj�}d||<t|d|�}	t||�}193|	t�
|�|194t�
|�}|jdkrct	||�}t||�}t|d|�}
||
}nd}t	||�}t�||�t�|�}|js�d|d}||j||j||j}nd|d}d||j}|r�|S|s�||||ffS||||f|fS)NT)rCrDrGr1g�?g@r195)rNrr�requires_grad_rr^r�r��dist_x_bboxr>r�r`r�r�196bbox_coeffr�
dist_reg_only�	iou_coeff�197dist_coeff�	pgt_coeff)r�r%r�rGrE�	only_lossrJr�r��
dist_attr_pos�
dist_attr_neg�dist_reg�
attr_bbox_pos�
attr_bbox_neg�bbox_reg�
dist_reg_loss�	plaus_reg�198plaus_lossr)r)r*�get_plaus_losss@199200201202203204205��rr206rR�cpuc
)s�|j}
|
s	|��d|_|��|}||�}|r�gg}}tt|��D]H}t�|�|kr7�|d|��|<�|��dkr_�|dd�df}|�dd�|D��t�|�}|�|�q#|�gd��|�d�q#dd��D�}�fd	d�tt���D�}t�	|�}t�207|�}tt���D]2}t||�|kr�||gt�|t||��}t
�|�d|��d�||<q�||�d�||<q�tt|�dd208d209�D]}||��dkr�|�|�q�t
�|�}|r�t|�nd}|r�dn|}g}t|�D]�}|r�|r�|}|�r||nd}|�sdn|}g}t|�D]�}|du�r?||} |�r9||dd�dd�dd�dd�|||f} t
�| �}!nF|�rK|dd�|f}"n|}"z|||"||d�\}#}$Wn'|"�|�}"|�|�}|D]}%|%�|�}%�qitd�|||"||d�\}#}$Y|#} d}!|��t
jj| ||!dd
�}&|&d}'|210�r�t
j|'ddd�}'|	�r�t
�|'�}'|�r�t|'�}'|�|'��qt|�dk�r�|�t
�|��d��|��q�|�t
�|��|��q�|}(|
�s�|��|(S)a$211    Generate vanilla gradients for the given model and input tensor.212 213    Args:214        model (nn.Module): The model to generate gradients for.215        input_tensor (torch.Tensor): The input tensor for which gradients are computed.216        loss_func (callable, optional): The loss function to compute gradients with respect to. Defaults to None.217        targets_list (list, optional): The list of target tensors. Defaults to None.218        metric (callable, optional): The metric function to evaluate the loss. Defaults to None.219        out_num (int, optional): The index of the output tensor to compute gradients with respect to. Defaults to 1.220        n_max_labels (int, optional): The maximum number of labels to consider. Defaults to 3.221        norm (bool, optional): Whether to normalize the attribution map. Defaults to True.222        abs (bool, optional): Whether to take the absolute values of gradients. Defaults to True.223        grayscale (bool, optional): Whether to convert the attribution map to grayscale. Defaults to True.224        class_specific_attr (bool, optional): Whether to compute class-specific attribution maps. Defaults to True.225        device (str, optional): The device to use for computation. Defaults to 'cpu'.226    227    Returns:228        torch.Tensor: The generated vanilla gradients.229    TNrr230c	Ss g|]}dddddt|�g�qS�rr231rPrRrX)r\)r��ucr)r)r*r�~s z)generate_vanilla_grad.<locals>.<listcomp>rcSsg|]}|�����qSr))rr)r��targr)r)r*r���csg|]}t�|��qSr))r@)r��ih��targets_listr)r*r��rrW)r[z:Error in loss function, trying again with device specified)r�retain_graphr)�training�train�
requires_grad�	zero_gradr?r@�numelr��nprr��math�ceilrrArb�poprr�r�rrrrrrBr�r�))�model�input_tensor�	loss_funcrr�r[�out_num�n_max_labelsr rr"�class_specific_attrrT�232train_mode�inpt�	train_out�n_attr_list�
index_classesrK�
class_numbers�	num_attrs�targets_list_filled�233labels_len�234max_labels�	max_index�tlist�n_img_attrs�attrs_batch�i_batch�
n_label_attrs�	attrs_img�i_attrrr$�target_indiv�loss�235loss_items�tro�	gradientsr%�out_attrr)rr*�generate_vanilla_gradOs�236237238 239�240241.242243244�245r?cs(eZdZdZ�fdd�Zdd�Z�ZS)�RVNonLinearFunczf246    Custom Bayesian ReLU activation function for random variables.247 248    Attributes:249        None250    cstt|���||_dSr�)�superr@�__init__�func)�selfrC��	__class__r)r*rB�s251zRVNonLinearFunc.__init__c252Csr|�d�}|�|�}tjj||t�|�dd�d�|d�}|jdd�}|jdd�}t�||�}t�	||�}	||	fS)aN253        Forward pass of the Bayesian ReLU activation function.254 255        Args:256            mu_in (torch.Tensor): A tensor of shape (batch_size, input_size),257                representing the mean input to the ReLU activation function.258            Sigma_in (torch.Tensor): A tensor of shape (batch_size, input_size, input_size),259                representing the covariance input to the ReLU activation function.260 261        Returns:262            Tuple[torch.Tensor, torch.Tensor]: A tuple of two tensors,263                including the mean of the output and the covariance of the output.264        rTrrWrPr4r265)266�sizerCrrrrrcrb�bmm�mul)267rD�mu_in�Sigma_in�268batch_size�mu_out�gradi�grad1�grad2�
outer_product�	Sigma_outr)r)r*�forward�s269270&zRVNonLinearFunc.forward)�__name__�271__module__�__qualname__�__doc__rBrS�
__classcell__r)r)rEr*r@�sr@)FTFF)TTTF)FFNr/)Nr�F)F)NFF)NNNNr272rRTTTTr)*r�numpyr�plot_functsrrrr�time�matplotlib.path�path�mplPathr�
utils.generalrrr	r+r.rNrOrrar�r�r�r�r_rr�r��torchvision.utilsr�r�r�r�r>rr?�nn�Moduler@r)r)r)r*�<module>sH273274(275#A276Z"	277J278279G280�