Sorobot/Document-modify-Toolkit
0
1U
2 h�Hf� � @ sh d dl Z d dlZd dlZd dlm mZ d dlm Z ddd�Z3dd� Z dd � Z dd
d�Z
ddd�ZdS )� N)�special� c K sr t | d�r:| jdk r:tt|�f |�| j� |dk r:|| j_t | d�rn| jdk rnt�|�| j� |dk rn|| j_dS )a� 4 In-place params init function.5 Usage:6 .. code-block:: python7 8 import paddle9 import numpy as np10 11 data = np.ones([3, 4], dtype='float32')12 linear = paddle.nn.Linear(4, 4)13 input = paddle.to_tensor(data)14 print(linear.weight)15 linear(input)16 17 weight_init_(linear, 'Normal', 'fc_w0', 'fc_b0', std=0.01, mean=0.1)18 print(linear.weight)19 �weightN�bias)�hasattrr �getattr�init�namer �Constant)�layer�func� weight_name� bias_name�20bias_value�kwargs� r �,D:\CodeProjects\as1\Transform\weight_init.py� weight_init_ s r c 21 C s� dd� }||d| k s(||d| kr0t d� t�� �� ||| | �}||| | �}tjjd| d d| d t| j�d��tj �}t22� |�}||t �
d� 9 }||7 }t�|||�}| �t�|�� | W 5 Q R � S Q R X d S )Nc S s dt �| t �d� � d S )N� �?� @)�math�erf�sqrt)�xr r r �norm_cdf, s z(_no_grad_trunc_normal_.<locals>.norm_cdf� zjmean is more than 2 std from [a, b] in nn.init.trunc_normal_. The distribution of values may be incorrect.� )�sizer )�print�paddle�no_grad�np�random�uniform�list�shape�astype�float32r �erfinvr r �clip� set_value� to_tensor) �tensor�mean�std�a�br �lower�upper�tmpr r r �_no_grad_trunc_normal_+ s* �2324 25 ��26r4 c C sd | � � }|dk rtd��| jd }| jd }d}| � � dkrL| d d �� }|| }|| }||fS )Nr zNFan in and fan out can not be computed for tensor with fewer than 2 dimensionsr r )�dim�27ValueErrorr% �numel)r, �28dimensionsZnum_input_fmapsZnum_output_fmaps�receptive_field_size�fan_in�fan_outr r r � _calculate_fan_in_and_fan_outQ s �2930 r< r � �r c C s t | ||||�S )N)r4 )r, r- r. r/ r0 r r r �
trunc_normal_d s r> r: �31leaky_reluc 32 C sl dd� }ddd�}|| |�}|||�}|t �|� }t�� �& tjj�d|�| � | W 5 Q R � S Q R X d S )Nc S sD |� � }ddg}||kr(td�||���t| �\}}|dkr@|S |S )Nr: r; z+Mode {} not supported, please use one of {})r1 r6 �formatr< )r, �modeZ valid_modesr: r; r r r �_calculate_correct_fani s 33� z/kaiming_normal_.<locals>._calculate_correct_fanc S s� dddddddg}| |ks"| dkr&d S | d34kr2d S | d krDt �d
�S | dkr�|d krZd}n2t|t�snt|t�sxt|t�r~|}ntd�|���t �d
d |d �S td�| ���d S )N�linear�conv1d�conv2d�conv3dZconv_transpose1dZconv_transpose2dZconv_transpose3d�sigmoidr �tanhg�������?�relur r? g{�G�z�?z$negative_slope {} not a valid numberr zUnsupported nonlinearity {})r r �35isinstance�bool�int�floatr6 r@ )� nonlinearity�paramZ36linear_fns�negative_sloper r r �calculate_gaint s6 � 37���z'kaiming_normal_.<locals>.calculate_gainr )N)r r r r �nn� initializer�Normal) r, r/ rA rN rB rQ �fan�gainr. r r r �kaiming_normal_h s 38 394041rW )NNr )r r r= r )r r: r? )r �numpyr! r �paddle.nn.initializerrR rS r Zscipyr r r4 r< r> rW r r r r �<module> s �42"&43