Aluode/PerceptionLabPortable
0
1o
2 0�Yi� � @ s� d dl Z d dlmZmZmZ d dlmZ ddlmZ ddl m3Z4 er(ddl m Z dd l
mZmZmZmZ dd5lmZ e� rAd dlZe�e�ZG d d � d e�ZdS )
� N)�
TYPE_CHECKING�Optional�Union)�version� )� HfQuantizer)�get_module_from_name� )�PreTrainedModel)�is_accelerate_available�is_optimum_quanto_available�is_torch_available�logging)� QuantoConfigc s$ e Zd ZdZddgZdZdZdef� fdd� Zd d6� Z d d � Z7d
d� Z d3dd�Z de
e dede
e fdd�Zdddedefdd �Zd eeeeef f deeeeef f fd d �Zddd d!ded"d#fd$d%�Zd4d'd(�Z )d5ddd*ee
e fd+d,�Zd-d.� Zedefd/d0��Z d5d1d2�Z � Z S )6�QuantoHfQuantizerz*8 Quantizer for the quanto library9 Zquanto�10accelerateTF�quantization_configc s t � j|fi |�� | �� d S �N)�super�__init__� post_init)�selfr �kwargs�� __class__� ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\transformers/quantizers/quantizer_quanto.pyr 2 s zQuantoHfQuantizer.__init__c C s" | j jdur
| jstd��dS dS )z 11 Safety checker12 Nz�We don't support quantizing the activations with transformers library.Use quanto library for more complex use cases such as activations quantization, calibration and quantization aware training.)r Z activationsZ
pre_quantized�13ValueError�r r r r r 6 s14 ��zQuantoHfQuantizer.post_initc O s t � std��t� std��d S )NzhLoading an optimum-quanto quantized model requires optimum-quanto library (`pip install optimum-quanto`)z`Loading an optimum-quanto quantized model requires accelerate library (`pip install accelerate`))r � ImportErrorr )r �argsr r r r �validate_environment@ s ���z&QuantoHfQuantizer.validate_environmentc C s |d u r
ddi}t �d� |S )N� �cpuz�The device_map was not initialized. Setting device_map to {'':'cpu'}. If you want to use the model for inference, please set device_map ='auto')�logger�info)r Z15device_mapr r r �update_device_mapJ s �z#QuantoHfQuantizer.update_device_map�dtype� torch.dtype�returnc C s |d u r t �d� tj}|S )NzPYou did not specify `dtype` in `from_pretrained`. Setting it to `torch.float32`.)r$ r% �torchZfloat32)r r' r r r � update_dtypeT s 16z QuantoHfQuantizer.update_dtype� missing_keys�prefixc s� t � r ddlm} g � |�� D ]*\}}t||�r9|D ] }||v s)||� d|� �v r8|�d�s8|�d�s8� �|� qq� fdd�|D �S )Nr �� QModuleMixin�.z.weightz.biasc s g | ]}|� vr|�qS r r )�.0�k�Znot_missing_keysr r �17<listcomp>h s z9QuantoHfQuantizer.update_missing_keys.<locals>.<listcomp>)r �optimum.quantor/ Z
named_modules�18isinstance�endswith�append)r �modelr, r- r/ �name�module�missingr r3 r �update_missing_keysZ s 19��20�z%QuantoHfQuantizer.update_missing_keysr9 r21 �22param_namec K s> t � r ddlm} t||�\}}t||�r d|v r |j S dS )Nr r. �weightF)r r5 r/ r r6 �frozen)r r9 r> r r/ r; Z tensor_namer r r �param_needs_quantizationj s z*QuantoHfQuantizer.param_needs_quantization�23max_memoryc C s dd� |� � D �}|S )Nc S s i | ] \}}||d �qS )g�������?r )r1 �key�valr r r �24<dictcomp>w s z7QuantoHfQuantizer.adjust_max_memory.<locals>.<dictcomp>)�items)r rB r r r �adjust_max_memoryv s z#QuantoHfQuantizer.adjust_max_memory� param_valuez torch.Tensor�
target_devicez torch.devicec K s@ ddl m} ||||�|�� t||�\}}|�� d|j_d S )Nr )�_load_parameter_into_modelF)�modeling_utilsrJ �tor �freezer? Z
requires_grad) r r9 rH r> rI r rJ r; �_r r r �create_quantized_paramz s25 z(QuantoHfQuantizer.create_quantized_param� target_dtypec C sV t �tj� d��t �d�kr'ddlm} tj|j|j |j26d�}|| j j }|S t
d��)Nr z0.27.0r )� CustomDtype)�int8Zfloat8Zint4Zint2z�You are using `device_map='auto'` on an optimum-quanto quantized model. To automatically compute the appropriate device map, you should upgrade your `accelerate` library,`pip install --upgrade accelerate` or install it from source.)r �parse� importlib�metadataZaccelerate.utilsrQ r* rR ZFP8ZINT4ZINT2r �weightsr )r rP rQ �mappingr r r �adjust_target_dtype� s � �z%QuantoHfQuantizer.adjust_target_dtypeN�keep_in_fp32_modulesc K sD ddl m} | �|| jj|�| _||| j| jd�\}}| j|j_d S )Nr )�replace_with_quanto_layers)�modules_to_not_convertr )Z integrationsrZ Zget_modules_to_not_convertr r[ �config)r r9 rY r rZ rN r r r �$_process_model_before_weight_loading� s 27�2829�z6QuantoHfQuantizer._process_model_before_weight_loadingc K s |S r r )r r9 r r r r �#_process_model_after_weight_loading� � z5QuantoHfQuantizer._process_model_after_weight_loadingc C � dS )NTr r r r r � is_trainable� s z QuantoHfQuantizer.is_trainablec C r` )NFr )r Zsafe_serializationr r r �is_serializable� r_ z!QuantoHfQuantizer.is_serializable)r' r( r) r( )rP r( r) r( r ) �__name__�30__module__� __qualname__�__doc__Zrequired_packagesZ requires_parameters_quantizationZrequires_calibrationr r r r! r& r+ �list�strr= �boolrA �dictr �intrG rO rX r r] r^ �propertyra rb �
__classcell__r r r r r ) sB 3132 33 34 2 ���35�36��3738�r )rT �typingr r r � packagingr �baser Zquantizers_utilsr rK r39 �utilsr r r
r Zutils.quantization_configr r* Z40get_loggerrc r$ r r r r r �<module> s
41