Aluode/PerceptionLabPortable
0
1o
2 0�Yi � @ s� d dl Z d dlmZ d dlmZ ddlmZ er ddlmZ ddl m3Z4 dd l m Z m
Z
mZmZ dd5lmZ e� r=d dlZe�e�ZG d d � d e�ZdS )
� N)�
TYPE_CHECKING)�version� )� HfQuantizer� )�PreTrainedModel)�replace_with_aqlm_linear)�is_accelerate_available�is_aqlm_available�is_torch_available�logging)�QuantizationConfigMixinc sz e Zd ZdZdZdgZdZdef� fdd� Zdd � Z dd
d�Z6 ddd�Z ddd�Z e
d efdd��Zddd�Z� ZS ) �AqlmHfQuantizerzS7 Quantizer of the AQLM method. Enables the loading of prequantized models.8 T�aqlmN�quantization_configc s t � j|fi |�� || _d S �N)�super�__init__r )�selfr �kwargs�� __class__� ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\transformers/quantizers/quantizer_aqlm.pyr - s 9zAqlmHfQuantizer.__init__c O s t � std��t� std��d S )NzGUsing `aqlm` quantization requires Accelerate: `pip install accelerate`zDUsing `aqlm` quantization requires AQLM: `pip install aqlm[gpu,cpu]`)r � ImportErrorr10 )r �argsr r r r �validate_environment1 s11 �z$AqlmHfQuantizer.validate_environment�dtype� torch.dtype�returnc C s: |d u rt j�� rt j}t�d� |S t j}t�d� |S )NzCUDA available. Assuming AQLM inference on GPU and loading the model in `torch.float16`. To overwrite it, set `dtype` manually.z�CUDA is unavailable. Assuming AQLM inference on CPU and loading the model in `torch.float32`. To overwrite it, set `dtype` manually.)�torch�cudaZ is_availableZfloat16�logger�infoZfloat32)r r r r r � update_dtype8 s 12���z AqlmHfQuantizer.update_dtype�modelr c K s" t || j| jjd� | j|j_d S )N)r � linear_weights_not_to_quantize)r r r& �config�r r% r r r r �$_process_model_before_weight_loadingF s �z4AqlmHfQuantizer._process_model_before_weight_loadingc K s |S r r r( r r r �#_process_model_after_weight_loadingR � z3AqlmHfQuantizer._process_model_after_weight_loadingc C sB t �tj� d��t �d�k}|rdS t�dtj� d�� d�� dS )Nr z1.0.2Tz$Currently installed `aqlm` version (zw) doesn't support training. If you wish to train a quantized model, please update `aqlm` with `pip install aqlm>=1.0.2`F)r �parse� importlib�metadatar" �warning)r Zaqlm_supports_trainingr r r � is_trainableU s �z AqlmHfQuantizer.is_trainablec C s dS )NTr )r Zsafe_serializationr r r �is_serializable` r+ z AqlmHfQuantizer.is_serializable)r r r r )r% r r )�__name__�13__module__� __qualname__�__doc__Zrequires_calibrationZrequired_packagesZoptimum_quantizerr
r r r$ r) r* �property�boolr0 r1 �
__classcell__r r r r r $ s 1415�16 17r )r- �typingr � packagingr �baser Zmodeling_utilsr Z integrationsr �utilsr r18 r r Zutils.quantization_configr
r Z19get_loggerr2 r" r r r r r �<module> s
20