Aluode/PerceptionLabPortable
0
1o
2 0�Yi� � @ s� d dl mZmZ ddlmZ erddlmZ ddlmZ ddl m3Z4m Z m Z m
Z
ddlmZ e � r5d d lZe
�e�ZG d5d � d e�Zd S ) � )�
TYPE_CHECKING�Optional� )� HfQuantizer� )�PreTrainedModel)�replace_with_spqr_linear)�is_accelerate_available�is_spqr_available�is_torch_available�logging)�QuantizationConfigMixinNc sz e Zd ZdZdZdef� fdd� Zdd� Zdd d �Z
dddde e6e fdd�Z ddd�Z
edd� �Zddd�Z� ZS ) �SpQRHfQuantizerzS7 Quantizer of the SpQR method. Enables the loading of prequantized models.8 T�quantization_configc s t � j|fi |�� || _d S �N)�super�__init__r )�selfr �kwargs�� __class__� ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\transformers/quantizers/quantizer_spqr.pyr ( s 9zSpQRHfQuantizer.__init__c O s2 t j�� s td��t� std��t� std��d S )Nz,GPU is required to run SpQR quantized model.zGUsing `spqr` quantization requires Accelerate: `pip install accelerate`zFUsing `spqr` quantization requires SpQR: `pip install spqr_quant[gpu]`)�torch�cudaZ is_available� RuntimeErrorr � ImportErrorr10 )r �argsr r r r �validate_environment, s 11�z$SpQRHfQuantizer.validate_environment�dtype� torch.dtype�returnc C s2 |d u rt j}t�d� |S |t jkrtd��|S )NzHAssuming SpQR inference on GPU and loading the model in `torch.float16`.z|You cannot use any type other than torch.float16 for SpQR. Please either leave it None or set it totorch.float16 explicitly.)r Zfloat16�logger�info�12ValueError)r r r r r � update_dtype6 s 1314��z SpQRHfQuantizer.update_dtypeN�modelr �keep_in_fp32_modulesc K s4 | � || jj|�| _t|| j| jd� | j|j_d S )N)r �modules_to_not_convert)Zget_modules_to_not_convertr r( r �config)r r& r' r r r r �$_process_model_before_weight_loadingA s 15��z4SpQRHfQuantizer._process_model_before_weight_loadingc K s |S r r )r r& r r r r �#_process_model_after_weight_loadingR � z3SpQRHfQuantizer._process_model_after_weight_loadingc C � dS )NFr )r r r r � is_trainableU s z SpQRHfQuantizer.is_trainablec C r- )NTr )r Zsafe_serializationr r r �is_serializableY r, z SpQRHfQuantizer.is_serializable)r r r! r r )r& r )�__name__�16__module__� __qualname__�__doc__Zrequires_calibrationr
r r r% r �list�strr* r+ �propertyr. r/ �
__classcell__r r r r r ! s 17 18��1920�2122r )�typingr r �baser Zmodeling_utilsr Z integrationsr �utilsr r23 r r Zutils.quantization_configr
r Z24get_loggerr0 r" r r r r r �<module> s
25