Aluode/PerceptionLabPortable
0
1o
2 0�Yi�5 � @ s� d dl Z d dlmZmZ ddlmZ ddlmZ ddlm Z m3Z4m Z m Z m
Z
mZmZmZmZmZmZmZmZmZmZmZmZmZmZm Z m Z ddl m Z dd l m!Z! dd5l"m#Z# dd l$m%Z% dd l&m'Z' dd
l(m)Z) ddl*m+Z+ ddl,m-Z- ddl.m/Z/ ddl0m1Z1 ddl2m3Z3 ddl4m5Z5 ddl6m7Z7 ddl8m9Z9 ddl:m;Z; ddl<m=Z= ddl>m?Z? ddl@mAZA ddlBmCZC ddlDmEZE dd lFmGZG i d e%�d e)�d e+�d e7�d!e!�d"e?�d#eA�d$e5�d%e/�d&e9�d'e;�d(e-�d)e1�d*eE�d+e'�d,eG�d-eC�e3e#e=d.��ZHi d e �d e
�d e
�d%e�d e�d!e �d"e�d#e�d$e�d'e�d(e�d)e�d&e�d*e �d+e �d,e �d-e�ee6ed.��ZIe�JeK�ZLG d/d0� d0�ZMG d1d2� d2�ZNd3eOfd4d5�ZPd6eOfd7d8�ZQd9d:� ZRdS );� N)�Optional�Union� )�7AutoConfig)�logging)�8AqlmConfig�AutoRoundConfig� AwqConfig�BitNetQuantConfig�BitsAndBytesConfig�CompressedTensorsConfig�9EetqConfig�FbgemmFp8Config�FineGrainedFP8Config�
FPQuantConfig�10GPTQConfig� HiggsConfig� HqqConfig� Mxfp4Config�QuantizationConfigMixin�QuantizationMethod� QuantoConfig� QuarkConfig�11SpQRConfig�
TorchAoConfig�12VptqConfig� )� HfQuantizer)�AqlmHfQuantizer)�AutoRoundQuantizer)� AwqQuantizer)�BitNetHfQuantizer)�Bnb4BitHfQuantizer)�Bnb8BitHfQuantizer)� CompressedTensorsHfQuantizer)�EetqHfQuantizer)�FbgemmFp8HfQuantizer)�FineGrainedFP8HfQuantizer)�FPQuantHfQuantizer)�GptqHfQuantizer)�HiggsHfQuantizer)�HqqHfQuantizer)�Mxfp4HfQuantizer)�QuantoHfQuantizer)�QuarkHfQuantizer)�SpQRHfQuantizer)�TorchAoHfQuantizer)�VptqHfQuantizerZawqZbitsandbytes_4bitZbitsandbytes_8bitZgptqZaqlmZquantoZquarkZfp_quantZeetqZhiggsZhqqzcompressed-tensorsZ13fbgemm_fp8ZtorchaoZbitnetZvptqZspqr)Zfp8z14auto-roundZmxfp4c @ s. e Zd ZdZedefdd��Zedd� �ZdS )�AutoQuantizationConfigz�15 The Auto-HF quantization config class that takes care of automatically dispatching to the correct16 quantization config given a quantization config stored in a dictionary.17 �quantization_config_dictc C s� |� d�}|� dd�s|� dd�r!|� dd�rdnd}tj| }n|d u r)td��|tvr;td|� d tt�� �� ���t| }|�|�S )18N� quant_method� load_in_8bitF� load_in_4bit�_4bit�_8bit��The model's quantization config from the arguments has no `quant_method` attribute. Make sure that the model has been correctly quantized� Unknown quantization type, got � - supported types are: ) �getr �BITS_AND_BYTES�19ValueError� AUTO_QUANTIZATION_CONFIG_MAPPING�list�AUTO_QUANTIZER_MAPPING�keys� from_dict)�clsr3 r4 �suffix�20target_cls� rG ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\transformers/quantizers/auto.pyrC y s 21 �22��23z AutoQuantizationConfig.from_dictc K sV t j|fi |��}t|dd �d u rtd|� d���|j}| �|�}|jdi |�� |S )N�quantization_configz)Did not found a `quantization_config` in z2. Make sure that the model is correctly quantized.rG )r �from_pretrained�getattrr> rI rC �update)rD � pretrained_model_name_or_path�kwargsZ model_configr3 rI rG rG rH rJ � s 24�25z&AutoQuantizationConfig.from_pretrainedN)�__name__�26__module__� __qualname__�__doc__� classmethod�dictrC rJ rG rG rG rH r2 s s r2 c @ sd e Zd ZdZedeeef fdd��Zedd� �Z edeeef de27e fdd ��Z e d28d � �Z
d S )
�AutoHfQuantizerz�29 The Auto-HF quantizer class that takes care of automatically instantiating to the correct30 `HfQuantizer` given the `QuantizationConfig`.31 rI c K sx t |t�r32t�|�}|j}|tjkr |jr|d7 }n|d7 }|tvr0t d|� dt33t� � �� ���t| }||fi |��S )Nr8 r7 r: r; ) �34isinstancerT r2 rC r4 r r= r5 rA r> r@ rB )rD rI rN r4 rF rG rG rH � from_config� s 3536373839��zAutoHfQuantizer.from_configc K s t j|fi |��}| �|�S )N)r2 rJ rW )rD rM rN rI rG rG rH rJ � s 40z AutoHfQuantizer.from_pretrained� quantization_config_from_argsc C s� |durd}nd}t |t�r t |t�rt�|�}nt�|�}|dur9|jj|jjkr9td|jj� d|jj� d���t |tt tt41t t f�rg|durg|�
� }|�� D ]42\}}t|||� qP|dt|�� �� d�7 }|dkrwt |t �swt�|� |S t�|� |S ) z�43 handles situations where both quantization_config from args and quantization_config from model config are present.44 Nz�You passed `quantization_config` or equivalent parameters to `from_pretrained` but the model you're loading already has a `quantization_config` attribute. The `quantization_config` from the model will be used.� z The model is quantized with z but you are passing a z| config. Please make sure to pass the same quantization config class to `from_pretrained` with different loading attributes.z"However, loading attributes (e.g. z]) will be overwritten with the one you passed to `from_pretrained`. The rest will be ignored.)rV rT r rC r2 � __class__rO r> r r r r r Zget_loading_attributes�items�setattrr@ rB �warnings�warn�logger�info)rD rI rX Z warning_msgZloading_attr_dict�attr�valrG rG rH �merge_quantization_configs� s8 �4546 47���4849�z*AutoHfQuantizer.merge_quantization_configsc C s� | � dd �}| � dd�s| � dd�r"| � dd�rdnd}tj| }n|d u r*td��|tvr@t�d|� d tt� � �� d50�� dS d S ) Nr4 r5 Fr6 r7 r8 r9 r: r; z~. Hence, we will skip the quantization. To remove the warning, you can delete the quantization_config attribute in config.jsonT)51r< r r= r> r? r_ �warningr@ rA rB )r3 r4 rE rG rG rH �supports_quant_method� s �52��z%AutoHfQuantizer.supports_quant_methodN)rO rP rQ rR rS r r rT rW rJ r rc � staticmethodre rG rG rG rH rU � s 5354� �4rU �methodc � � fdd�}|S )z-Register a custom quantization configuration.c s6 � t v r td� � d���t| t�std��| t � < | S )NzConfig '�' already registeredz*Config must extend QuantizationConfigMixin)r? r> �55issubclassr � TypeError�rD �rg rG rH �register_config_fn
� 56z8register_quantization_config.<locals>.register_config_fnrG )rg rn rG rm rH � register_quantization_config57 � 58rp �namec rh )z Register a custom quantizer.c s6 � t v r td� � d���t| t�std��| t � < | S )Nz Quantizer 'ri z!Quantizer must extend HfQuantizer)rA r> rj r rl �rr rG rH �register_quantizer_fn ro z1register_quantizer.<locals>.register_quantizer_fnrG )rr rt rG rs rH �register_quantizer rq ru c C s� t | d�}|rt�| j�sd}|s|d ur,|r t�| j|�| _n|| _tj| j|d�} nd } | d urc| j|||||d� | �|�}| �|�}| � | �} | �59| �} t | jdd�sc| jj }60t |61d|62�|d< | | ||fS )NrI F)�
pre_quantized)�dtype�from_tf� from_flax�63device_map� weights_onlyZ64dequantize�valueZquant)
�hasattrrU re rI rc rW Zvalidate_environmentZ update_dtypeZupdate_device_mapZupdate_tp_planZupdate_ep_planrK r4 ) �configrI rw rx ry rz r{ �65user_agentrv Z hf_quantizerr4 rG rG rH �get_hf_quantizer* s< 66 ���67686970 r� )Sr] �typingr r Z models.auto.configuration_autor �utilsr Zutils.quantization_configr r r r71 r r r
r r r r r r r r r r r r r r �baser Zquantizer_aqlmr Zquantizer_auto_roundr Z
quantizer_awqr Zquantizer_bitnetr! Zquantizer_bnb_4bitr" Zquantizer_bnb_8bitr# Z quantizer_compressed_tensorsr$ Zquantizer_eetqr% Zquantizer_fbgemm_fp8r& Zquantizer_finegrained_fp8r' Zquantizer_fp_quantr( Zquantizer_gptqr) Zquantizer_higgsr* Z
quantizer_hqqr+ Zquantizer_mxfp4r, Zquantizer_quantor- Zquantizer_quarkr. Zquantizer_spqrr/ Zquantizer_torchaor0 Zquantizer_vptqr1 rA r? Z72get_loggerrO r_ r2 rU �strrp ru r� rG rG rG rH �<module> s� \ �������� �73� � �
�������������� �74� � �
������75)n