Aluode/PerceptionLabPortable
0
1o
2 0�Yi= � @ sp d dl mZmZmZ ddlmZ erddlmZ ddlm Z m3Z4m Z e5� r)d dl Z e �
e�ZG dd � d e�ZdS )6� )�
TYPE_CHECKING�Optional�Union� )� HfQuantizer� )�PreTrainedModel)�is_accelerate_available�is_torch_available�loggingNc s� e Zd ZdZdZdZdgZ� fdd�Zdd� Zd d d �Z
d d d7de8e e fdd�Z
dee eee f f dee eee f f fdd�Zd!dd�Zd dd�Zedefdd ��Zedefd d ��Z� ZS )"�BitNetHfQuantizerz�9 1.58-bit quantization from BitNet quantization method:10 Before loading: it converts the linear layers into BitLinear layers during loading.11 12 Check out the paper introducing this method: https://huggingface.co/papers/2402.1776413 FTZ14acceleratec s t � j|fi |�� || _d S �N)�super�__init__�quantization_config)�selfr �kwargs�� __class__� ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\transformers/quantizers/quantizer_bitnet.pyr - s 15zBitNetHfQuantizer.__init__c O s� t � std��|�dd�s|�dd�rtd��tj�� s#t�d� d S |�d�}|d u r3t�d� d S |d urLt |t16�rNd |� � v sHd17|� � v rPtd ��d S d S d S ) NzOLoading a BitNet quantized model requires accelerate (`pip install accelerate`)Zfrom_tfFZ from_flaxztLoading ternary weights from tf/flax is currently not supported, please make sure the weights are in PyTorch format.zhYou don't have a GPU available to load the model, the inference will be slow because of weight unpacking�18device_mapz�You have loaded a BitNet model on CPU and have a CUDA device available, make sure to set your model on a GPU device in order to run your model.�cpuZdiskz�You are attempting to load a BitNet model with a device_map that contains a CPU or disk device.This is not supported. Please remove the CPU or disk device from the device_map.) r � ImportError�get�19ValueError�torch�cudaZ is_available�loggerZ warning_once�20isinstance�dict�values)r �argsr r r r r �validate_environment1 s. �21�22�"��z&BitNetHfQuantizer.validate_environment�modelr c K s |S r
r )r r$ r r r r �#_process_model_after_weight_loadingN � z5BitNetHfQuantizer._process_model_after_weight_loadingN�keep_in_fp32_modulesc K s: ddl m} | �|| jj|�| _||| j| j| jd�}d S )Nr )�replace_with_bitnet_linear)�modules_to_not_convertr �
pre_quantized)Z integrationsr( Zget_modules_to_not_convertr r) r* )r r$ r' r r( r r r �$_process_model_before_weight_loadingQ s 23�24�z6BitNetHfQuantizer._process_model_before_weight_loading�25max_memory�returnc C s dd� |� � D �}|S )Nc S s i | ] \}}||d �qS )g�������?r )�.0�key�valr r r �26<dictcomp>e s z7BitNetHfQuantizer.adjust_max_memory.<locals>.<dictcomp>)�items)r r, r r r �adjust_max_memoryd s z#BitNetHfQuantizer.adjust_max_memory� target_dtype� torch.dtypec C s27 t j}|S r
)r Zint8)r r4 r r r �adjust_target_dtypeh s z%BitNetHfQuantizer.adjust_target_dtypec C s dS )NTr )r Zsafe_serializationr r r �is_serializablel r& z!BitNetHfQuantizer.is_serializablec C � | j jdko | j jdkS )N�
autobitlinear�online�r Z linear_classZquantization_mode�r r r r � is_trainableo s 28�z BitNetHfQuantizer.is_trainablec C r8 )zUFlag indicating whether the quantized model can carry out quantization aware trainingr9 r: r; r< r r r �is_qat_trainablev s 29�z"BitNetHfQuantizer.is_qat_trainable)r$ r r
)r4 r5 r- r5 )�__name__�30__module__� __qualname__�__doc__Z requires_parameters_quantizationZrequires_calibrationZrequired_packagesr r# r% r �list�strr+ r r �intr3 r6 r7 �property�boolr= r> �
__classcell__r r r r r s* 31 ��3233�23435r )�typingr r r �baser Zmodeling_utilsr �utilsr r36 r r Z37get_loggerr? r r r r r r �<module> s
38