Aluode/PerceptionLabPortable
0
1o
2 .�YiC � @ sN d Z ddlZddlZddlZddlZddlmZmZmZm Z m3Z4 ddl m Z ddl
mZ ddlmZ ddlmZ dd lmZmZmZmZmZmZmZmZm Z dd5l m Z erZddl Z e � e!�Z"e d d d
�Z#G dd � d e�Z$de%e& de&fdd�Z'd dd�Z(ee$j)�e$_)e$j)j dur�e$j)j j*dddd�e$j)_ dZ+d de%e& dee, fd d �Z-dS ) z'Configuration base class and utilities.� N)�
TYPE_CHECKING�Any�Optional�TypeVar�Union)�version� )� __version__)�custom_object_save)�load_gguf_checkpoint) � CONFIG_NAME�PushToHubMixin� cached_file� copy_func� download_url�extract_commit_hash�
is_remote_url�is_torch_available�logging)�is_timm_config_dict� SpecificPretrainedConfigType�PretrainedConfig)�boundc 8 sr e Zd ZU dZdZeed< dZeed< i Ze ee6d f ed< dZ e ed< i Z
e eef ed< d Zee eef ed7< d Zee eeee f ed < d Zee eeee f ed < d Zee ed
< � fdd�Z� fdd�Zddddd d ddddd ddd d d d d d d d d d d d d d d�de de de de deeedf dee eee f d e d ed e d e d ee d!e d"e d#eee d$ee d%ee eef d&ee eef d'ee d(ee eef d)ee d*ee d+ee d,ee d-ee d.ee d/ee d0ee f6d1d2�Zd'efd3d4�Ze d5ee fd6d7��Z e j d8d7� �Z e d9d:� �Z e j d;e fd<d:��Z e d5e fd=d>��Z e d5efd?d@��Z!e!j d'efdAd@��Z!e dBdC� �Z"e"j d;eeee f fdDdC��Z"e dEdF� �Z#e#j dGdF� �Z#d�dHeee$j%f dIe fdJdK�Z&e'd�dLdM��Z(e) Nd�dOe8e* dPeee$j%f dQeeee$j%f dRe dSe dTeeee f dUed5e*fdVdW��Z+e)dPeee$j%f d5ee eef e eef f fdXdY��Z,e)dPeee$j%f d5ee eef e eef f fdZd[��Z-e)dOe9e* d\e eef d5e*fd]d^��Z.e)dOe10e* d_eee$j%f d5e*fd`da��Z/e)d_eee$j%f fdbdc��Z0ddde� Z1dfdg� Z2dhdi� Z3d5e eef fdjdk�Z4d5e eef fdldm�Z5d�dne d5efdodp�Z6d�dqeee$j%f dne fdrds�Z7d\e eef fdtdu�Z8dvefdwdx�Z9dye eef d5d fdzd{�Z:dye eef d5d fd|d}�Z;e)d�dd���Z<e'd5e eef fd�d���Z=d5e eef fd�d��Z>d�d�d�d��Z?e)d�d�� �Z@e)d�d�� �ZA� ZBS )�r a[! 11 Base class for all configuration classes. Handles a few parameters common to all models' configurations as well as12 methods for loading/downloading/saving configurations.13 14 <Tip>15 16 A configuration file can be loaded and saved to disk. Loading the configuration file and using this file to17 initialize a model does **not** load the model weights. It only affects the model's configuration.18 19 </Tip>20 21 Class attributes (overridden by derived classes):22 23 - **model_type** (`str`) -- An identifier for the model type, serialized into the JSON file, and used to recreate24 the correct object in [`~transformers.AutoConfig`].25 - **has_no_defaults_at_init** (`bool`) -- Whether the config class can be initialized without providing input arguments.26 Some configurations requires inputs to be defined at init and have no default values, usually these are composite configs,27 (but not necessarily) such as [`~transformers.EncoderDecoderConfig`] or [`~RagConfig`]. They have to be initialized from28 two or more configs of type [`~transformers.PretrainedConfig`].29 - **keys_to_ignore_at_inference** (`list[str]`) -- A list of keys to ignore by default when looking at dictionary30 outputs of the model during inference.31 - **attribute_map** (`dict[str, str]`) -- A dict that maps model specific attribute names to the standardized32 naming of attributes.33 - **base_model_tp_plan** (`dict[str, Any]`) -- A dict that maps sub-modules FQNs of a base model to a tensor34 parallel plan applied to the sub-module when `model.tensor_parallel` is called.35 - **base_model_pp_plan** (`dict[str, tuple[list[str]]]`) -- A dict that maps child-modules of a base model to a36 pipeline parallel plan that enables users to place the child-module on the appropriate device.37 38 Common attributes (present in all subclasses):39 40 - **vocab_size** (`int`) -- The number of tokens in the vocabulary, which is also the first dimension of the41 embeddings matrix (this attribute may be missing for models that don't have a text modality like ViT).42 - **hidden_size** (`int`) -- The hidden size of the model.43 - **num_attention_heads** (`int`) -- The number of attention heads used in the multi-head attention layers of the44 model.45 - **num_hidden_layers** (`int`) -- The number of blocks in the model.46 47 <Tip warning={true}>48 49 Setting parameters for sequence generation in the model config is deprecated. For backward compatibility, loading50 some of them will still be possible, but attempting to overwrite them will throw an exception -- you should set51 them in a [~transformers.GenerationConfig]. Check the documentation of [~transformers.GenerationConfig] for more52 information about the individual parameters.53 54 </Tip>55 56 Arg:57 name_or_path (`str`, *optional*, defaults to `""`):58 Store the string that was passed to [`PreTrainedModel.from_pretrained`] or59 [`TFPreTrainedModel.from_pretrained`] as `pretrained_model_name_or_path` if the configuration was created60 with such a method.61 output_hidden_states (`bool`, *optional*, defaults to `False`):62 Whether or not the model should return all hidden-states.63 output_attentions (`bool`, *optional*, defaults to `False`):64 Whether or not the model should returns all attentions.65 return_dict (`bool`, *optional*, defaults to `True`):66 Whether or not the model should return a [`~transformers.utils.ModelOutput`] instead of a plain tuple.67 is_encoder_decoder (`bool`, *optional*, defaults to `False`):68 Whether the model is used as an encoder/decoder or not.69 is_decoder (`bool`, *optional*, defaults to `False`):70 Whether to only use the decoder in an encoder-decoder architecture, otherwise it has no effect on71 decoder-only or encoder-only architectures.72 cross_attention_hidden_size (`bool`, *optional*):73 The hidden size of the cross-attention layer in case the model is used as a decoder in an encoder-decoder74 setting and the cross-attention hidden dimension differs from `self.config.hidden_size`.75 add_cross_attention (`bool`, *optional*, defaults to `False`):76 Whether cross-attention layers should be added to the model. Note, this option is only relevant for models77 that can be used as decoder models within the [`EncoderDecoderModel`] class, which consists of all models78 in `AUTO_MODELS_FOR_CAUSAL_LM`.79 tie_encoder_decoder (`bool`, *optional*, defaults to `False`):80 Whether all encoder weights should be tied to their equivalent decoder weights. This requires the encoder81 and decoder model to have the exact same parameter names.82 prune_heads (`dict[int, list[int]]`, *optional*, defaults to `{}`):83 Pruned heads of the model. The keys are the selected layer indices and the associated values, the list of84 heads to prune in said layer.85 86 For instance `{1: [0, 2], 2: [2, 3]}` will prune heads 0 and 2 on layer 1 and heads 2 and 3 on layer 2.87 chunk_size_feed_forward (`int`, *optional*, defaults to `0`):88 The chunk size of all feed forward layers in the residual attention blocks. A chunk size of `0` means that89 the feed forward layer is not chunked. A chunk size of n means that the feed forward layer processes `n` <90 sequence_length embeddings at a time. For more information on feed forward chunking, see [How does Feed91 Forward Chunking work?](../glossary.html#feed-forward-chunking).92 93 > Parameters for fine-tuning tasks94 95 architectures (`list[str]`, *optional*):96 Model architectures that can be used with the model pretrained weights.97 finetuning_task (`str`, *optional*):98 Name of the task used to fine-tune the model. This can be used when converting from an original (TensorFlow99 or PyTorch) checkpoint.100 id2label (`dict[int, str]`, *optional*):101 A map from index (for instance prediction index, or target index) to label.102 label2id (`dict[str, int]`, *optional*):103 A map from label to index for the model.104 num_labels (`int`, *optional*):105 Number of labels to use in the last layer added to the model, typically for a classification task.106 task_specific_params (`dict[str, Any]`, *optional*):107 Additional keyword arguments to store for the current task.108 problem_type (`str`, *optional*):109 Problem type for `XxxForSequenceClassification` models. Can be one of `"regression"`,110 `"single_label_classification"` or `"multi_label_classification"`.111 112 > Parameters linked to the tokenizer113 114 tokenizer_class (`str`, *optional*):115 The name of the associated tokenizer class to use (if none is set, will use the tokenizer associated to the116 model by default).117 prefix (`str`, *optional*):118 A specific prompt that should be added at the beginning of each text before calling the model.119 bos_token_id (`int`, *optional*):120 The id of the _beginning-of-stream_ token.121 pad_token_id (`int`, *optional*):122 The id of the _padding_ token.123 eos_token_id (`int`, *optional*):124 The id of the _end-of-stream_ token.125 decoder_start_token_id (`int`, *optional*):126 If an encoder-decoder model starts decoding with a different token than _bos_, the id of that token.127 sep_token_id (`int`, *optional*):128 The id of the _separation_ token.129 130 > PyTorch specific parameters131 132 torchscript (`bool`, *optional*, defaults to `False`):133 Whether or not the model should be used with Torchscript.134 tie_word_embeddings (`bool`, *optional*, defaults to `True`):135 Whether the model's input and output word embeddings should be tied. Note that this is only relevant if the136 model has a output word embedding layer.137 dtype (`str`, *optional*):138 The `dtype` of the weights. This attribute can be used to initialize the model to a non-default `dtype`139 (which is normally `float32`) and thus allow for optimal storage allocation. For example, if the saved140 model is `float16`, ideally we want to load it back using the minimal amount of memory needed to load141 `float16` weights.142 � �143model_type�base_config_key� sub_configsF�has_no_defaults_at_init�
attribute_mapN�base_model_tp_plan�base_model_pp_plan�base_model_ep_plan� _auto_classc s2 |t � �d�v rt � �d�| }t � �||� d S �Nr )�super�__getattribute__� __setattr__)�self�key�value�� __class__� ��E:\DocsHouse\542 percep lab latest\PerceptionLab\PerceptionLab_Portable\python_embed\Lib\site-packages\transformers/configuration_utils.pyr& � s z PretrainedConfig.__setattr__c s4 |dkr|t � �d�v rt � �d�| }t � �|�S r# )r$ r% )r' r( r* r, r- r% � s z!PretrainedConfig.__getattribute__Tr )�output_hidden_states�output_attentions� return_dict� torchscript�dtype� pruned_heads�tie_word_embeddings�chunk_size_feed_forward�is_encoder_decoder�144is_decoder�cross_attention_hidden_size�add_cross_attention�tie_encoder_decoder�
architectures�finetuning_task�id2label�label2id�145num_labels�task_specific_params� problem_type�tokenizer_class�prefix� bos_token_id� pad_token_id� eos_token_id� sep_token_id�decoder_start_token_idr. r/ r0 r1 r2 z torch.dtyper3 r4 r5 r6 r7 r8 r9 r: r; r< r= r> r? r@ rA rB rC rD rE rF rG rH c $ K s� |d ur
t |t�s
td��|d urt |t�std��|d ur6|d ur6t|�|kr6t�d|� dt|�� d�� |d urF|dvrFtd|� d���| �d d � } d urX|d urV|n| }|d urmt |t�rmt� rmd146d l } t147| |�}|| _ || _ || _
|| _|| _|d ur�|ni | _|| _|| _| | _|148| _| | _| | _|
| _|| _|| _|| _|| _|| _ || _ | jd u r�| � |d ur�|nd � n d d
� | j� � D �| _|| _ || _!|| _"|| _#|| _$|| _%|| _&| �'� � � D ]\} } t(| | | �| | �� q�t| �dd��| _)| �dd �| _*| �dd �| _+| �dd �| _,| �-dd��r t.�/d� | � � D ]*\}!}"z t(| |!|"� W �q t0�yJ }# zt�1d|!� d|"� d| � �� |#�d }#~#ww | �dd�| _2| �dd�| _3d S )Nz)Argument label2id should be a dictionary.z)Argument id2label should be a dictionary.zYou passed `num_labels=z9` which is incompatible to the `id2label` map of length `z`.)Z149regressionZsingle_label_classificationZmulti_label_classificationzAThe config parameter `problem_type` was not understood: received za but only 'regression', 'single_label_classification' and 'multi_label_classification' are valid.� torch_dtyper � c S � i | ] \}}t |�|�qS r, ��int��.0r( r) r, r, r- �150<dictcomp>3 � z-PretrainedConfig.__init__.<locals>.<dictcomp>� name_or_pathr � _commit_hash�attn_implementation�transformers_versionZgradient_checkpointingFa Passing `gradient_checkpointing` to a config initialization is deprecated and will be removed in v5 Transformers. Using `model.gradient_checkpointing_enable()` instead, or if you are using the `Trainer` API, pass `gradient_checkpointing=True` in your `TrainingArguments`.z151Can't set z with value z for �tf_legacy_loss� use_bfloat16)4�152isinstance�dict�153ValueError�len�logger�warning�pop�strr �torch�getattrr0 r. r1 r2 �_output_attentionsr3 r4 r5 r6 r7 r8 r9 r: r; r< r= r> r@ rA �_create_id_label_maps�itemsrB rC rD rE rF rG rH � _get_global_generation_defaults�setattr�
_name_or_pathrS �_attn_implementationrU �get�warnings�warn�AttributeError�errorrV rW )$r' r. r/ r0 r1 r2 r3 r4 r5 r6 r7 r8 r9 r: r; r< r= r> r? r@ rA rB rC rD rE rF rG rH �kwargsrI r` �parameter_name�
default_valuer( r) �errr, r, r- �__init__� s� % ��154�155156� ��zPretrainedConfig.__init__c C s4 dd� t |�D �| _tt| j�� | j�� ��| _d S )Nc S s i | ]}|d |� ��qS )ZLABEL_r, )rO �ir, r, r- rP b s z:PretrainedConfig._create_id_label_maps.<locals>.<dictcomp>)�ranger= rY �zip�values�keysr> �r' r? r, r, r- rc a s z&PretrainedConfig._create_id_label_maps�returnc C s t | dd �S )Nrg )ra �r' r, r, r- rR e s z PretrainedConfig.name_or_pathc C s t |�| _d S �N)r_ rg �r' r) r, r, r- rR i s c C � | j S )zQ157 `bool`: Whether or not the model should returns all attentions.158 )rb rz r, r, r- r/ m s z"PretrainedConfig.output_attentionsr) c C s> |r159| j d u r160d| _ |r| j dkrtd| j � d���|| _d S )N�eagerz_The `output_attentions` attribute is not supported when using the `attn_implementation` set to z#. Please set it to 'eager' instead.)rh rZ rb r| r, r, r- r/ t s ��161c C s | j o| j S )zY162 `bool`: Whether or not return [`~utils.ModelOutput`] instead of tuples.163 )r0 r1 rz r, r, r- �use_return_dict� s z PretrainedConfig.use_return_dictc C s164 t | j�S )zH165 `int`: The number of labels for classification models.166 )r[ r= rz r, r, r- r? � s 167zPretrainedConfig.num_labelsc C s& | j d u s168| j|kr| �|� d S d S r{ )r= r? rc rx r, r, r- r? � s �c C r} r{ )� _attn_implementation_internalrz r, r, r- rh � s z%PretrainedConfig._attn_implementationc C s| t | dd�}t|t�s
|n|�d|�}|| _| jD ]"}t | |d�}|dur;t |dd�}t|t�s2|n|�||�}||_qdS )z0We set it recursively on the sub-configs as wellrh Nr )ra rX rY ri r� r rh )r' r) Z current_attnrT Z
subconfig_keyZ subconfigZcurrent_subconfig_attnZsub_implementationr, r, r- rh � s 169 ���c C s t �d� | jS �N�1`torch_dtype` is deprecated! Use `dtype` instead!�r\ � warning_oncer2 rz r, r, r- rI � s 170z PretrainedConfig.torch_dtypec C s t �d� || _d S r� r� r| r, r, r- rI � s 171172�save_directory� push_to_hubc K s | � |� tj�|�rtd|� d���| �� }t|�dkr(t�dt |�� �t173� tj |dd� |rR|� dd�}|� d |�
tjj�d174 �}| j|fi |��}| �|�}d | v r[t| d � | jdurgt| || d � tj�|t�}| j|dd
� t�d|� �� |r�| j|||||�d�d� dS dS )aS 175 Save a configuration object to the directory `save_directory`, so that it can be re-loaded using the176 [`~PretrainedConfig.from_pretrained`] class method.177 178 Args:179 save_directory (`str` or `os.PathLike`):180 Directory where the configuration JSON file will be saved (will be created if it does not exist).181 push_to_hub (`bool`, *optional*, defaults to `False`):182 Whether or not to push your model to the Hugging Face model hub after saving it. You can specify the183 repository you want to push to with `repo_id` (will default to the name of `save_directory` in your184 namespace).185 kwargs (`dict[str, Any]`, *optional*):186 Additional key word arguments passed along to the [`~utils.PushToHubMixin.push_to_hub`] method.187 zProvided path (z#) should be a directory, not a filer a� Some non-default generation parameters are set in the model config. These should go into either a) `model.generation_config` (as opposed to `model.config`); OR b) a GenerationConfig file (https://huggingface.co/docs/transformers/generation_strategies#save-a-custom-decoding-strategy-with-your-model).This warning will become an exception in the future.188Non-default generation parameters: T)�exist_ok�commit_messageN�repo_id�����Ztransformers_weights)�config��use_diffzConfiguration saved in �token)r� r� )�_set_token_in_kwargs�os�path�isfile�AssertionError�&_get_non_default_generation_parametersr[ rj rk r_ � UserWarning�makedirsr^ �split�sepZ _create_repoZ_get_files_timestamps�delattrr" r189 �joinr � to_json_filer\ �infoZ_upload_modified_filesri ) r' r� r� rn �!non_default_generation_parametersr� r� Zfiles_timestampsZoutput_config_filer, r, r- �save_pretrained� sB 190 �� 191192193194��z PretrainedConfig.save_pretrainedc C s` |du r195| � dd�}| � dd�}|dur$t�dt� |dur"td��|}|dur.|| d< dS dS )z�Temporary method to deal with `token` and `use_auth_token`.196 197 This method is to avoid apply the same changes in all model config classes that overwrite `from_pretrained`.198 199 Need to clean up `use_auth_token` in a follow PR.200 Nr� �use_auth_tokenzrThe `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.zV`token` and `use_auth_token` are both specified. Please set only the argument `token`.)r^ rj rk �
FutureWarningrZ )rn r� r� r, r, r- r� � s �� �z%PretrainedConfig._set_token_in_kwargs�main�cls� pretrained_model_name_or_path� cache_dir�force_download�local_files_onlyr� �revisionc 201 K s� ||d< ||d< ||d< ||d< | � ||� | j|fi |��\}}| jr.| j|v r.|| j }d|v rjt| d�rj|d | jkrj|�� D ]} t| t�rS| �d�| jkrS| }qB|d | jkrjt �202d|d � d| j� d�� | j |fi |��S ) a� 203 Instantiate a [`PretrainedConfig`] (or a derived class) from a pretrained model configuration.204 205 Args:206 pretrained_model_name_or_path (`str` or `os.PathLike`):207 This can be either:208 209 - a string, the *model id* of a pretrained model configuration hosted inside a model repo on210 huggingface.co.211 - a path to a *directory* containing a configuration file saved using the212 [`~PretrainedConfig.save_pretrained`] method, e.g., `./my_model_directory/`.213 - a path or url to a saved configuration JSON *file*, e.g., `./my_model_directory/configuration.json`.214 cache_dir (`str` or `os.PathLike`, *optional*):215 Path to a directory in which a downloaded pretrained model configuration should be cached if the216 standard cache should not be used.217 force_download (`bool`, *optional*, defaults to `False`):218 Whether or not to force to (re-)download the configuration files and override the cached versions if219 they exist.220 resume_download:221 Deprecated and ignored. All downloads are now resumed by default when possible.222 Will be removed in v5 of Transformers.223 proxies (`dict[str, str]`, *optional*):224 A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128',225 'http://hostname': 'foo.bar:4012'}.` The proxies are used on each request.226 token (`str` or `bool`, *optional*):227 The token to use as HTTP bearer authorization for remote files. If `True`, or not specified, will use228 the token generated when running `hf auth login` (stored in `~/.huggingface`).229 revision (`str`, *optional*, defaults to `"main"`):230 The specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a231 git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any232 identifier allowed by git.233 234 <Tip>235 236 To test a pull request you made on the Hub, you can pass `revision="refs/pr/<pr_number>"`.237 238 </Tip>239 240 return_unused_kwargs (`bool`, *optional*, defaults to `False`):241 If `False`, then this function returns just the final configuration object.242 243 If `True`, then this functions returns a `Tuple(config, unused_kwargs)` where *unused_kwargs* is a244 dictionary consisting of the key/value pairs whose keys are not configuration attributes: i.e., the245 part of `kwargs` which has not been used to update `config` and is otherwise ignored.246 subfolder (`str`, *optional*, defaults to `""`):247 In case the relevant files are located inside a subfolder of the model repo on huggingface.co, you can248 specify the folder name here.249 kwargs (`dict[str, Any]`, *optional*):250 The values in kwargs of any keys which are configuration attributes will be used to override the loaded251 values. Behavior concerning key/value pairs whose keys are *not* configuration attributes is controlled252 by the `return_unused_kwargs` keyword parameter.253 254 Returns:255 [`PretrainedConfig`]: The configuration object instantiated from this pretrained model.256 257 Examples:258 259 ```python260 # We can't instantiate directly the base class *PretrainedConfig* so let's show the examples on a261 # derived class: BertConfig262 config = BertConfig.from_pretrained(263 "google-bert/bert-base-uncased"264 ) # Download configuration from huggingface.co and cache.265 config = BertConfig.from_pretrained(266 "./test/saved_model/"267 ) # E.g. config (or model) was saved using *save_pretrained('./test/saved_model/')*268 config = BertConfig.from_pretrained("./test/saved_model/my_configuration.json")269 config = BertConfig.from_pretrained("google-bert/bert-base-uncased", output_attentions=True, foo=False)270 assert config.output_attentions == True271 config, unused_kwargs = BertConfig.from_pretrained(272 "google-bert/bert-base-uncased", output_attentions=True, foo=False, return_unused_kwargs=True273 )274 assert config.output_attentions == True275 assert unused_kwargs == {"foo": False}276 ```r� r� r� r� r z You are using a model of type z to instantiate a model of type zN. This is not supported for all configurations of models and can yield errors.) r� �get_config_dictr �hasattrr rv rX rY ri r\ r] � from_dict)277r� r� r� r� r� r� r� rn � config_dict�vr, r, r- �from_pretrained s( V 278 � ��z PretrainedConfig.from_pretrainedc K s� | � |� t�|�}| j|fi |��\}}|du r i |fS d|v r'|d |d< d|v r>t|d �}| j|fd|i|��\}}||fS )a� 279 From a `pretrained_model_name_or_path`, resolve to a dictionary of parameters, to be used for instantiating a280 [`PretrainedConfig`] using `from_dict`.281 282 Parameters:283 pretrained_model_name_or_path (`str` or `os.PathLike`):284 The identifier of the pre-trained checkpoint from which we want the dictionary of parameters.285 286 Returns:287 `tuple[Dict, Dict]`: The dictionary(ies) that will be used to instantiate the configuration object.288 289 NrS �configuration_files�_configuration_file)r� �copy�deepcopy�_get_config_dict�get_configuration_file)r� r� rn Zoriginal_kwargsr� �configuration_filer, r, r- r� � s" 290291 ��292�z PretrainedConfig.get_config_dictc K sR |� dd �}|� dd�}|� dd �}|� dd �}|� dd �}|� dd�}|� dd �} |� d d �}293|� d294d �} |� d d �} |� d
d�}
|� dd �}|�d�}|295du rVt�d� d|
d�}| d urc| |d< t|�}tj�|�}tj�tj� | |��r}|}d}nZt296|�r�|d u r�|n|}t |�}nI|d u r�|� dt �n|}z t
|||||||||| | |d� }|d u r�d |fW S t||�}W n ty� � ty� td|� d|� d|� d���w z|r�t|dd�d }n| �|�}||d< W n tjtfy� td |� d ���w |�r t�d |� �� n t�d |� d |� �� d |v�r%t|��r%d!|d < ||fS )"Nr� r� F�resume_download�proxiesr� r� r� �trust_remote_code� subfolderr �_from_pipeline�297_from_autorS � gguf_fileTzgThe argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.r� )� file_type�from_auto_classZusing_pipeliner� )298r� r� r� r� r� r� �299user_agentr� r� rS z!Can't load the configuration of 'z�'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure 'z2' is the correct path to a directory containing a z file)Zreturn_tensorsz"It looks like the config file at 'z' is not a valid JSON file.zloading configuration file z from cache at r Z timm_wrapper)r^ ri r\ r] r_ r� r� �isdirr� r� r r r r r �OSError� Exceptionr �_dict_from_json_file�json�JSONDecodeError�UnicodeDecodeErrorr� r )r� r� rn r� r� r� r� r� r� r� r� r� Z
from_pipeliner� Z commit_hashr� r� �is_localZresolved_config_filer� r� r, r, r- r� � s� 300�301 302�303 ���� 304 �z!PretrainedConfig._get_config_dictr� c
K s� |� dd�}|� dd� |� dd� d|v r d|v r |d |d< |� dd� }dur7t�d� |�d |�|d < |� d305d�|d306< | di |��}t|d �rVd d
� |j�� D �|_d|v r�d|v r�|d }|d durl|d ng }t|�|kr�td|� d|d � d���g }|�� D ]8\} }307t|| �r�t || �} t308| t �r�t309|310t �r�| �
� } | �|311� | jdi | ��}312t|| |313� | d kr�|�| � q�|D ]} |� | d� q�t�d|� �� |r�||fS |S )ao 314 Instantiates a [`PretrainedConfig`] from a Python dictionary of parameters.315 316 Args:317 config_dict (`dict[str, Any]`):318 Dictionary that will be used to instantiate the configuration object. Such a dictionary can be319 retrieved from a pretrained checkpoint by leveraging the [`~PretrainedConfig.get_config_dict`] method.320 kwargs (`dict[str, Any]`):321 Additional parameters from which to initialize the configuration object.322 323 Returns:324 [`PretrainedConfig`]: The configuration object instantiated from those parameters.325 �return_unused_kwargsFr� Nr� rS rI r� r2 rT r3 c S rK r, rL rN r, r, r- rP + rQ z.PretrainedConfig.from_dict.<locals>.<dictcomp>r? r= z You passed along `num_labels=z(` with an incompatible id to label map: zX. Since those arguments are inconsistent with each other, you should remove one of them.z
Model config r, )r^ r\ r� ri r� r3 rd r[ rZ ra rX r rY �to_dict�updater+ rf �appendr� )
r� r� rn r� rI r� r? r= � to_remover( r) Z current_attrZcurrent_attr_updatedr, r, r- r� sN 326327 ��328329330 331�zPretrainedConfig.from_dict� json_filec C s | � |�}| di |��S )aQ 332 Instantiates a [`PretrainedConfig`] from the path to a JSON file of parameters.333 334 Args:335 json_file (`str` or `os.PathLike`):336 Path to the JSON file containing the parameters.337 338 Returns:339 [`PretrainedConfig`]: The configuration object instantiated from that JSON file.340 341 Nr, )r� )r� r� r� r, r, r- �from_json_fileM s 342z PretrainedConfig.from_json_filec C s> t |dd�� }|�� }W d � n1 sw Y t�|�S )N�utf-8��encoding)�open�readr� �loads)r� r� �reader�textr, r, r- r� _ s 343 �344z%PretrainedConfig._dict_from_json_filec C s t |t�o345| j|jkS r{ )rX r �__dict__)r' �otherr, r, r- �__eq__e � zPretrainedConfig.__eq__c C s | j j� d| �� � �S )N� )r+ �__name__�to_json_stringrz r, r, r- �__repr__h r� zPretrainedConfig.__repr__c c s � | j E d H d S r{ )r� rz r, r, r- �__iter__k s �zPretrainedConfig.__iter__c C s8 | � � }t� � � }| js| �� � � ni }i }|�� D ]Y\}}tt| |d�t�r2||v r2t|| t�s7|| jv rQt ||t| |d�d�}d|v rL|d |d< |||< q||vso|dkso|dkso||| kso||v rs||�346||�krs|||< q| � |� d|v r�|d= t | d�r�t| j
t�s�| j
� � n| j
|d< | �|� |S )ar 347 Removes all attributes from the configuration that correspond to the default config attributes for348 better readability, while always retaining the `config` attribute from the class. Serializes to a349 Python dictionary.350 351 Returns:352 dict[str, Any]: Dictionary of all the attributes that make up this configuration instance.353 N��354config_objr rU Z355vocab_filerg �quantization_config)r� r r r+ rd rX ra rY r �recursive_diff_dictri �_remove_keys_not_serializedr� r� �dict_dtype_to_str)r' r� Zdefault_config_dictZclass_config_dictZserializable_config_dictr( r) �diffr, r, r- � to_diff_dictn s@ 356� �357 358 �359360361 ��362z PretrainedConfig.to_diff_dictc C s� t �| j�}t| jd�r| jj|d< t|d< |�� D ]\}}t|t �r*|�363� }|d= |||< q| � |� t| d�rIt| j t
�sD| j �364� n| j |d< | �|� |S )z�365 Serializes this instance to a Python dictionary.366 367 Returns:368 `dict[str, Any]`: Dictionary of all the attributes that make up this configuration instance.369 r rU r� )r� r� r� r� r+ r r rd rX r r� r� r� rY r� )r' �outputr( r) r, r, r- r� � s" 370371372373374 ��375zPretrainedConfig.to_dictr� c C s. |du r | � � }n| �� }tj|ddd�d S )a� 376 Serializes this instance to a JSON string.377 378 Args:379 use_diff (`bool`, *optional*, defaults to `True`):380 If set to `True`, only the difference between the config instance and the default `PretrainedConfig()`381 is serialized to JSON string.382 383 Returns:384 `str`: String containing all the attributes that make up this configuration instance in JSON format.385 TrJ )�indent� sort_keys�386)r� r� r� �dumps)r' r� r� r, r, r- r� � s 387z PretrainedConfig.to_json_string�json_file_pathc C sF t |ddd��}|�| j|d�� W d � dS 1 s w Y dS )a� 388 Save this instance to a JSON file.389 390 Args:391 json_file_path (`str` or `os.PathLike`):392 Path to the JSON file in which this configuration instance's parameters will be saved.393 use_diff (`bool`, *optional*, defaults to `True`):394 If set to `True`, only the difference between the config instance and the default `PretrainedConfig()`395 is serialized to JSON file.396 �wr� r� r� N)r� �writer� )r' r� r� �writerr, r, r- r� � s "�z PretrainedConfig.to_json_filec C s" |� � D ]397\}}t| ||� qdS )z�398 Updates attributes of this class with attributes from `config_dict`.399 400 Args:401 config_dict (`dict[str, Any]`): Dictionary of attributes that should be updated for this class.402 N)rd rf )r' r� r( r) r, r, r- r� � s �zPretrainedConfig.update�403update_strc C s� t dd� |�d�D ��}|�� D ]a\}}t| |�s!td|� d���t| |�}t|t�rH|�� dv r4d}n7|�� dv r=d }n.td404|� d |� d ���t|t �rRt |�}nt|t405�r\t406|�}nt|t �skt d
|� d|� ���t
| ||� qdS )a� 407 Updates attributes of this class with attributes from `update_str`.408 409 The expected format is ints, floats and strings as is, and for booleans use `true` or `false`. For example:410 "n_embd=10,resid_pdrop=0.2,scale_attn_weights=false,summary_type=cls_index"411 412 The keys to change have to already exist in the config object.413 414 Args:415 update_str (`str`): String with attributes that should be updated for this class.416 417 c s s � | ]}|� d �V qdS )�=N)r� )rO �xr, r, r- � <genexpr> s � z6PretrainedConfig.update_from_string.<locals>.<genexpr>�,zkey z" isn't in the original config dict)�true�1�y�yesT)�false�0�n�noFz can't derive true or false from z (key �)zIYou can only update int, float, bool or string values in the config, got z for key N)rY r� rd r� rZ ra rX �bool�lowerrM �floatr_ � TypeErrorrf )r' r� �d�kr� �old_vr, r, r- �update_from_string� s* 418419420 421422423424425��z#PretrainedConfig.update_from_stringr c C s� |� d�dur2t|d t�r dd� |d �� D �|d< nt|d ttf�s2t|d ��d�d |d< |�� D ] }t|t�rB| �|� q6dS )a( 426 Checks whether the passed dictionary and its nested dicts have a *dtype* key and if it's not None,427 converts torch.dtype to a string of just the type. For example, `torch.float32` get converted into *"float32"*428 string, which can then be stored in the json format.429 r2 Nc S s$ i | ]\}}|t |��d �d �qS )�.r� )r_ r� )rO r r� r, r, r- rP $ s $ z6PretrainedConfig.dict_dtype_to_str.<locals>.<dictcomp>r r ) ri rX rY rd r_ rM r� rv r� )r' r r) r, r, r- r� s 430431��z"PretrainedConfig.dict_dtype_to_strc C s� t | d�r |�dd�}d|v r|d= d|v r |�d�|d< d|v r$|d= d|v r+|d= d |v r2|d = d432|v r9|d433= |�� D ] }t|t�rI| �|� q=dS ) z�434 Checks and removes if there are any keys in the dict that should not be serialized when saving the config.435 Runs recursive check on the dict, to remove from all sub configs.436 r� Z_pre_quantization_dtypeNr" rb r/ rS r� r r )r� r^ rv rX rY r� )r' r �_r) r, r, r- r� - s& 437 438439��z,PretrainedConfig._remove_keys_not_serialized�440AutoConfigc C sD t |t�s|j}ddlm m} t||�s t|� d���|| _dS )ae 441 Register this class with a given auto class. This should only be used for custom configurations as the ones in442 the library are already mapped with `AutoConfig`.443 444 445 446 Args:447 auto_class (`str` or `type`, *optional*, defaults to `"AutoConfig"`):448 The auto class to register this new configuration with.449 r Nz is not a valid auto class.) rX r_ r� Ztransformers.models.auto�models�autor� rZ r" )r� Z450auto_classZ auto_moduler, r, r- �register_for_auto_classH s 451 452453z(PretrainedConfig.register_for_auto_classc 454 C s� i dd�dd�dd�dd�dd �d455d �d d
�dd �dd �dd �dd �dd�dd�dd �dd �dd�dd�d d dd d d d dd��S )N�456max_length� Z457min_lengthr Z do_sampleFZearly_stoppingZ num_beamsr Z temperatureg �?Ztop_k�2 Ztop_pZ typical_pZrepetition_penaltyZlength_penaltyZno_repeat_ngram_sizeZ encoder_no_repeat_ngram_sizeZ
bad_words_idsZnum_return_sequencesZ
output_scoresZreturn_dict_in_generateg )Zforced_bos_token_idZforced_eos_token_idZremove_invalid_valuesZ exponential_decay_length_penaltyZsuppress_tokensZbegin_suppress_tokensZnum_beam_groupsZdiversity_penaltyr, r, r, r, r- re ^ sX �������� �458� � �
������z0PretrainedConfig._get_global_generation_defaultsc C s� i }d}z| � � }W n ty$ | jdd�}|| ur |� � }nd}Y nw |du r+| nt| |�}| �� �� D ]<\}}t||�rrd }} t||�}459|460du rMq6|durY|461t||�k}n|462|k} |du ph|du oh| du } | rrt||�||< q6|S )z]463 Gets the non-default generation parameters on the PretrainedConfig instance464 NT)�decoderF)r+ rZ �get_text_configra re rd r� ) r' r� Zdecoder_attribute_nameZdefault_configZdecoder_configZself_decoder_configro Zdefault_global_valueZis_default_in_configZis_default_generation_valueZparameter_valueZis_non_defaultr, r, r- r� } s8 465��466467��z7PretrainedConfig._get_non_default_generation_parametersc C s� ||k}d}d}|r|| }n|r|}n|}g }|D ]}t | |�r0t| |d�} | dur0||g7 }qt|�dkr?td|� d���t|�dkrMt| |d �}468n| }469|s�t|�dkr�|470jr�t�|471�}472|rcdnd } |rid nd} |473�� D ]N}
|
�| �r�|
|474j �475� vr�t |476|
� |
�| �r�|
| d477 kr�d }n|
| d kr�d
}n478|
t| �d d� }||479j v r�|480j | }t|481|
�}t |482|
� t |483||� qo|484S )a� 485 Returns the text config related to the text input (encoder) or text output (decoder) of the model. The486 `decoder` and `encoder` input arguments can be used to specify which end of the model we are interested in,487 which is useful on models that have both text input and output modalities.488 489 There are three possible outcomes of using this method:490 1. On most models, it returns the original config instance itself.491 2. On newer (2024+) composite models, it returns the text section of the config, which is nested under a set492 of valid names.493 3. On older (2023-) composite models, it discards decoder-only parameters when `encoder=True` and vice-versa.494 495 Args:496 decoder (`Optional[bool]`, *optional*):497 If set to `True`, then only search for decoder config names.498 encoder (`Optional[bool]`, *optional*):499 If set to `True`, then only search for encoder config names.500 )r � generator� text_config)Z text_encoderNr z<Multiple valid text configs were found in the model config: z�. In this case, using `get_text_config()` would be ambiguous. Please specify the desired text config directly, e.g. `text_config = config.sub_config_name`r �encoderr Z_layers�num_hidden_layersZ_attention_headsZnum_attention_heads)
r� ra r[ rZ r6 r� r� r� �501startswithr rv r� rf )r' r r Z return_bothZ"decoder_possible_text_config_namesZ"encoder_possible_text_config_namesZpossible_text_config_namesZvalid_text_config_namesZtext_config_namer Zconfig_to_returnZprefix_to_discardZprefix_to_keepr( Znew_keyr) r, r, r- r � sT 502503 504� 505� 506 507508 509510511512 �z PretrainedConfig.get_text_configc K �( t �dt� | d|�� |�� d�|��S )z�513 Instantiate a model config (or a derived class) from text model configuration and vision model514 configuration.515 516 Returns:517 [`PreTrainedConfig`]: An instance of a configuration object518 z�The `from_text_vision_configs` method is deprecated and will be removed in v4.60 of Transformers. Please instantiate the config class directly with `MyConfig(text_config=text_config, vision_config=vision_config, **kwargs)` instead.)r �
vision_configNr, �rj rk r� r� )r� r r rn r, r, r- �from_text_vision_configs� �519 520� z)PretrainedConfig.from_text_vision_configsc K r )z�521 Instantiate a model config (or a derived class) from text model configuration and audio model522 configuration.523 524 Returns:525 [`PreTrainedConfig`]: An instance of a configuration object526 z�The `from_text_audio_configs` method is deprecated and will be removed in v4.60 of Transformers. Please instantiate the config class directly with `MyConfig(text_config=text_config, audio_config=audio_config, **kwargs)` instead.)r � audio_configNr, r )r� r r rn r, r, r- �from_text_audio_configs r z(PretrainedConfig.from_text_audio_configs)Fr{ )NFFNr� )T)r )NN)ry r )Cr� �527__module__� __qualname__�__doc__r r_ �__annotations__r r rY �typer r� r r r r r �tuple�listr! r" r& r% r rM rr rc �propertyrR �setterr/ r r? rh rI r� �PathLiker� � staticmethodr� � classmethodr r� r� r� r� r� r� r� r� r� r� r� r� r� r� r r� r� r re r� r r r �
__classcell__r, r, r* r- r 5 s� 528 ������529� � �
�����530�������� � � � � �!531� 532533 534 535536 @ �� ������ �p � �" � �_�537� �G� � �<" 538& ,N539r� ry c C s� i }| D ] }|� d�r |�d�r |dkr |�d��d�}|||< qt|�� �}t}t�t �}|D ]}t�|�|kr>|| }q0 |S |S )z�540 Get the configuration file to use for this version of transformers.541 542 Args:543 configuration_files (`list[str]`): The list of available configuration files.544 545 Returns:546 `str`: The configuration file to use.547 zconfig.z.jsonz config.json)548r �endswith� removeprefix� removesuffix�sortedrw r r �parser )r� Zconfiguration_files_map� file_namer� Zavailable_versionsr� rU r, r, r- r� s 549 � 550551r� c C s� i }|dur |� � �� ni }| �� D ]8\}}t|t|�d�}t|t�r<||v r<t|| t�r<t||| |d�}|||< q||vsF||| krJ|||< q|S )a 552 Helper function to recursively take the diff between two nested dictionaries. The resulting diff only contains the553 values from `dict_a` that are different from values in `dict_b`.554 555 dict_b : the default config dictionary. We want to remove values that are in this one556 Nr� ) r+ r� rd ra r_ rX r rY r� ) Zdict_aZdict_br� r� �defaultr( r) Z obj_valueZ557diff_valuer, r, r- r� : s 558�r� r� r zconfiguration file)�objectZ object_classZ object_files)Zfull_attentionZsliding_attentionZchunked_attentionZlinear_attention� layer_typesr c C sV t dd� | D ��stdt� ���|dur'|t| �kr)td|� dt| �� d���dS dS )z.Check that `layer_types` is correctly defined.c s s � | ]}|t v V qd S r{ )�ALLOWED_LAYER_TYPES)rO Z559layer_typer, r, r- r� ^ s � z(layer_type_validation.<locals>.<genexpr>z%The `layer_types` entries must be in Nz`num_hidden_layers` (z.) must be equal to the number of layer types (r� )�allrZ r3 r[ )r2 r r, r, r- �layer_type_validation\ s ���r5 r{ ).r r� r� r� rj �typingr r r r r � packagingr r r Zdynamic_module_utilsr560 Zmodeling_gguf_pytorch_utilsr �utilsr r
r r r r r r r Z
utils.genericr r` Z561get_loggerr� r\ r r r# r_ r� r� r� �formatr3 rM r5 r, r, r, r- �<module> sF , 562 p563 564�