Apppublic
XaviXva/Video-LLaVA

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face
0likes
rootllavamodelmultimodal_encoderlanguagebinddepth__pycache__configuration_depth.cpython-310.pyc
configuration_depth.cpython-310.pyc206 linesDownload Raw Back to __pycache__
1o

2b e�N�@srddlZddlZddlmZddlmZddlmZe�e	�Z3Gdd�de�ZGdd�de�ZGd	d4�d5e�Z
dS)�N)�Union)�PretrainedConfig)�loggingcsdeZdZdZdZ										6			
			d�fdd�	Zedeee	j7fddfdd��Z�ZS)�CLIPTextConfiga8    This is the configuration class to store the configuration of a [`CLIPTextModel`]. It is used to instantiate a CLIP9    text encoder according to the specified arguments, defining the model architecture. Instantiating a configuration10    with the defaults will yield a similar configuration to that of the text encoder of the CLIP11    [openai/clip-vit-base-patch32](https://huggingface.co/openai/clip-vit-base-patch32) architecture.12 13    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the14    documentation from [`PretrainedConfig`] for more information.15 16    Args:17        vocab_size (`int`, *optional*, defaults to 49408):18            Vocabulary size of the CLIP text model. Defines the number of different tokens that can be represented by19            the `inputs_ids` passed when calling [`CLIPModel`].20        hidden_size (`int`, *optional*, defaults to 512):21            Dimensionality of the encoder layers and the pooler layer.22        intermediate_size (`int`, *optional*, defaults to 2048):23            Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder.24        num_hidden_layers (`int`, *optional*, defaults to 12):25            Number of hidden layers in the Transformer encoder.26        num_attention_heads (`int`, *optional*, defaults to 8):27            Number of attention heads for each attention layer in the Transformer encoder.28        max_position_embeddings (`int`, *optional*, defaults to 77):29            The maximum sequence length that this model might ever be used with. Typically set this to something large30            just in case (e.g., 512 or 1024 or 2048).31        hidden_act (`str` or `function`, *optional*, defaults to `"quick_gelu"`):32            The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`,33            `"relu"`, `"selu"` and `"gelu_new"` `"quick_gelu"` are supported.34        layer_norm_eps (`float`, *optional*, defaults to 1e-5):35            The epsilon used by the layer normalization layers.36        attention_dropout (`float`, *optional*, defaults to 0.0):37            The dropout ratio for the attention probabilities.38        initializer_range (`float`, *optional*, defaults to 0.02):39            The standard deviation of the truncated_normal_initializer for initializing all weight matrices.40        initializer_factor (`float`, *optional*, defaults to 1):41            A factor for initializing all weight matrices (should be kept to 1, used internally for initialization42            testing).43 44    Example:45 46    ```python47    >>> from transformers import CLIPTextConfig, CLIPTextModel48 49    >>> # Initializing a CLIPTextConfig with openai/clip-vit-base-patch32 style configuration50    >>> configuration = CLIPTextConfig()51 52    >>> # Initializing a CLIPTextModel (with random weights) from the openai/clip-vit-base-patch32 style configuration53    >>> model = CLIPTextModel(configuration)54 55    >>> # Accessing the model configuration56    >>> configuration = model.config57    ```�clip_text_model�������M�58quick_gelu��h㈵��>��{�G�z�?��?�������cslt�jd|
||d�|��||_||_||_||_||_||_||_|	|_	||_59||_||_|60|_
d|_dS)N)�pad_token_id�bos_token_id�eos_token_idF�)�super�__init__�61vocab_size�hidden_size�intermediate_size�projection_dim�num_hidden_layers�num_attention_heads�max_position_embeddings�layer_norm_eps�62hidden_act�initializer_range�initializer_factor�attention_dropout�
add_time_attn)�selfrrrrrr r!r#r"r&r$r%rrr�kwargs��	__class__r�i/scc_cephfs/yy/lb/LLaVA-Video-YY/llava/model/multimodal_encoder/languagebind/depth/configuration_depth.pyrFs63zCLIPTextConfig.__init__�pretrained_model_name_or_path�returnrcK�|�|�|j|fi|��\}}|�d�dkr|d}d|vr:t|d�r:|d|jkr:t�d|d�d|j�d��|j|fi|��S)N�64model_type�clip�text_config�You are using a model of type �  to instantiate a model of type �N. This is not supported for all configurations of models and can yield errors.��_set_token_in_kwargs�get_config_dict�get�hasattrr0�logger�warning�	from_dict��clsr-r)�config_dictrrr,�from_pretrainedk�65 ��zCLIPTextConfig.from_pretrained)rrr	rr66rrr
rrrrrrr�
�__name__�67__module__�__qualname__�__doc__r0r�classmethodr�str�os�PathLikerA�
__classcell__rrr*r,rs*3�%&rcsteZdZdZdZ										68			
											d�fdd�	Zedeee	j69fddfdd��Z�ZS)�CLIPVisionConfiga70    This is the configuration class to store the configuration of a [`CLIPVisionModel`]. It is used to instantiate a71    CLIP vision encoder according to the specified arguments, defining the model architecture. Instantiating a72    configuration with the defaults will yield a similar configuration to that of the vision encoder of the CLIP73    [openai/clip-vit-base-patch32](https://huggingface.co/openai/clip-vit-base-patch32) architecture.74 75    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the76    documentation from [`PretrainedConfig`] for more information.77 78    Args:79        hidden_size (`int`, *optional*, defaults to 768):80            Dimensionality of the encoder layers and the pooler layer.81        intermediate_size (`int`, *optional*, defaults to 3072):82            Dimensionality of the "intermediate" (i.e., feed-forward) layer in the Transformer encoder.83        num_hidden_layers (`int`, *optional*, defaults to 12):84            Number of hidden layers in the Transformer encoder.85        num_attention_heads (`int`, *optional*, defaults to 12):86            Number of attention heads for each attention layer in the Transformer encoder.87        image_size (`int`, *optional*, defaults to 224):88            The size (resolution) of each image.89        patch_size (`int`, *optional*, defaults to 32):90            The size (resolution) of each patch.91        hidden_act (`str` or `function`, *optional*, defaults to `"quick_gelu"`):92            The non-linear activation function (function or string) in the encoder and pooler. If string, `"gelu"`,93            `"relu"`, `"selu"` and `"gelu_new"` ``"quick_gelu"` are supported.94        layer_norm_eps (`float`, *optional*, defaults to 1e-5):95            The epsilon used by the layer normalization layers.96        attention_dropout (`float`, *optional*, defaults to 0.0):97            The dropout ratio for the attention probabilities.98        initializer_range (`float`, *optional*, defaults to 0.02):99            The standard deviation of the truncated_normal_initializer for initializing all weight matrices.100        initializer_factor (`float`, *optional*, defaults to 1):101            A factor for initializing all weight matrices (should be kept to 1, used internally for initialization102            testing).103 104    Example:105 106    ```python107    >>> from transformers import CLIPVisionConfig, CLIPVisionModel108 109    >>> # Initializing a CLIPVisionConfig with openai/clip-vit-base-patch32 style configuration110    >>> configuration = CLIPVisionConfig()111 112    >>> # Initializing a CLIPVisionModel (with random weights) from the openai/clip-vit-base-patch32 style configuration113    >>> model = CLIPVisionModel(configuration)114 115    >>> # Accessing the model configuration116    >>> configuration = model.config117    ```�clip_vision_model��rr118���� r
rrrrFr���119�decordcs�t�jdi|��||_||_||_||_||_||_||_||_	||_120|
|_||_|121|_
|	|_||_||_||_||_||_||_||_||_||_||_dS)Nr)rrrrrrr �num_channels�122patch_size�123image_sizer$r%r&r"r#r'�124num_frames�force_patch_dropout�lora_r�125lora_alpha�lora_dropout�num_mel_bins�
target_length�	max_depth�video_decode_backend)r(rrrrr rXrZrYr#r"r&r$r%r'r[r\r]r^r_r`rarbrcr)r*rr,r�s0126zCLIPVisionConfig.__init__r-r.rcKr/)Nr0r1�
vision_configr3r4r5r6r>rrr,rA�rBz CLIPVisionConfig.from_pretrained)rOrPrr127r128rQrRrSr
rrrrFrrrTrUrrrrVrWrCrrr*r,rM�s:2�7&rMcsJeZdZdZdZdZ	d�fdd�	Zed	ed129e	fdd��Z130d
d�Z�ZS)�LanguageBindDepthConfigaL131    [`CLIPConfig`] is the configuration class to store the configuration of a [`CLIPModel`]. It is used to instantiate132    a CLIP model according to the specified arguments, defining the text model and vision model configs. Instantiating133    a configuration with the defaults will yield a similar configuration to that of the CLIP134    [openai/clip-vit-base-patch32](https://huggingface.co/openai/clip-vit-base-patch32) architecture.135 136    Configuration objects inherit from [`PretrainedConfig`] and can be used to control the model outputs. Read the137    documentation from [`PretrainedConfig`] for more information.138 139    Args:140        text_config (`dict`, *optional*):141            Dictionary of configuration options used to initialize [`CLIPTextConfig`].142        vision_config (`dict`, *optional*):143            Dictionary of configuration options used to initialize [`CLIPVisionConfig`].144        projection_dim (`int`, *optional*, defaults to 512):145            Dimentionality of text and vision projection layers.146        logit_scale_init_value (`float`, *optional*, defaults to 2.6592):147            The inital value of the *logit_scale* paramter. Default is used as per the original CLIP implementation.148        kwargs (*optional*):149            Dictionary of keyword arguments.150 151    Example:152 153    ```python154    >>> from transformers import CLIPConfig, CLIPModel155 156    >>> # Initializing a CLIPConfig with openai/clip-vit-base-patch32 style configuration157    >>> configuration = CLIPConfig()158 159    >>> # Initializing a CLIPModel (with random weights) from the openai/clip-vit-base-patch32 style configuration160    >>> model = CLIPModel(configuration)161 162    >>> # Accessing the model configuration163    >>> configuration = model.config164 165    >>> # We can also initialize a CLIPConfig from a CLIPTextConfig and a CLIPVisionConfig166    >>> from transformers import CLIPTextConfig, CLIPVisionConfig167 168    >>> # Initializing a CLIPText and CLIPVision configuration169    >>> config_text = CLIPTextConfig()170    >>> config_vision = CLIPVisionConfig()171 172    >>> config = CLIPConfig.from_text_vision_configs(config_text, config_vision)173    ```�LanguageBindDepthTNr�/L�174F@c
s�|�dd�}|�dd�}t�jdi|��|dur]|duri}tdi|����}|��D]+\}	}175|	|vrW|176||	krW|	dvrW|	|vrLd|	�d|	�d�}nd|	�d�}t�|�q,|�|�|dur�|durgi}t	di|����}d	|vr�d177d�|d	��D�|d	<|��D]+\}	}178|	|vr�|179||	kr�|	dvr�|	|vr�d|	�d|	�d�}nd
|	�d�}t�|�q�|�|�|dur�i}t�180d�|dur�i}t�181d�tdi|��|_t	di|��|_||_
||_d|_dS)N�text_config_dict�vision_config_dict)�transformers_version�`zp` is found in both `text_config_dict` and `text_config` but with different values. The value `text_config_dict["z"]` will be used instead.zj`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["z"]` will be overriden.�id2labelcSsi|]	\}}t|�|�qSr)rI)�.0�key�valuerrr,�182<dictcomp>as�z4LanguageBindDepthConfig.__init__.<locals>.<dictcomp>zv` is found in both `vision_config_dict` and `vision_config` but with different values. The value `vision_config_dict["zp`vision_config_dict` is provided which will be used to initialize `CLIPVisionConfig`. The value `vision_config["zO`text_config` is `None`. Initializing the `CLIPTextConfig` with default values.zS`vision_config` is `None`. initializing the `CLIPVisionConfig` with default values.rr)�poprrr�to_dict�itemsr;r<�updaterM�infor2rdr�logit_scale_init_valuer%)
r(r2rdrrvr)rhri�_text_config_dictrnro�message�_vision_config_dictr*rr,r0sl����183�184185186�����187�188189190191z LanguageBindDepthConfig.__init__r2rdcKs|d|��|��d�|��S)z�192        Instantiate a [`CLIPConfig`] (or a derived class) from clip text model configuration and clip vision model193        configuration.194 195        Returns:196            [`CLIPConfig`]: An instance of a configuration object197        )r2rdNr)rr)r?r2rdr)rrr,�from_text_vision_configs�s198z0LanguageBindDepthConfig.from_text_vision_configscCs8t�|j�}|j��|d<|j��|d<|jj|d<|S)z�199        Serializes this instance to a Python dictionary. Override the default [`~PretrainedConfig.to_dict`].200 201        Returns:202            `Dict[str, any]`: Dictionary of all the attributes that make up this configuration instance,203        r2rdr0)�copy�deepcopy�__dict__r2rrrdr+r0)r(�outputrrr,rr�s204zLanguageBindDepthConfig.to_dict)NNrrg)
rDrErFrGr0�is_compositionrrHrrMrzrrrLrrr*r,re�s-�Xre)r{rJ�typingr�transformersr�transformers.utilsr�205get_loggerrDr;rrMrerrrr,�<module>s206p