Aluode/PerceptionLabPortable
0
1# Copyright 2020 The HuggingFace Team. All rights reserved.2#3# Licensed under the Apache License, Version 2.0 (the "License");4# you may not use this file except in compliance with the License.5# You may obtain a copy of the License at6#7# http://www.apache.org/licenses/LICENSE-2.08#9# Unless required by applicable law or agreed to in writing, software10# distributed under the License is distributed on an "AS IS" BASIS,11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.12# See the License for the specific language governing permissions and13# limitations under the License.14 15import warnings16from dataclasses import dataclass17from typing import Optional18 19import torch20 21from .cache_utils import Cache, EncoderDecoderCache22from .utils import ModelOutput23 24 25@dataclass26class BaseModelOutput(ModelOutput):27 """28 Base class for model's outputs, with potential hidden states and attentions.29 30 Args:31 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):32 Sequence of hidden-states at the output of the last layer of the model.33 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):34 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +35 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.36 37 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.38 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):39 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,40 sequence_length)`.41 42 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention43 heads.44 """45 46 last_hidden_state: Optional[torch.FloatTensor] = None47 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None48 attentions: Optional[tuple[torch.FloatTensor, ...]] = None49 50 51@dataclass52class BaseModelOutputWithNoAttention(ModelOutput):53 """54 Base class for model's outputs, with potential hidden states.55 56 Args:57 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`):58 Sequence of hidden-states at the output of the last layer of the model.59 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):60 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +61 one for the output of each layer) of shape `(batch_size, num_channels, height, width)`.62 63 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.64 """65 66 last_hidden_state: Optional[torch.FloatTensor] = None67 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None68 69 70@dataclass71class BaseModelOutputWithPooling(ModelOutput):72 """73 Base class for model's outputs that also contains a pooling of the last hidden states.74 75 Args:76 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):77 Sequence of hidden-states at the output of the last layer of the model.78 pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`):79 Last layer hidden-state of the first token of the sequence (classification token) after further processing80 through the layers used for the auxiliary pretraining task. E.g. for BERT-family of models, this returns81 the classification token after processing through a linear layer and a tanh activation function. The linear82 layer weights are trained from the next sentence prediction (classification) objective during pretraining.83 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):84 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +85 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.86 87 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.88 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):89 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,90 sequence_length)`.91 92 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention93 heads.94 """95 96 last_hidden_state: Optional[torch.FloatTensor] = None97 pooler_output: Optional[torch.FloatTensor] = None98 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None99 attentions: Optional[tuple[torch.FloatTensor, ...]] = None100 101 102@dataclass103class BaseModelOutputWithPoolingAndNoAttention(ModelOutput):104 """105 Base class for model's outputs that also contains a pooling of the last hidden states.106 107 Args:108 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, num_channels, height, width)`):109 Sequence of hidden-states at the output of the last layer of the model.110 pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`):111 Last layer hidden-state after a pooling operation on the spatial dimensions.112 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):113 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +114 one for the output of each layer) of shape `(batch_size, num_channels, height, width)`.115 116 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.117 """118 119 last_hidden_state: Optional[torch.FloatTensor] = None120 pooler_output: Optional[torch.FloatTensor] = None121 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None122 123 124@dataclass125class BaseModelOutputWithPast(ModelOutput):126 """127 Base class for model's outputs that may also contain a past key/values (to speed up sequential decoding).128 129 Args:130 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):131 Sequence of hidden-states at the output of the last layer of the model.132 133 If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1,134 hidden_size)` is output.135 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):136 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).137 138 Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if139 `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values`140 input) to speed up sequential decoding.141 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):142 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +143 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.144 145 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.146 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):147 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,148 sequence_length)`.149 150 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention151 heads.152 """153 154 last_hidden_state: Optional[torch.FloatTensor] = None155 past_key_values: Optional[Cache] = None156 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None157 attentions: Optional[tuple[torch.FloatTensor, ...]] = None158 159 160@dataclass161class BaseModelOutputWithCrossAttentions(ModelOutput):162 """163 Base class for model's outputs, with potential hidden states and attentions.164 165 Args:166 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):167 Sequence of hidden-states at the output of the last layer of the model.168 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):169 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +170 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.171 172 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.173 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):174 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,175 sequence_length)`.176 177 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention178 heads.179 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` and `config.add_cross_attention=True` is passed or when `config.output_attentions=True`):180 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,181 sequence_length)`.182 183 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the184 weighted average in the cross-attention heads.185 """186 187 last_hidden_state: Optional[torch.FloatTensor] = None188 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None189 attentions: Optional[tuple[torch.FloatTensor, ...]] = None190 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None191 192 193@dataclass194class BaseModelOutputWithPoolingAndCrossAttentions(ModelOutput):195 """196 Base class for model's outputs that also contains a pooling of the last hidden states.197 198 Args:199 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):200 Sequence of hidden-states at the output of the last layer of the model.201 pooler_output (`torch.FloatTensor` of shape `(batch_size, hidden_size)`):202 Last layer hidden-state of the first token of the sequence (classification token) after further processing203 through the layers used for the auxiliary pretraining task. E.g. for BERT-family of models, this returns204 the classification token after processing through a linear layer and a tanh activation function. The linear205 layer weights are trained from the next sentence prediction (classification) objective during pretraining.206 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):207 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +208 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.209 210 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.211 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):212 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,213 sequence_length)`.214 215 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention216 heads.217 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` and `config.add_cross_attention=True` is passed or when `config.output_attentions=True`):218 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,219 sequence_length)`.220 221 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the222 weighted average in the cross-attention heads.223 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):224 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).225 226 Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if227 `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values`228 input) to speed up sequential decoding.229 """230 231 last_hidden_state: Optional[torch.FloatTensor] = None232 pooler_output: Optional[torch.FloatTensor] = None233 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None234 past_key_values: Optional[Cache] = None235 attentions: Optional[tuple[torch.FloatTensor, ...]] = None236 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None237 238 239@dataclass240class BaseModelOutputWithPastAndCrossAttentions(ModelOutput):241 """242 Base class for model's outputs that may also contain a past key/values (to speed up sequential decoding).243 244 Args:245 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):246 Sequence of hidden-states at the output of the last layer of the model.247 248 If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1,249 hidden_size)` is output.250 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):251 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).252 253 Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if254 `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values`255 input) to speed up sequential decoding.256 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):257 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +258 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.259 260 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.261 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):262 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,263 sequence_length)`.264 265 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention266 heads.267 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` and `config.add_cross_attention=True` is passed or when `config.output_attentions=True`):268 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,269 sequence_length)`.270 271 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the272 weighted average in the cross-attention heads.273 """274 275 last_hidden_state: Optional[torch.FloatTensor] = None276 past_key_values: Optional[Cache] = None277 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None278 attentions: Optional[tuple[torch.FloatTensor, ...]] = None279 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None280 281 282@dataclass283class MoECausalLMOutputWithPast(ModelOutput):284 """285 Base class for causal language model (or autoregressive) outputs as well as Mixture of Expert's router hidden286 states terms, to train a MoE model.287 288 Args:289 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):290 Language modeling loss (for next-token prediction).291 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`):292 Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).293 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):294 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).295 296 Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see297 `past_key_values` input) to speed up sequential decoding.298 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):299 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +300 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.301 302 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.303 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):304 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,305 sequence_length)`.306 307 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention308 heads.309 z_loss (`torch.FloatTensor`, *optional*, returned when `labels` is provided):310 z_loss for the sparse modules.311 aux_loss (`torch.FloatTensor`, *optional*, returned when `labels` is provided):312 aux_loss for the sparse modules.313 router_logits (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_logits=True` is passed or when `config.add_router_probs=True`):314 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.315 316 Router logits of the encoder model, useful to compute the auxiliary loss and the z_loss for the sparse317 modules.318 """319 320 loss: Optional[torch.FloatTensor] = None321 logits: Optional[torch.FloatTensor] = None322 past_key_values: Optional[Cache] = None323 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None324 attentions: Optional[tuple[torch.FloatTensor, ...]] = None325 z_loss: Optional[torch.FloatTensor] = None326 aux_loss: Optional[torch.FloatTensor] = None327 router_logits: Optional[tuple[torch.FloatTensor]] = None328 329 330@dataclass331class MoEModelOutput(ModelOutput):332 """333 Base class for model's outputs, with potential hidden states and attentions.334 335 Args:336 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):337 Sequence of hidden-states at the output of the last layer of the model.338 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):339 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +340 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.341 342 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.343 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):344 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,345 sequence_length)`.346 347 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention348 heads.349 router_probs (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_probs=True` and `config.add_router_probs=True` is passed or when `config.output_router_probs=True`):350 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.351 352 Raw router probabilities that are computed by MoE routers, these terms are used to compute the auxiliary353 loss and the z_loss for Mixture of Experts models.354 """355 356 last_hidden_state: Optional[torch.FloatTensor] = None357 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None358 attentions: Optional[tuple[torch.FloatTensor, ...]] = None359 router_probs: Optional[tuple[torch.FloatTensor]] = None360 361 362@dataclass363class MoeModelOutputWithPast(ModelOutput):364 """365 Base class for model's outputs, with potential hidden states and attentions.366 367 Args:368 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):369 Sequence of hidden-states at the output of the last layer of the model.370 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):371 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).372 373 Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if374 `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values`375 input) to speed up sequential decoding.376 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):377 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +378 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.379 380 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.381 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):382 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,383 sequence_length)`.384 385 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention386 heads.387 router_logits (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_probs=True` and `config.add_router_probs=True` is passed or when `config.output_router_probs=True`):388 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.389 390 Raw router logtis (post-softmax) that are computed by MoE routers, these terms are used to compute the auxiliary391 loss for Mixture of Experts models.392 """393 394 last_hidden_state: Optional[torch.FloatTensor] = None395 past_key_values: Optional[Cache] = None396 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None397 attentions: Optional[tuple[torch.FloatTensor, ...]] = None398 router_logits: Optional[tuple[torch.FloatTensor]] = None399 400 401@dataclass402class MoeCausalLMOutputWithPast(ModelOutput):403 """404 Base class for causal language model (or autoregressive) with mixture of experts outputs.405 406 Args:407 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):408 Language modeling loss (for next-token prediction).409 410 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`):411 Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).412 413 aux_loss (`torch.FloatTensor`, *optional*, returned when `labels` is provided):414 aux_loss for the sparse modules.415 416 router_logits (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_probs=True` and `config.add_router_probs=True` is passed or when `config.output_router_probs=True`):417 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.418 419 Raw router logtis (post-softmax) that are computed by MoE routers, these terms are used to compute the auxiliary420 loss for Mixture of Experts models.421 422 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):423 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).424 425 Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see426 `past_key_values` input) to speed up sequential decoding.427 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):428 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +429 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.430 431 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.432 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):433 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,434 sequence_length)`.435 436 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention437 heads.438 """439 440 loss: Optional[torch.FloatTensor] = None441 aux_loss: Optional[torch.FloatTensor] = None442 logits: Optional[torch.FloatTensor] = None443 past_key_values: Optional[Cache] = None444 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None445 attentions: Optional[tuple[torch.FloatTensor, ...]] = None446 router_logits: Optional[tuple[torch.FloatTensor]] = None447 448 449@dataclass450class MoEModelOutputWithPastAndCrossAttentions(ModelOutput):451 """452 Base class for model's outputs that may also contain a past key/values (to speed up sequential decoding) as well as453 Mixture of Expert's router hidden states terms, to train a MoE model.454 455 Args:456 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):457 Sequence of hidden-states at the output of the last layer of the model.458 459 If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1,460 hidden_size)` is output.461 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):462 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).463 464 Contains pre-computed hidden-states (key and values in the self-attention blocks and optionally if465 `config.is_encoder_decoder=True` in the cross-attention blocks) that can be used (see `past_key_values`466 input) to speed up sequential decoding.467 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):468 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +469 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.470 471 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.472 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):473 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,474 sequence_length)`.475 476 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention477 heads.478 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` and `config.add_cross_attention=True` is passed or when `config.output_attentions=True`):479 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,480 sequence_length)`.481 482 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the483 weighted average in the cross-attention heads.484 router_probs (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_probs=True` and `config.add_router_probs=True` is passed or when `config.output_router_probs=True`):485 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.486 487 Raw router probabilities that are computed by MoE routers, these terms are used to compute the auxiliary488 loss and the z_loss for Mixture of Experts models.489 """490 491 last_hidden_state: Optional[torch.FloatTensor] = None492 past_key_values: Optional[Cache] = None493 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None494 attentions: Optional[tuple[torch.FloatTensor, ...]] = None495 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None496 router_probs: Optional[tuple[torch.FloatTensor]] = None497 498 499@dataclass500class Seq2SeqModelOutput(ModelOutput):501 """502 Base class for model encoder's outputs that also contains : pre-computed hidden states that can speed up sequential503 decoding.504 505 Args:506 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):507 Sequence of hidden-states at the output of the last layer of the decoder of the model.508 509 If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1,510 hidden_size)` is output.511 past_key_values (`EncoderDecoderCache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):512 It is a [`~cache_utils.EncoderDecoderCache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).513 514 Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention515 blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.516 decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):517 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +518 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.519 520 Hidden-states of the decoder at the output of each layer plus the optional initial embedding outputs.521 decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):522 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,523 sequence_length)`.524 525 Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the526 self-attention heads.527 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):528 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,529 sequence_length)`.530 531 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the532 weighted average in the cross-attention heads.533 encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*):534 Sequence of hidden-states at the output of the last layer of the encoder of the model.535 encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):536 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +537 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.538 539 Hidden-states of the encoder at the output of each layer plus the optional initial embedding outputs.540 encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):541 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,542 sequence_length)`.543 544 Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the545 self-attention heads.546 """547 548 last_hidden_state: Optional[torch.FloatTensor] = None549 past_key_values: Optional[EncoderDecoderCache] = None550 decoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None551 decoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None552 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None553 encoder_last_hidden_state: Optional[torch.FloatTensor] = None554 encoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None555 encoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None556 557 558@dataclass559class Seq2SeqMoEModelOutput(ModelOutput):560 """561 Base class for model encoder's outputs that also contains : pre-computed hidden states that can speed up sequential562 decoding.563 564 Args:565 last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`):566 Sequence of hidden-states at the output of the last layer of the decoder of the model.567 568 If `past_key_values` is used only the last hidden-state of the sequences of shape `(batch_size, 1,569 hidden_size)` is output.570 past_key_values (`EncoderDecoderCache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):571 It is a [`~cache_utils.EncoderDecoderCache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).572 573 Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention574 blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.575 decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):576 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +577 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.578 579 Hidden-states of the decoder at the output of each layer plus the optional initial embedding outputs.580 decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):581 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,582 sequence_length)`.583 584 Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the585 self-attention heads.586 decoder_router_logits (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_logits=True` is passed or when `config.add_router_probs=True`):587 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.588 589 Router logits of the decoder model, useful to compute the auxiliary loss for Mixture of Experts models.590 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):591 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,592 sequence_length)`.593 594 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the595 weighted average in the cross-attention heads.596 encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*):597 Sequence of hidden-states at the output of the last layer of the encoder of the model.598 encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):599 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +600 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.601 602 Hidden-states of the encoder at the output of each layer plus the optional initial embedding outputs.603 encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):604 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,605 sequence_length)`.606 607 Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the608 self-attention heads.609 encoder_router_logits (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_logits=True` is passed or when `config.add_router_probs=True`):610 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.611 612 Router logits of the encoder model, useful to compute the auxiliary loss and the z_loss for the sparse613 modules.614 """615 616 last_hidden_state: Optional[torch.FloatTensor] = None617 past_key_values: Optional[EncoderDecoderCache] = None618 decoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None619 decoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None620 decoder_router_logits: Optional[tuple[torch.FloatTensor]] = None621 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None622 encoder_last_hidden_state: Optional[torch.FloatTensor] = None623 encoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None624 encoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None625 encoder_router_logits: Optional[tuple[torch.FloatTensor]] = None626 627 628@dataclass629class CausalLMOutput(ModelOutput):630 """631 Base class for causal language model (or autoregressive) outputs.632 633 Args:634 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):635 Language modeling loss (for next-token prediction).636 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`):637 Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).638 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):639 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +640 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.641 642 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.643 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):644 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,645 sequence_length)`.646 647 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention648 heads.649 """650 651 loss: Optional[torch.FloatTensor] = None652 logits: Optional[torch.FloatTensor] = None653 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None654 attentions: Optional[tuple[torch.FloatTensor, ...]] = None655 656 657@dataclass658class CausalLMOutputWithPast(ModelOutput):659 """660 Base class for causal language model (or autoregressive) outputs.661 662 Args:663 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):664 Language modeling loss (for next-token prediction).665 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`):666 Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).667 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):668 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).669 670 Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see671 `past_key_values` input) to speed up sequential decoding.672 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):673 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +674 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.675 676 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.677 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):678 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,679 sequence_length)`.680 681 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention682 heads.683 """684 685 loss: Optional[torch.FloatTensor] = None686 logits: Optional[torch.FloatTensor] = None687 past_key_values: Optional[Cache] = None688 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None689 attentions: Optional[tuple[torch.FloatTensor, ...]] = None690 691 692@dataclass693class CausalLMOutputWithCrossAttentions(ModelOutput):694 """695 Base class for causal language model (or autoregressive) outputs.696 697 Args:698 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):699 Language modeling loss (for next-token prediction).700 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`):701 Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).702 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):703 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +704 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.705 706 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.707 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):708 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,709 sequence_length)`.710 711 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention712 heads.713 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):714 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,715 sequence_length)`.716 717 Cross attentions weights after the attention softmax, used to compute the weighted average in the718 cross-attention heads.719 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):720 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).721 722 Contains pre-computed hidden-states (key and values in the attention blocks) that can be used (see723 `past_key_values` input) to speed up sequential decoding.724 """725 726 loss: Optional[torch.FloatTensor] = None727 logits: Optional[torch.FloatTensor] = None728 past_key_values: Optional[Cache] = None729 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None730 attentions: Optional[tuple[torch.FloatTensor, ...]] = None731 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None732 733 734@dataclass735class SequenceClassifierOutputWithPast(ModelOutput):736 """737 Base class for outputs of sentence classification models.738 739 Args:740 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):741 Classification (or regression if config.num_labels==1) loss.742 logits (`torch.FloatTensor` of shape `(batch_size, config.num_labels)`):743 Classification (or regression if config.num_labels==1) scores (before SoftMax).744 past_key_values (`Cache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):745 It is a [`~cache_utils.Cache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).746 747 Contains pre-computed hidden-states (key and values in the self-attention blocks) that can be used (see748 `past_key_values` input) to speed up sequential decoding.749 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):750 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +751 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.752 753 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.754 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):755 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,756 sequence_length)`.757 758 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention759 heads.760 """761 762 loss: Optional[torch.FloatTensor] = None763 logits: Optional[torch.FloatTensor] = None764 past_key_values: Optional[Cache] = None765 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None766 attentions: Optional[tuple[torch.FloatTensor, ...]] = None767 768 769@dataclass770class MaskedLMOutput(ModelOutput):771 """772 Base class for masked language models outputs.773 774 Args:775 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):776 Masked language modeling (MLM) loss.777 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`):778 Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).779 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):780 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +781 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.782 783 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.784 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):785 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,786 sequence_length)`.787 788 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention789 heads.790 """791 792 loss: Optional[torch.FloatTensor] = None793 logits: Optional[torch.FloatTensor] = None794 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None795 attentions: Optional[tuple[torch.FloatTensor, ...]] = None796 797 798@dataclass799class Seq2SeqLMOutput(ModelOutput):800 """801 Base class for sequence-to-sequence language models outputs.802 803 Args:804 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):805 Language modeling loss.806 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`):807 Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).808 past_key_values (`EncoderDecoderCache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):809 It is a [`~cache_utils.EncoderDecoderCache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).810 811 Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention812 blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.813 decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):814 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +815 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.816 817 Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.818 decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):819 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,820 sequence_length)`.821 822 Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the823 self-attention heads.824 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):825 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,826 sequence_length)`.827 828 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the829 weighted average in the cross-attention heads.830 encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*):831 Sequence of hidden-states at the output of the last layer of the encoder of the model.832 encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):833 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +834 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.835 836 Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.837 encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):838 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,839 sequence_length)`.840 841 Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the842 self-attention heads.843 """844 845 loss: Optional[torch.FloatTensor] = None846 logits: Optional[torch.FloatTensor] = None847 past_key_values: Optional[EncoderDecoderCache] = None848 decoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None849 decoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None850 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None851 encoder_last_hidden_state: Optional[torch.FloatTensor] = None852 encoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None853 encoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None854 855 856@dataclass857class Seq2SeqMoEOutput(ModelOutput):858 """859 Base class for sequence-to-sequence language models outputs.860 861 Args:862 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):863 Language modeling loss.864 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.vocab_size)`):865 Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).866 past_key_values (`EncoderDecoderCache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):867 It is a [`~cache_utils.EncoderDecoderCache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).868 869 Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention870 blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.871 decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):872 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +873 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.874 875 Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.876 decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):877 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,878 sequence_length)`.879 880 Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the881 self-attention heads.882 decoder_router_logits (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_logits=True` is passed or when `config.add_router_probs=True`):883 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.884 885 Router logits of the decoder model, useful to compute the auxiliary loss for Mixture of Experts models.886 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):887 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,888 sequence_length)`.889 890 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the891 weighted average in the cross-attention heads.892 encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*):893 Sequence of hidden-states at the output of the last layer of the encoder of the model.894 encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):895 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +896 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.897 898 Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.899 encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):900 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,901 sequence_length)`.902 903 Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the904 self-attention heads.905 encoder_router_logits (`tuple(torch.FloatTensor)`, *optional*, returned when `output_router_logits=True` is passed or when `config.add_router_probs=True`):906 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, sequence_length, num_experts)`.907 908 Router logits of the encoder model, useful to compute the auxiliary loss and z_loss for Mixture of Experts909 models.910 """911 912 loss: Optional[torch.FloatTensor] = None913 logits: Optional[torch.FloatTensor] = None914 encoder_z_loss: Optional[torch.FloatTensor] = None915 decoder_z_loss: Optional[torch.FloatTensor] = None916 encoder_aux_loss: Optional[torch.FloatTensor] = None917 decoder_aux_loss: Optional[torch.FloatTensor] = None918 past_key_values: Optional[EncoderDecoderCache] = None919 decoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None920 decoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None921 decoder_router_logits: Optional[tuple[torch.FloatTensor]] = None922 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None923 encoder_last_hidden_state: Optional[torch.FloatTensor] = None924 encoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None925 encoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None926 encoder_router_logits: Optional[tuple[torch.FloatTensor]] = None927 928 929@dataclass930class NextSentencePredictorOutput(ModelOutput):931 """932 Base class for outputs of models predicting if two sentences are consecutive or not.933 934 Args:935 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `next_sentence_label` is provided):936 Next sequence prediction (classification) loss.937 logits (`torch.FloatTensor` of shape `(batch_size, 2)`):938 Prediction scores of the next sequence prediction (classification) head (scores of True/False continuation939 before SoftMax).940 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):941 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +942 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.943 944 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.945 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):946 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,947 sequence_length)`.948 949 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention950 heads.951 """952 953 loss: Optional[torch.FloatTensor] = None954 logits: Optional[torch.FloatTensor] = None955 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None956 attentions: Optional[tuple[torch.FloatTensor, ...]] = None957 958 959@dataclass960class SequenceClassifierOutput(ModelOutput):961 """962 Base class for outputs of sentence classification models.963 964 Args:965 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):966 Classification (or regression if config.num_labels==1) loss.967 logits (`torch.FloatTensor` of shape `(batch_size, config.num_labels)`):968 Classification (or regression if config.num_labels==1) scores (before SoftMax).969 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):970 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +971 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.972 973 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.974 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):975 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,976 sequence_length)`.977 978 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention979 heads.980 """981 982 loss: Optional[torch.FloatTensor] = None983 logits: Optional[torch.FloatTensor] = None984 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None985 attentions: Optional[tuple[torch.FloatTensor, ...]] = None986 987 988@dataclass989class Seq2SeqSequenceClassifierOutput(ModelOutput):990 """991 Base class for outputs of sequence-to-sequence sentence classification models.992 993 Args:994 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `label` is provided):995 Classification (or regression if config.num_labels==1) loss.996 logits (`torch.FloatTensor` of shape `(batch_size, config.num_labels)`):997 Classification (or regression if config.num_labels==1) scores (before SoftMax).998 past_key_values (`EncoderDecoderCache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):999 It is a [`~cache_utils.EncoderDecoderCache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).1000 1001 Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention1002 blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.1003 decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):1004 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +1005 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.1006 1007 Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.1008 decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1009 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1010 sequence_length)`.1011 1012 Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the1013 self-attention heads.1014 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1015 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1016 sequence_length)`.1017 1018 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the1019 weighted average in the cross-attention heads.1020 encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*):1021 Sequence of hidden-states at the output of the last layer of the encoder of the model.1022 encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):1023 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +1024 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.1025 1026 Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.1027 encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1028 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1029 sequence_length)`.1030 1031 Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the1032 self-attention heads.1033 """1034 1035 loss: Optional[torch.FloatTensor] = None1036 logits: Optional[torch.FloatTensor] = None1037 past_key_values: Optional[EncoderDecoderCache] = None1038 decoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None1039 decoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None1040 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None1041 encoder_last_hidden_state: Optional[torch.FloatTensor] = None1042 encoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None1043 encoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None1044 1045 1046@dataclass1047class MultipleChoiceModelOutput(ModelOutput):1048 """1049 Base class for outputs of multiple choice models.1050 1051 Args:1052 loss (`torch.FloatTensor` of shape *(1,)*, *optional*, returned when `labels` is provided):1053 Classification loss.1054 logits (`torch.FloatTensor` of shape `(batch_size, num_choices)`):1055 *num_choices* is the second dimension of the input tensors. (see *input_ids* above).1056 1057 Classification scores (before SoftMax).1058 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):1059 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +1060 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.1061 1062 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.1063 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1064 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1065 sequence_length)`.1066 1067 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention1068 heads.1069 """1070 1071 loss: Optional[torch.FloatTensor] = None1072 logits: Optional[torch.FloatTensor] = None1073 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None1074 attentions: Optional[tuple[torch.FloatTensor, ...]] = None1075 1076 1077@dataclass1078class TokenClassifierOutput(ModelOutput):1079 """1080 Base class for outputs of token classification models.1081 1082 Args:1083 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided) :1084 Classification loss.1085 logits (`torch.FloatTensor` of shape `(batch_size, sequence_length, config.num_labels)`):1086 Classification scores (before SoftMax).1087 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):1088 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +1089 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.1090 1091 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.1092 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1093 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1094 sequence_length)`.1095 1096 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention1097 heads.1098 """1099 1100 loss: Optional[torch.FloatTensor] = None1101 logits: Optional[torch.FloatTensor] = None1102 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None1103 attentions: Optional[tuple[torch.FloatTensor, ...]] = None1104 1105 1106@dataclass1107class QuestionAnsweringModelOutput(ModelOutput):1108 """1109 Base class for outputs of question answering models.1110 1111 Args:1112 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):1113 Total span extraction loss is the sum of a Cross-Entropy for the start and end positions.1114 start_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length)`):1115 Span-start scores (before SoftMax).1116 end_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length)`):1117 Span-end scores (before SoftMax).1118 hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):1119 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +1120 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.1121 1122 Hidden-states of the model at the output of each layer plus the optional initial embedding outputs.1123 attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1124 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1125 sequence_length)`.1126 1127 Attentions weights after the attention softmax, used to compute the weighted average in the self-attention1128 heads.1129 """1130 1131 loss: Optional[torch.FloatTensor] = None1132 start_logits: Optional[torch.FloatTensor] = None1133 end_logits: Optional[torch.FloatTensor] = None1134 hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None1135 attentions: Optional[tuple[torch.FloatTensor, ...]] = None1136 1137 1138@dataclass1139class Seq2SeqQuestionAnsweringModelOutput(ModelOutput):1140 """1141 Base class for outputs of sequence-to-sequence question answering models.1142 1143 Args:1144 loss (`torch.FloatTensor` of shape `(1,)`, *optional*, returned when `labels` is provided):1145 Total span extraction loss is the sum of a Cross-Entropy for the start and end positions.1146 start_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length)`):1147 Span-start scores (before SoftMax).1148 end_logits (`torch.FloatTensor` of shape `(batch_size, sequence_length)`):1149 Span-end scores (before SoftMax).1150 past_key_values (`EncoderDecoderCache`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`):1151 It is a [`~cache_utils.EncoderDecoderCache`] instance. For more details, see our [kv cache guide](https://huggingface.co/docs/transformers/en/kv_cache).1152 1153 Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention1154 blocks) that can be used (see `past_key_values` input) to speed up sequential decoding.1155 decoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):1156 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +1157 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.1158 1159 Hidden-states of the decoder at the output of each layer plus the initial embedding outputs.1160 decoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1161 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1162 sequence_length)`.1163 1164 Attentions weights of the decoder, after the attention softmax, used to compute the weighted average in the1165 self-attention heads.1166 cross_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1167 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1168 sequence_length)`.1169 1170 Attentions weights of the decoder's cross-attention layer, after the attention softmax, used to compute the1171 weighted average in the cross-attention heads.1172 encoder_last_hidden_state (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*):1173 Sequence of hidden-states at the output of the last layer of the encoder of the model.1174 encoder_hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`):1175 Tuple of `torch.FloatTensor` (one for the output of the embeddings, if the model has an embedding layer, +1176 one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`.1177 1178 Hidden-states of the encoder at the output of each layer plus the initial embedding outputs.1179 encoder_attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`):1180 Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length,1181 sequence_length)`.1182 1183 Attentions weights of the encoder, after the attention softmax, used to compute the weighted average in the1184 self-attention heads.1185 """1186 1187 loss: Optional[torch.FloatTensor] = None1188 start_logits: Optional[torch.FloatTensor] = None1189 end_logits: Optional[torch.FloatTensor] = None1190 past_key_values: Optional[EncoderDecoderCache] = None1191 decoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None1192 decoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None1193 cross_attentions: Optional[tuple[torch.FloatTensor, ...]] = None1194 encoder_last_hidden_state: Optional[torch.FloatTensor] = None1195 encoder_hidden_states: Optional[tuple[torch.FloatTensor, ...]] = None1196 encoder_attentions: Optional[tuple[torch.FloatTensor, ...]] = None1197 1198 1199@dataclass1200class SemanticSegmenterOutput(ModelOutput):