keras/flan_base_multi

1---2library_name: keras-hub3license: apache-2.04tags:5- text-classification6- keras7pipeline_tag: text-generation8---9### Model Overview10⚠️ T5 is currently only available via the `keras-hub-nightly` package. Use `pip install keras-hub-nightly` to try this model.11 12T5 encoder-decoder backbone model.13 14T5 is a LLM pretrained on a mix of unsupervised and supervised tasks,15where each task is converted to a sequence-to-sequence format.16T5 works well on a variety of tasks out-of-the-box by prepending17various prefixes to the input sequence, e.g., for translation:18`"translate English to German: ..."`, for summarization:19`"summarize: ..."`.20 21T5 was introduced in22[Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://arxiv.org/abs/1910.10683)23 24The default constructor gives a fully customizable, randomly initialized T525model with any number of layers, heads, and embedding dimensions. To load26preset architectures and weights, use the `from_preset` constructor.27 28Disclaimer: Pre-trained models are provided on an "as is" basis, without29warranties or conditions of any kind.30 31## Links32 33* [T5 Quickstart Notebook](coming soon)34* [T5  API Documentation](https://keras.io/keras_hub/api/models/t5/)35* [T5 Model Card](https://github.com/google-research/text-to-text-transfer-transformer/tree/main)36* [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)37* [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)38 39## Installation40 41Keras and KerasHub can be installed with:42 43```44pip install -U -q keras-hub45pip install -U -q keras46```47 48Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.49 50## Presets51 52The following model checkpoints are provided by the Keras team. Full code examples for each are available below.53| Preset name    | Parameters | Description                                      |54|----------------|------------|--------------------------------------------------|55| t5_small_multi |   0    | 8-layer T5 model. Trained on the Colossal Clean Crawled Corpus (C4).|56| t5_base_multi|    0  | 12-layer T5 model. Trained on the Colossal Clean Crawled Corpus (C4). |57| t5_large_multi |   0   | 24-layer T5 model. Trained on the Colossal Clean Crawled Corpus (C4).   |58| flan_small_multi |   0    |  8-layer T5 model. Trained on the Colossal Clean Crawled Corpus (C4).  |59| flan_base_multi |   0    |  12-layer T5 model. Trained on the Colossal Clean Crawled Corpus (C4). |60| flan_large_multi |   0    |  	24-layer T5 model. Trained on the Colossal Clean Crawled Corpus (C4).  |61| t5_1.1_small |   60.51M |    |62| tt5_1.1_base |  247.58M |    | 63| t5_1.1_large | 750.25M  |    | 64| t5_1.1_xl | 2.85B  |    | 65| t5_1.1_xxl | 11.14B  |    | 66 67__Arguments__68 69 70- __vocabulary_size__: int. The size of the token vocabulary.71- __num_layers__: int. The number of Transformer layers.72- __num_heads__: int. The number of attention heads for each Transformer.73    The hidden size must be divisible by the number of attention heads.74- __hidden_dim__: int. The hidden size of the Transformer layers.75- __intermediate_dim__: int. The output dimension of the first Dense layer in76    a two-layer feedforward network for each Transformer layer.77- __key_value_dim__: int. The dimension of each head of the key/value78    projections in the multi-head attention layers. Defaults to79    hidden_dim / num_heads.80- __dropout__: float. Dropout probability for the Transformer layers.81- __activation__: activation function (or activation string name). The82    activation to be used in the inner dense blocks of the83    Transformer layers. Defaults to `"relu"`.84- __use_gated_activation__: boolean. Whether to use activation gating in85    the inner dense blocks of the Transformer layers.86    The original T5 architecture didn't use gating, but more87    recent versions do. Defaults to `True`.88- __layer_norm_epsilon__: float. Epsilon factor to be used in the89    layer normalization layers in the Transformer layers.90- __tie_embedding_weights__: boolean. If `True`, the weights of the token91    embedding and the weights projecting language model outputs from92    `hidden_dim`93 94