Modelpublic

CATIE-AQ/FAT5-large-flan-en

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face

0likes8downloads

Model Card

Description

Adaptation of the flan-t5-large weights to make it compatible with the FAT5 framework (Flash Attention T5). This adaptation should enable the user to efficiently continue the pre-training of the flan-t5 to adapt it to more recent data, or to specialize it in a specific domain, for example.

Usage

from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained("CATIE-AQ/FAT5-large-flan-en", trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained("google/flan-t5-large")