Flan-T5
Datasets
All datasets matching “Flan-T5”flan-t5-large-embed-refinedwebAll of the data together is around 81.3GB. It's the last hidden states of 131,072 samples from refinedweb padded/truncated to 512 tokens on the left, fed through google/flan-t5-base.
Structure:
{
"encoding": List, shaped (512, 1024) aka (tokens, d_model),
"text": String, the original text that was encoded,
"attention_mask": List, binary mask to pass to your model with encoding to not attend to pad tokens
}
flan-t5-base-embed-refinedwebAll of the data together is around 61GB. It's the last hidden states of 131,072 samples from refinedweb padded/truncated to 512 tokens on the left, fed through google/flan-t5-base.
Structure:
{
"encoding": List, shaped (512, 768) aka (tokens, d_model),
"text": String, the original text that was encoded,
"attention_mask": List, binary mask to pass to your model with encoding to not attend to pad tokens
}
flan-t5-small-embed-refinedwebAll of the data together is around 41GB. It's the last hidden states of 131,072 samples from refinedweb padded/truncated to 512 tokens on the left, fed through google/flan-t5-small.
Structure:
{
"encoding": List, shaped (512, 512) aka (tokens, d_model),
"text": String, the original text that was encoded,
"attention_mask": List, binary mask to pass to your model with encoding to not attend to pad tokens
}
just a tip, you cannot load this with the RAM in the free ver of google colab, not… See the full description on the dataset page: https://huggingface.co/datasets/crumb/flan-t5-small-embed-refinedweb.flan-t5-boosting-mmlu_cotflan-t5-xl-embed-refinedwebpeanuts-flan-t5-xl
Peanut Comic Strip Dataset (Snoopy & Co.)
This is a dataset Peanuts comic strips from 1950/10/02 to 2000/02/13.
There are 77,456 panels extracted from 17,816 comic strips.
The dataset size is approximately 4.4G.
Each row in the dataset contains the following fields:
image: PIL.Image containing the extracted panel.
panel_name: unique identifier for the row.
characters: tuple[str, ...] of characters included in the comic strip the panel is part of.
themes: tuple[str, ...] of theme… See the full description on the dataset page: https://huggingface.co/datasets/afmck/peanuts-flan-t5-xl.