datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
flan-t5-large-embed-refinedwebAll of the data together is around 81.3GB. It's the last hidden states of 131,072 samples from refinedweb padded/truncated to 512 tokens on the left, fed through google/flan-t5-base.
Structure:
{
"encoding": List, shaped (512, 1024) aka (tokens, d_model),
"text": String, the original text that was encoded,
"attention_mask": List, binary mask to pass to your model with encoding to not attend to pad tokens
}
flan-t5-base-embed-refinedwebAll of the data together is around 61GB. It's the last hidden states of 131,072 samples from refinedweb padded/truncated to 512 tokens on the left, fed through google/flan-t5-base.
Structure:
{
"encoding": List, shaped (512, 768) aka (tokens, d_model),
"text": String, the original text that was encoded,
"attention_mask": List, binary mask to pass to your model with encoding to not attend to pad tokens
}
flan-t5-small-embed-refinedwebAll of the data together is around 41GB. It's the last hidden states of 131,072 samples from refinedweb padded/truncated to 512 tokens on the left, fed through google/flan-t5-small.
Structure:
{
"encoding": List, shaped (512, 512) aka (tokens, d_model),
"text": String, the original text that was encoded,
"attention_mask": List, binary mask to pass to your model with encoding to not attend to pad tokens
}
just a tip, you cannot load this with the RAM in the free ver of google colab, not… See the full description on the dataset page: https://huggingface.co/datasets/crumb/flan-t5-small-embed-refinedweb.flan-t5-boosting-mmlu_cotflan-t5-xl-embed-refinedwebpeanuts-flan-t5-xl
Peanut Comic Strip Dataset (Snoopy & Co.)
This is a dataset Peanuts comic strips from 1950/10/02 to 2000/02/13.
There are 77,456 panels extracted from 17,816 comic strips.
The dataset size is approximately 4.4G.
Each row in the dataset contains the following fields:
image: PIL.Image containing the extracted panel.
panel_name: unique identifier for the row.
characters: tuple[str, ...] of characters included in the comic strip the panel is part of.
themes: tuple[str, ...] of theme… See the full description on the dataset page: https://huggingface.co/datasets/afmck/peanuts-flan-t5-xl.Imagenet1k_validation_google_flan_t5_xxl_mode_T_SPECIFIC_A_ns_50000
Dataset Card for "Imagenet1k_validation_google_flan_t5_xxl_mode_T_SPECIFIC_A_ns_50000"
More Information needed
Imagenet1k_validation_google_flan_t5_xl_mode_T_SPECIFIC_A_ns_50000
Dataset Card for "Imagenet1k_validation_google_flan_t5_xl_mode_T_SPECIFIC_A_ns_50000"
More Information needed
flan-t5-boosting-bbh_cotVQAv2_sample_validation_google_flan_t5_xl_mode_A_D_PNP_GENERIC_C_Q_rices_ns_1000
Dataset Card for "VQAv2_sample_validation_google_flan_t5_xl_mode_A_D_PNP_GENERIC_C_Q_rices_ns_1000"
More Information needed
DTD_parition1_test_google_flan_t5_xxl_mode_A_ns_1880
Dataset Card for "DTD_parition1_test_google_flan_t5_xxl_mode_A_ns_1880"
More Information needed
Hatefulmemes_test_google_flan_t5_xxl_mode_OCR_rices_ns_1000
Dataset Card for "Hatefulmemes_test_google_flan_t5_xxl_mode_OCR_rices_ns_1000"
More Information needed
rts_flan-t5-small_sqlFood101_test_google_flan_t5_xxl_mode_T_SPECIFIC_A_ns_25250
Dataset Card for "Food101_test_google_flan_t5_xxl_mode_T_SPECIFIC_A_ns_25250"
More Information needed
Hatefulmemes_test_google_flan_t5_xxl_mode_A_OCR_rices_ns_1000
Dataset Card for "Hatefulmemes_test_google_flan_t5_xxl_mode_A_OCR_rices_ns_1000"
More Information needed
FGVC_Aircraft_test_google_flan_t5_xxl_mode_T_SPECIFIC_A_ns_3333
Dataset Card for "FGVC_Aircraft_test_google_flan_t5_xxl_mode_T_SPECIFIC_A_ns_3333"
More Information needed
Hatefulmemes_test_google_flan_t5_xxl_mode_T_A_OCR_rices_ns_1000
Dataset Card for "Hatefulmemes_test_google_flan_t5_xxl_mode_T_A_OCR_rices_ns_1000"
More Information needed
Food101_test_google_flan_t5_small_mode_T_SPECIFIC_A_ns_100
Dataset Card for "Food101_test_google_flan_t5_small_mode_T_SPECIFIC_A_ns_100"
More Information needed
CIFAR100_test_google_flan_t5_xl_mode_T_SPECIFIC_A_ns_10000
Dataset Card for "CIFAR100_test_google_flan_t5_xl_mode_T_SPECIFIC_A_ns_10000"
More Information needed
Hatefulmemes_test_google_flan_t5_xxl_mode_T_C_A_OCR_rices_ns_1000
Dataset Card for "Hatefulmemes_test_google_flan_t5_xxl_mode_T_C_A_OCR_rices_ns_1000"
More Information needed
DTD_parition1_test_google_flan_t5_xxl_mode_C_A_T_SPECIFIC_ns_1880
Dataset Card for "DTD_parition1_test_google_flan_t5_xxl_mode_C_A_T_SPECIFIC_ns_1880"
More Information needed
flan-t5-boosting-bbh_directVQAv2_sample_validation_google_flan_t5_xxl_mode_T_D_PNP_GENERIC_C_Q_rices_ns_1000
Dataset Card for "VQAv2_sample_validation_google_flan_t5_xxl_mode_T_D_PNP_GENERIC_C_Q_rices_ns_1000"
More Information needed
Hatefulmemes_test_google_flan_t5_xxl_mode_T_OCR_rices_ns_1000
Dataset Card for "Hatefulmemes_test_google_flan_t5_xxl_mode_T_OCR_rices_ns_1000"
More Information needed
FGVC_Aircraft_test_google_flan_t5_xxl_mode_A_ns_3333
Dataset Card for "FGVC_Aircraft_test_google_flan_t5_xxl_mode_A_ns_3333"
More Information needed
Hatefulmemes_test_google_flan_t5_xxl_mode_C_OCR_rices_ns_1000
Dataset Card for "Hatefulmemes_test_google_flan_t5_xxl_mode_C_OCR_rices_ns_1000"
More Information needed
VQAv2_sample_validation_google_flan_t5_xl_mode_Q_rices_ns_1000
Dataset Card for "VQAv2_sample_validation_google_flan_t5_xl_mode_Q_rices_ns_1000"
More Information needed
VQAv2_sample_validation_google_flan_t5_xl_mode_C_Q_rices_ns_1000
Dataset Card for "VQAv2_sample_validation_google_flan_t5_xl_mode_C_Q_rices_ns_1000"
More Information needed
Hatefulmemes_test_google_flan_t5_xxl_mode_T_C_A_rices_ns_1000
Dataset Card for "Hatefulmemes_test_google_flan_t5_xxl_mode_T_C_A_rices_ns_1000"
More Information needed
xsum-watermarked-flan-t5-small-20240828230657