datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
newyorker_caption_contest
Dataset Card for New Yorker Caption Contest Benchmarks
Dataset Summary
See capcon.dev for more!
Data from:
Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest
@inproceedings{hessel2023androids,
title={Do Androids Laugh at Electric Sheep? {Humor} ``Understanding''
Benchmarks from {The New Yorker Caption Contest}},
author={Hessel, Jack and Marasovi{\'c}, Ana and Hwang, Jena D. and Lee, Lillian
and… See the full description on the dataset page: https://huggingface.co/datasets/jmhessel/newyorker_caption_contest.newyorker_caption_ranking
New Yorker Caption Ranking Dataset
Dataset Descriptions
Homepage: https://nextml.github.io/caption-contest-data/
Repository: https://github.com/yguooo/cartoon-caption-generation
Paper: Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning
Point of Contact: yguo@cs.wisc.edu
Dataset Summary
We present a novel multimodal preference dataset for creative tasks, consisting of over 250 million human ratings on more than 2.2… See the full description on the dataset page: https://huggingface.co/datasets/yguooo/newyorker_caption_ranking.coco-captions-pt-br
🎉 COCO Captions Dataset Translation for Portuguese Image Captioning
💾 Dataset Summary
COCO Captions Portuguese Translation, a multimodal dataset for Portuguese image captioning with 123,287 images, each accompanied by five descriptive captions that have been
generated by human annotators for every individual image. The original English captions were rendered into Portuguese
through the utilization of the Google Translator API.
🧑💻 Hot to Get… See the full description on the dataset page: https://huggingface.co/datasets/laicsiifes/coco-captions-pt-br.danbooru-2408-blind-captions
Danbooru 2408 Blind Captions
from datasets import load_dataset
ds = load_dataset(
"dartags/danbooru-2408-blind-captions",
split="train",
)
human-templated-captions-1bcsv delimiter is = ".,|,."
apparently python doesn't like multichar delimiters using the native csv so there's some issues with environments when loading.
This seemed like a good idea to avoid overlapping potential characters, but in practice it turned into additional overhead and bugs. I'll be manually converting the split to parquet and providing a proper file split soon.
Additionally with the parquet will introduce the large caption split; which are considerably longer captions for the… See the full description on the dataset page: https://huggingface.co/datasets/AbstractPhil/human-templated-captions-1b.youtube_caption_corrections
Dataset Card for YouTube Caption Corrections
Dataset Summary
This dataset is built from pairs of YouTube captions where both an auto-generated and a manually-corrected caption are available for a single specified language. It currently only in English, but scripts at repo support other languages. The motivation for creating it was from viewing errors in auto-generated captions at a recent virtual conference, with the hope that there could be some way to help correct those… See the full description on the dataset page: https://huggingface.co/datasets/community-datasets/youtube_caption_corrections.astrobridge-image-captions
AstroBridge Legacy Survey Captions
3,487 imaging cutouts from the Legacy Survey (DR10 South + North), crossmatched against
published literature mentions and captioned in four independent stages by Gemini
(gemini-3.7-flash), following the AstroLLaVA data-generation approach (Zaman et al. 2025,
arXiv:2504.08583): no caption is ever told the object's
real name or catalog designation, and no caption states a fact that isn't derivable from the
pixels or the (redacted-at-the-model… See the full description on the dataset page: https://huggingface.co/datasets/gapatron/astrobridge-image-captions.danbooru2023-florence2-caption
Danbooru2023 - Florence2 Caption dataset
This dataset contains captions of danbooru2023 images generated by microsoft/Florence-2-large
I use original one with task token
Format
parquet:
key: the danbooru id of the image
parsed: parsed florence 2 output of the image
Stat
MORE_DETAILED_CAPTION
Entries: 7,438,449
Output Tokens (Min/Max/Mean/Median):
Flan T5 Tokenizer: 19/736/120/114
DFN CLIP Tokenizer: 19/826/108.7/103
Qwen2 Tokenizer:… See the full description on the dataset page: https://huggingface.co/datasets/KBlueLeaf/danbooru2023-florence2-caption.pokemon-gpt4o-captionsBorrowed from: https://huggingface.co/datasets/jugg1024/pokemon-gpt4o-captions
You can use it in LLaMA Factory by specifying dataset: pokemon_cap.
random-captions-10mRandomly generated captions using tokenization templates and lists.
.,|,. is the caption delimiter, so split accordingly.
jamendo-fma-captionsfrom datasets import load_dataset
captions = load_dataset("disco-eth/jamendo-fma-captions")
What is Jamendo-FMA-captions?
This dataset contains synthetically generated captions for the MTG-Jamendo and the FMA dataset.
We used SALMONN to caption both datasets and then improved the generated captions using ChatGPT. For more information or to cite our paper, see Coarse-to-Fine Text-to-Music Latent Diffusion.
Features
id: The sample id that can be used to match the caption… See the full description on the dataset page: https://huggingface.co/datasets/disco-eth/jamendo-fma-captions.leetcode_with_youtube_captionsWATER-Z_Captions
WATER-Z Captions: Prompts for Artistic-Text Image Generation
WATER-Z Captions is the prompt/caption resource used to build the WATER-Z subset of
WATER-S in the paper "Advancing WordArt-Oriented Scene Text Recognition: Datasets and
Methods" (ECCV 2026).
It contains 273,488 high-quality, fine-grained text prompts tailored for generating artistic
(WordArt) text images. Each prompt describes the visual style, texture, and layout of an artistic
text design and contains an editable… See the full description on the dataset page: https://huggingface.co/datasets/Yesianrohn/WATER-Z_Captions.Filtered-COCO-Captions
Dataset Summary
This dataset is derived from the MS COCO caption annotations.
Source
Original annotations: MS COCO / COCO Consortium
License
The original annotation set is licensed under CC BY 4.0.
This repository redistributes a filtered/adapted version of the annotation text only.
No original COCO images are included.
Modifications
Removed captions deemed unsuitable for TOEIC educational materials
Normalized punctuation and whitespace
Filtered for… See the full description on the dataset page: https://huggingface.co/datasets/kknono668/Filtered-COCO-Captions.coco-captions-pt-br
🎉 COCO Captions Dataset Translation for Portuguese Image Captioning
💾 Dataset Summary
COCO Captions Portuguese Translation, a multimodal dataset for Portuguese image captioning with 123,287 images, each accompanied by five descriptive captions that have been
generated by human annotators for every individual image. The original English captions were rendered into Portuguese
through the utilization of the Google Translator API.
🧑💻 Hot to Get… See the full description on the dataset page: https://huggingface.co/datasets/EliMC/coco-captions-pt-br.img_captioning_oilcanvas_styleflickr30k-captions_marathi
Flickr30K-Captions Marathi Dataset: High-Quality Marathi NLP Corpus
📌 Overview
The Flickr30K-Captions Marathi dataset is a meticulously curated collection of 158881 rows of Marathi text, ensuring linguistic accuracy and natural flow. Every sentence has been verified by native Marathi speakers to maintain contextual integrity and correctness.
This dataset is designed for semantic search, text classification, and various NLP tasks, making it a valuable resource for machine… See the full description on the dataset page: https://huggingface.co/datasets/Singhchandann/flickr30k-captions_marathi.coco-captions_marathi
Coco-Captions Marathi Dataset: High-Quality Marathi NLP Corpus
📌 Overview
The Coco-Captions Marathi dataset is a meticulously curated collection of 414010 rows of Marathi text, ensuring linguistic accuracy and natural flow. Every sentence has been verified by native Marathi speakers to maintain contextual integrity and correctness.
This dataset is designed for semantic search, text classification, and various NLP tasks, making it a valuable resource for machine learning… See the full description on the dataset page: https://huggingface.co/datasets/Singhchandann/coco-captions_marathi.JourneyBench_Captioningpicture_short_captionIt is used for training to generate short sentence copywriting according to image content, the source of the image dataset is https://unsplash.com/, and the source of short sentence copywriting is Claude3.7
用做图片内容生成短句文案训练，图片数据集来自 https://unsplash.com/，短句文案来自 Claude3.7 模型
wikiart_captions
WikiArt Captions Subset — Multimodal Art Retrieval Dataset
This dataset is a curated subset of 6,000 paintings from the WikiArt collection.It was created as part of a project on multimodal art retrieval, combining visual, textual, and semantic information.
Each record represents one artwork and includes:
Field
Description
image_row
Row index in the source subset (integer)
caption
Automatically generated textual description (caption) using the BLIP model… See the full description on the dataset page: https://huggingface.co/datasets/Lizagrin/wikiart_captions.