datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
newyorker_caption_contest
Dataset Card for New Yorker Caption Contest Benchmarks
Dataset Summary
See capcon.dev for more!
Data from:
Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest
@inproceedings{hessel2023androids,
title={Do Androids Laugh at Electric Sheep? {Humor} ``Understanding''
Benchmarks from {The New Yorker Caption Contest}},
author={Hessel, Jack and Marasovi{\'c}, Ana and Hwang, Jena D. and Lee, Lillian
and… See the full description on the dataset page: https://huggingface.co/datasets/jmhessel/newyorker_caption_contest.conceptual_captions
Dataset Card for Conceptual Captions
Dataset Summary
Conceptual Captions is a dataset consisting of ~3.3M images annotated with captions. In contrast with the curated style of other image caption annotations, Conceptual Caption images and their raw descriptions are harvested from the web, and therefore represent a wider variety of styles. More precisely, the raw descriptions are harvested from the Alt-text HTML attribute associated with web images. To arrive at the… See the full description on the dataset page: https://huggingface.co/datasets/google-research-datasets/conceptual_captions.pokemon-blip-captions
Notice of DMCA Takedown Action
We have received a DMCA takedown notice from The Pokémon Company International, Inc.
In response to this action, we have taken down the dataset.
We appreciate your understanding.
COCO-Caption
Large-scale Multi-modality Models Evaluation Suite
Accelerating the development of large-scale multi-modality models (LMMs) with lmms-eval
🏠 Homepage | 📚 Documentation | 🤗 Huggingface Datasets
This Dataset
This is a formatted version of COCO-Caption-2014-version. It is used in our lmms-eval pipeline to allow for one-click evaluations of large multi-modality models.
@misc{lin2015microsoft,
title={Microsoft COCO: Common Objects in Context}… See the full description on the dataset page: https://huggingface.co/datasets/lmms-lab-encoder/COCO-Caption.coco_captions
Dataset Card for "coco_captions"
More Information needed
COCO-Caption2017
Large-scale Multi-modality Models Evaluation Suite
Accelerating the development of large-scale multi-modality models (LMMs) with lmms-eval
🏠 Homepage | 📚 Documentation | 🤗 Huggingface Datasets
This Dataset
This is a formatted version of COCO-Caption-2017-version. It is used in our lmms-eval pipeline to allow for one-click evaluations of large multi-modality models.
@misc{lin2015microsoft,
title={Microsoft COCO: Common Objects in Context}… See the full description on the dataset page: https://huggingface.co/datasets/lmms-lab-encoder/COCO-Caption2017.ffhq512-captionCaptionQA
📌 CaptionQA Benchmark
A high-density, taxonomy-grounded benchmark for evaluating image caption quality and the alignment between image information and generated captions
📄 Paper: CaptionQA: Is Your Caption as Useful as the Image Itself? 📦 Evaluation Code: GitHub Repository
Sample Usage
You can load the dataset using the Hugging Face datasets library:
from datasets import load_dataset
# Load the entire dataset
dataset = load_dataset("Borise/CaptionQA")
# Load a… See the full description on the dataset page: https://huggingface.co/datasets/Borise/CaptionQA.Captioned_COCOStuffCOCO_captions_train
Dataset Card for "COCO_captions_train"
More Information needed
pokemon-gpt4-captions
Dataset Card for "pokemon-gpt4-captions"
This dataset is just lambdalabs/pokemon-blip-captions but the captions come from GPT-4 (Turbo).
Code used to generate the captions:
import base64
from io import BytesIO
import requests
from PIL import Image
def encode_image(image):
buffered = BytesIO()
image.save(buffered, format="JPEG")
img_str = base64.b64encode(buffered.getvalue())
returnimg_str.decode("utf-8")
def create_payload(image_string):
payload = {… See the full description on the dataset page: https://huggingface.co/datasets/diffusers/pokemon-gpt4-captions.movie-scenes-captionedSTAIR-Captions
Dataset Card for STAIR-Captions
Dataset Summary
STAIR Captions is a large-scale dataset containing 820,310 Japanese captions. This dataset can be used for caption generation, multimodal retrieval, and image generation.
Supported Tasks and Leaderboards
[More Information Needed]
Languages
The language data in JDocQA is in Japanese (BCP-47 ja-JP).
Dataset Structure
Data Instances
[More Information Needed]… See the full description on the dataset page: https://huggingface.co/datasets/shunk031/STAIR-Captions.220k-GPT4Vision-captions-from-LIVIS
220k-GPT4Vision-captions-from-LVIS
by: Christoph Schuhmann, Peter Bevan, 21 Nov, 2023
This dataset comprises 220,000 captioned images from the LVIS dataset. The captions were generated by summarising the LVIS-Instruct4V dataset released by X2FD. The instructions are converted into captions using Mistral-7B-OpenOrca.
PROMPT
"""<<SYS>> You are a highly intelligent, empathic, helpful, respectful, and honest assistant with high emotional intelligence. Always… See the full description on the dataset page: https://huggingface.co/datasets/laion/220k-GPT4Vision-captions-from-LIVIS.GBC10M
Graph-based captioning (GBC) is a new image annotation paradigm that combines the strengths of long captions, region captions, and scene graphs
GBC interconnects region captions to create a unified description akin to a long caption, while also providing structural information similar to scene graphs.
** The associated data point can be found at demo/water_tower.json
Description and data format
The GBC10M dataset, derived from the original images in CC12M, is… See the full description on the dataset page: https://huggingface.co/datasets/graph-based-captions/GBC10M.nordjylland-news-image-captioning
Dataset Card for "nordjylland-news-image-captioning"
Dataset Summary
This dataset is a collection of image-caption pairs from the Danish newspaper TV2 Nord.
Supported Tasks and Leaderboards
Image captioning is the intended task for this dataset. No leaderboard is active at this point.
Languages
The dataset is available in Danish (da).
Dataset Structure
An example from the dataset looks as follows.
{
"file_name": "1.jpg",
"caption":… See the full description on the dataset page: https://huggingface.co/datasets/alexandrainst/nordjylland-news-image-captioning.COCO_captions_validation
Dataset Card for "COCO_captions_validation"
More Information needed
CONCEPTUAL_CAPTIONS_HU_FILTEREDbokete_oogiri_captionwikiart_captionsdanbooru-multitier-captions-202606
Danbooru — multi-tier captions (202606)
Per-post Danbooru data for the 202606 crawl: native tags, the raw API metadata, model-generated
multi-tier natural-language captions (long / refined long / medium / short), and post flags.
One row per Danbooru post_id. Images are not included — each post is referenced by
post_id, danbooru_url, md5, and the Danbooru CDN URLs. (The two example previews below are
downscaled for illustration.)
Based on:… See the full description on the dataset page: https://huggingface.co/datasets/BootsofLagrangian/danbooru-multitier-captions-202606.anime-captionscoco-captions-pt-br
🎉 COCO Captions Dataset Translation for Portuguese Image Captioning
💾 Dataset Summary
COCO Captions Portuguese Translation, a multimodal dataset for Portuguese image captioning with 123,287 images, each accompanied by five descriptive captions that have been
generated by human annotators for every individual image. The original English captions were rendered into Portuguese
through the utilization of the Google Translator API.
🧑💻 Hot to Get… See the full description on the dataset page: https://huggingface.co/datasets/laicsiifes/coco-captions-pt-br.ffhq_with_llava_shorter_captions
Dataset Card for "ffhq_with_llava_shorter_captions"
More Information needed
IntraOral_Gingivitis_Image_Captioning
A DENTAL INTRAORAL IMAGE DATASET OF GINGIVITIS FOR IMAGE CAPTIONING
Dataset Description
This dataset is a copy of A Dental IntraOral Image Dataset of Gingivitis for Image Captioning which is shared with the license CC BY 4.0.
This dataset contains 1,096 samples organized across multiple splits.
The dataset includes image data.
Splits
train: 732 samples
test: 182 samples
validation: 182 samples
Dataset Creation
This dataset was created using… See the full description on the dataset page: https://huggingface.co/datasets/ekacare/IntraOral_Gingivitis_Image_Captioning.minecraft-skins-captioned-900k
🎮 minecraft-skins-captioned-900k
854,116 high-quality, captioned Minecraft player skins — deduplicated, Steve-model only, ready for text-to-image training.
📋 Dataset Summary
A rigorously filtered and quality-controlled version of neurlang/Minecraft-Skins-Captioned-1M specifically curated for training high-performance generative models that require precise UV topology constraints.
This dataset is optimized for models like ST-DiT (Sparse Template-Aware… See the full description on the dataset page: https://huggingface.co/datasets/summykai/minecraft-skins-captioned-900k.image-captioning-turkish
Türkçe Image Captioning Veri Seti
Bu veri seti BLIP3o modelinin pretrain eğitiminde kullanılan BLIP3o-Pretrain-Long-Caption ve BLIP3o-Pretrain-Short-Caption veri setlerinin Türkçeye çevirilmiş bir alt parçasıdır. Orijinal veri setinin oluşturulması ile ilgili detaylı bilgiye BLIP-3o makalesi üzerinden ulaşabilirsiniz.
Veri seti Image-to-Text modellerinin eğitilmesinde veya ince ayar sürecinde kullanılabilir. Veri seti, orijinal veri setinin lisansı olan Apache 2.0 altında… See the full description on the dataset page: https://huggingface.co/datasets/ituperceptron/image-captioning-turkish.llama4-maverick-coco-captionsimage_captions
From the Frontier Research Team at takara.ai we present over 1 million curated captioned images for multimodal text and image tasks.
Usage
from datasets import load_dataset
ds = load_dataset("takara-ai/image_captions")
print(ds)
Example
10,000 images from the dataset.
Methodology
We consolidated multiple open source datasets through an intensive 96-hour computational process across three nodes. This involved standardizing and validating the… See the full description on the dataset page: https://huggingface.co/datasets/takara-ai/image_captions.coco_captioning_complete_format