datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
LLaVA-Instruct-150K
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset date:
LLaVA Visual Instruct 150K was collected in April 2023, by prompting GPT-4-0314 API.
Paper or resources for more information:
https://llava-vl.github.io/
License:… See the full description on the dataset page: https://huggingface.co/datasets/liuhaotian/LLaVA-Instruct-150K.LLaVA-NeXT-Interleave-Bench
LLaVA-Interleave Bench Dataset Card
Dataset details
Dataset type:
LLaVA-Interleave Bench is a comprehensive set of multi-image datasets that are collected from public datasets or generated by the GPT-4V API.
It is constructed for evaluating the interleaved multi-image reaoning capbilities of LMMs.
Dataset date:
LLaVA-Interleave Bench was collected in April 2024, and released in June 2024.
Paper or resources for more information:
Blog:… See the full description on the dataset page: https://huggingface.co/datasets/lmms-lab-encoder/LLaVA-NeXT-Interleave-Bench.LLAVA-LibMoE
Download Instructions
This repository provides the full LLAVA-LibMoE dataset, including LLaVA-665K and OneVision-1M2 image sources, organized into the following required directory tree:
libmoe/
└── data/
├── image_onevision/
├── coco/
│ └── train2017/
├── gqa/
│ └── images/
├── ocr_vqa/
│ └── images/
├── textvqa/
│ └── train_images/
└── vg/
├── VG_100K/
└──… See the full description on the dataset page: https://huggingface.co/datasets/DavidNguyen/LLAVA-LibMoE.MMScan-llava-form
MMScan LLaVA-Form Data
This repository provides the processed LLaVA-formatted dataset for the MMScan Question Answering Benchmark.
Dataset Contents
(1) All image data(Depth&RGB) is distributed in split ZIP archives. Please combine the split ZIP files into a single archive and extract the merged ZIP file using the following command:
cat mmscan_val8.z* > mmscan_va.zip
unzip mmscan_va.zip
(2) Under ./annotations, we provide the MMScan Question Answering validation set with… See the full description on the dataset page: https://huggingface.co/datasets/rbler/MMScan-llava-form.llava-1.5-665k-instructionsThis dataset repository, LLaVA-1.5-665K-Instructions, is notably utilized in the paper Zero-Shot Vision Encoder Grafting via LLM Surrogates.
The official code repository for the paper can be found here: https://github.com/kaiyuyue/zero
LLaVA-1.5-665K-Instructions
This dataset repo contains the entire LLaVA-1.5-665K-Instructions in one place, including images and text sequences.
The images are in train_split/*.tars and the text sequences are in jsons:
llava_v1_5_mix665k.json is the… See the full description on the dataset page: https://huggingface.co/datasets/kaiyuyue/llava-1.5-665k-instructions.LLaVA-ReCap-676KThis is an integrated version of LLaVA-ReCap, sourced from lmms-lab/LLaVA-ReCap-558K and lmms-lab/LLaVA-ReCap-118K.
In this version, the conversations field has been split into two separate fields: prompt and response. Additionally, the <image> special token has been removed to facilitate customization.
Inspired by the original paper, the prompt field has been further expanded with human-crafted variations. Specifically, each prompt is sampled from one of the following 30 instructions:… See the full description on the dataset page: https://huggingface.co/datasets/LimeryJorge/LLaVA-ReCap-676K.vqa-nle-llavaVQA NLE synthetic dataset, made with LLaVA-1.5 using features from GQA dataset.
Total number of unique datas: 66684
Languages
eng
Supported Tasks
Question Answering
Dataset Usage
from datasets import load_dataset
dset = datasets.load_dataset("patrickamadeus/vqa-nle-llava", name='<CONFIG_NAME>', trust_remote_code=True)
Dataset Version
Source: 1.0.1. Date: 2024.09.25.
Dataset License
CC-BY 4.0
Citation
If you are using the VQA… See the full description on the dataset page: https://huggingface.co/datasets/patrickamadeus/vqa-nle-llava.Safe-LLaVA
🌟 Safe-LLaVA: A Privacy-Preserving Vision-Language Dataset
Safe-LLaVA is a privacy-enhanced version of the original LLaVA dataset, developed to systematically remove sensitive biometric attributes such as gender, race, age, eye color, and body weight.
This dataset is designed for privacy-safe pretraining, instruction tuning, and benchmarking Vision-Language Models (VLMs) under biometric privacy constraints.
📑 Dataset Summary
Name: Safe-LLaVA
Source: Derived from… See the full description on the dataset page: https://huggingface.co/datasets/kyh9191/Safe-LLaVA.Turkish-medical-visual-question-answering-LLaVa-dataset
Türkçe Radyoloji Görüntüleme Veri Seti - data_RAD
data_RAD veri seti, radyoloji görüntüleri üzerinde görsel soru-cevaplama (VQA) araştırmaları yapmak amacıyla Türkçeye çevrilmiş ve LLaVa mimarisiyle uyumlu hale getirilmiştir. Bu veri seti, tıbbi görüntü analizi ve yapay zeka destekli radyoloji uygulamalarını geliştirmek için kullanılabilir.
Veri Seti İçeriği
Toplam Görüntü Sayısı: 316
Veri Yapısı: DatasetDict({ train: Dataset({ features: ['image'], num_rows: 316 }) })
Özellikler:… See the full description on the dataset page: https://huggingface.co/datasets/nezahatkorkmaz/Turkish-medical-visual-question-answering-LLaVa-dataset.LLaVA-v1.5-Instruct-620K-JA
Dataset Details
Dataset Type:Japanese LLaVA v1.5 Instruct 620K is a localized version of part of the original LLaVA v1.5 Visual Instruct 655K dataset. This version is translated into Japanese using DeepL API and is aimed at serving similar purposes in the context of Japanese language.
Resources for More Information:For information on the original dataset: LLaVA
License:Attribution-NonCommercial 4.0 International (CC BY-NC-4.0)The dataset should abide by the policy of OpenAI: OpenAI… See the full description on the dataset page: https://huggingface.co/datasets/turing-motors/LLaVA-v1.5-Instruct-620K-JA.LLaVA-Instruct-150Khow I filtered llava_v1_5_mix665k.json into llava_v1_5_mix665k_vqa_clean_st_single_shuf.jsonl
https://colab.research.google.com/drive/1um_wlBFacXCHSabmJCg9Ll3DvP5rGxuE?usp=sharing
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.… See the full description on the dataset page: https://huggingface.co/datasets/HayatoHongo/LLaVA-Instruct-150K.LLaVA-Instruct-150K
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset date:
LLaVA Visual Instruct 150K was collected in April 2023, by prompting GPT-4-0314 API.
Paper or resources for more information:
https://llava-vl.github.io/
License:
Creative… See the full description on the dataset page: https://huggingface.co/datasets/sanaka87/LLaVA-Instruct-150K.Vietnamese-liuhaotian-llava_v1_5_mix665k-gg-translatedQUILT-LLaVA-Instruct-107KQUILT-LLaVA Visual Instruct 107K Dataset Card
Paper: Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videos
Paper or resources for more information:
https://quilt-llava.github.io/
Description and Details
YouTube educational histopathology videos are a valuable source of grounded histopathology data for instructional purposes, particularly for visual instruction tuning.
Similar to LLaVA, the approach involves using independent… See the full description on the dataset page: https://huggingface.co/datasets/wisdomik/QUILT-LLaVA-Instruct-107K.LLaVA-Pretrain-JA
Dataset Details
Dataset Type:Japanese LLaVA Pretrain is a localized version of the original LLaVA Pretrain dataset. This version is translated into Japanese using DeepL API and is aimed at serving similar purposes in the context of Japanese language.
Resources for More Information:For information on the original dataset: LLaVA
License:License: Must comply with license of CC-3M, BLIP (if you use their synthetic caption).
CC-3M The dataset may be freely used for any purpose, although… See the full description on the dataset page: https://huggingface.co/datasets/turing-motors/LLaVA-Pretrain-JA.LLaVA-Instruct-150Khow I filtered llava_v1_5_mix665k.json into llava_v1_5_mix665k_vqa_clean_st_single_shuf.jsonl
https://colab.research.google.com/drive/1um_wlBFacXCHSabmJCg9Ll3DvP5rGxuE?usp=sharing
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/gtz1/LLaVA-Instruct-150K.Open-LLaVA-NeXT-mix1M
Open-LLaVA-NeXT 1M Dataset Card
Dataset details
Dataset type: 1M SFT data for re-producing LLaVA-NeXT series.
We augmented the sharegpt4v_mix665k dataset with additional data. We have made every effort to align our training data with that of LLaVA-NeXT. However, we were unable to access the tens of thousands of real user interaction data that LLaVA-NeXT collected. As a result, we used 200K ALLaVA-Instruct-VFLAN-4V data as a substitute. Additionally, since TextVQA has been… See the full description on the dataset page: https://huggingface.co/datasets/Lin-Chen/Open-LLaVA-NeXT-mix1M.ViP-LLaVA-Instruct
ViP-LLaVA Instruct Dataset Card
Dataset details
Dataset type:
ViP-LLaVA Instruct is composed of a mixture of LLaVA-1.5 instruction data and the region-level visual prompting data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 level regional understanding capability.
Specifically, we use 1.2M data for stage 2 finetuning, and use 26K data for the optional stage 3 finetuning.
Dataset date:
ViP-LLaVA Instruct was collected… See the full description on the dataset page: https://huggingface.co/datasets/mucai/ViP-LLaVA-Instruct.llava-plus-data
LLaVA-Plus Instructed Dataset Card
Dataset details
Dataset type:
LLaVA-Plus-v1-117K is a set of GPT-generated multimodal tool-augmented instruction-following data.
It is constructed for tool use to build large multimodal agents with GPT-4-plus vision/language capability.
Dataset date:
LLaVA-Plus-v1-117K was collected in Sep 2023, by prompting ChatGPT/GPT-4-0314 API.
Paper or resources for more information:
https://llava-vl.github.io/llava-plus
License:… See the full description on the dataset page: https://huggingface.co/datasets/LLaVA-VL/llava-plus-data.multimodel_llava_med_zh_instruct_60kBorrowed from https://huggingface.co/datasets/BUAADreamer/llava-med-zh-instruct-60k
Fix the <image> placeholder issue, which will cause error during training:
raise ValueError(f"The number of images does not match the number of {IMAGE_PLACEHOLDER} tokens.")
Vietnamese-LLaVA-Instruct-150K-gg-translatedQuilt-LLaVA-PretrainQUILT-LLaVA Pretrain Dataset Card
Paper: Quilt-LLaVA: Visual Instruction Tuning by Extracting Localized Narratives from Open-Source Histopathology Videos
Paper or resources for more information:
https://quilt-llava.github.io/
Description and Details
The pretraining subset from [QUILT-1M]https://quilt1m.github.io/) for stage 1 of Quilt-LLaVA pretraining. For the accompanying images please use the following link to request images (GDrive)
Dataset date:
QUILT-LLaVA Pretrain was collected… See the full description on the dataset page: https://huggingface.co/datasets/wisdomik/Quilt-LLaVA-Pretrain.LLaVA-Instruct-150K-RU
LLaVA Visual Instruct 150K Dataset Card
https://huggingface.co/datasets/liuhaotian/LLaVA-Instruct-150K
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset date:
LLaVA Visual Instruct 150K was collected in April 2023, by prompting GPT-4-0314 API.
Paper or resources for more… See the full description on the dataset page: https://huggingface.co/datasets/mizinovmv/LLaVA-Instruct-150K-RU.Vietnamese-BAAI-SVIT-llava-v1.5-format-gg-translatedLLaVA-Spatial-Instruct-850K
LLaVA Spatial Instruct 850K Dataset Card
Dataset type:
LLaVA Spatial Instruct 850K is a combined set of LLaVA-v1.5 instruction tuning mixture dataset (llava_v1_5_mix665k.json), common benchmark training datasets, including Clevr, Textcaps, Visualmrc, VQAv2 fetched from the_cauldron, and spatial relation dataset including OpenSpaces and SpatialQA dataset created from the data pipeline of SpatialRGPT on OpenImages dataset.
Dataset proportion
LLaVA-v1.5 instruction tuning mixture… See the full description on the dataset page: https://huggingface.co/datasets/rogerxi/LLaVA-Spatial-Instruct-850K.LLaVA-Instruct-150K
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset date:
LLaVA Visual Instruct 150K was collected in April 2023, by prompting GPT-4-0314 API.
Paper or resources for more information:
https://llava-vl.github.io/
License:
Creative… See the full description on the dataset page: https://huggingface.co/datasets/dekjcn/LLaVA-Instruct-150K.LLaVA-Instruct-150K
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset date:
LLaVA Visual Instruct 150K was collected in April 2023, by prompting GPT-4-0314 API.
Paper or resources for more information:
https://llava-vl.github.io/
License:
Creative… See the full description on the dataset page: https://huggingface.co/datasets/zhtr6/LLaVA-Instruct-150K.LLaVA-Instruct-150K
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset date:
LLaVA Visual Instruct 150K was collected in April 2023, by prompting GPT-4-0314 API.
Paper or resources for more information:
https://llava-vl.github.io/
License:
Creative… See the full description on the dataset page: https://huggingface.co/datasets/akahana/LLaVA-Instruct-150K.LLaVA-Instruct-150K
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset date:
LLaVA Visual Instruct 150K was collected in April 2023, by prompting GPT-4-0314 API.
Paper or resources for more information:
https://llava-vl.github.io/
License:… See the full description on the dataset page: https://huggingface.co/datasets/sunsonzh/LLaVA-Instruct-150K.LLaVA-Instruct-150K
LLaVA Visual Instruct 150K Dataset Card
Dataset details
Dataset type:
LLaVA Visual Instruct 150K is a set of GPT-generated multimodal instruction-following data.
It is constructed for visual instruction tuning and for building large multimodal towards GPT-4 vision/language capability.
Dataset date:
LLaVA Visual Instruct 150K was collected in April 2023, by prompting GPT-4-0314 API.
Paper or resources for more information:
https://llava-vl.github.io/
License:… See the full description on the dataset page: https://huggingface.co/datasets/Sky-yijie/LLaVA-Instruct-150K.