kollessisopod/vqa-mixed

VQA Mixed (GQA + VizWiz + VQAv2 subset)

This dataset is a curated mixture of three major VQA benchmarks, subsampled and reformatted into Parquet shards for efficient model training (e.g., for VLMs like LLaVA or Qwen-VL).

Dataset Details

The dataset consists of image-question-answer triplets stored in Parquet format.

Data Splits

• —Train: train/data-*.parquet
• —Validation: validation/data-*.parquet

Columns

• —image: The visual input (Image feature).
• —question: The natural language query.
• —answer: The target response.
• —source: The origin dataset (to track performance per-domain).

Credits & Sources

This repository utilizes data from the following sources. If you use this mixed dataset, please cite the original authors.

1. VQAv2 (Subset)

• —Hugging Face Source: lmms-lab/VQAv2
• —Original Site: visualqa.org
• —Method: We selected 15% of the original training shards and reserved 5% for validation.
• —Hugging Face Source Used For Mixing: Multimodal-Fatima/VQAv2_train

2. GQA (Subsampled)

• —Hugging Face Source: lmms-lab/GQA
• —Original Site: GQA: Visual Reasoning
• —Method: We used 10% of the train_balanced split and 5% of the val_balanced split.
• —Hugging Face Source Used For Mixing: Mineru/GQA

3. VizWiz VQA (Full)

• —Hugging Face Source: lmms-lab/VizWiz-VQA
• —Original Site: vizwiz.org
• —Method: The dataset is included in full to emphasize accessibility-focused visual question answering.
• —Hugging Face Source Used For Mixing: ebrukilic/vizwiz_vqa_dataset

Reproduction Script

The following Python snippet demonstrates how the subsets were selected and processed using the datasets and huggingface_hub libraries.

from huggingface_hub import HfApi, hf_hub_download
from datasets import load_dataset, Image as HFImage

# --- VQAv2 Subsampling Logic ---
VQAV2_SHARD_FRACTION = 0.15
repo = "Multimodal-Fatima/VQAv2_train"
api = HfApi()

files = api.list_repo_files(repo, repo_type="dataset")
all_train_parquets = sorted([f for f in files if f.startswith("data/train-") and f.endswith(".parquet")])
num_shards = max(1, int(len(all_train_parquets) * VQAV2_SHARD_FRACTION))
train_parquets = all_train_parquets[:num_shards]

local_parquets = [hf_hub_download(repo_id=repo, filename=f, repo_type="dataset") for f in train_parquets]
vqav2_small = load_dataset("parquet", data_files=local_parquets, split="train")

# --- GQA & VizWiz Loading ---
gqa_train = load_dataset("Mineru/GQA", split="train_balanced")
vizwiz_train = load_dataset("ebrukilic/vizwiz_vqa_dataset", split="train")

# Cast to prevent auto-decoding during initial merge
gqa_train = gqa_train.cast_column("image", HFImage(decode=False))
vizwiz_train = vizwiz_train.cast_column("image", HFImage(decode=False))

Citation

If you would like to cite this dataset, you may use the following citation:

@misc{vqa-mixed,
  author = {kollessisopod},
  title = {VQA-Mixed Dataset},
  year = {2024},
  publisher = {Hugging Face},
  journal = {Hugging Face Repository},
  howpublished = {\url{https://huggingface.co/datasets/kollessisopod/vqa-mixed}},
}

Karışık var mı? Var. Karışık koy ya.