datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
GQA
Large-scale Multi-modality Models Evaluation Suite
Accelerating the development of large-scale multi-modality models (LMMs) with lmms-eval
🏠 Homepage | 📚 Documentation | 🤗 Huggingface Datasets
This Dataset
This is a formatted version of GQA. It is used in our lmms-eval pipeline to allow for one-click evaluations of large multi-modality models.
@inproceedings{hudson2019gqa,
title={Gqa: A new dataset for real-world visual reasoning and compositional… See the full description on the dataset page: https://huggingface.co/datasets/lmms-lab-encoder/GQA.GQA-Scene-Graph
Dataset Card for GQA-35k
The GQA (Visual Reasoning in the Real World) dataset is a large-scale visual question answering dataset that includes scene graph annotations for each image.
This is a FiftyOne dataset with 35000 samples.
Note: This is a 35,000 sample subset which does not contain questions, only the scene graph annotations as detection-level attributes.
You can find the recipe notebook for creating the dataset here
Installation
If you haven't already… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/GQA-Scene-Graph.NExT-GQA
Can I Trust Your Answer? Visually Grounded Video Question Answering
Introduction
We study visually grounded VideoQA by forcing vision-language models (VLMs) to answer questions and simultaneously ground the relevant video moments as visual evidences. We show that this task is easy for human yet is extremely challenging for existing VLMs, revealing that the strong QA performance of these models may largely due to short-cut learning (e.g., language priors and spurious vision-text… See the full description on the dataset page: https://huggingface.co/datasets/jinyoungkim/NExT-GQA.GQA
Large-scale Multi-modality Models Evaluation Suite
Accelerating the development of large-scale multi-modality models (LMMs) with lmms-eval
🏠 Homepage | 📚 Documentation | 🤗 Huggingface Datasets
This Dataset
This is a formatted version of GQA. It is used in our lmms-eval pipeline to allow for one-click evaluations of large multi-modality models.
@inproceedings{hudson2019gqa,
title={Gqa: A new dataset for real-world visual reasoning and compositional question… See the full description on the dataset page: https://huggingface.co/datasets/pppop7/GQA.gqagqa_tsvGQAGQA200-coco-format
GQA — General Question Answering (COCO format)
This dataset is the GQA200 split of
the GQA dataset
(Hudson et al., 2019), reformatted in standard COCO-JSON format.
GQA200 contains the top 200 object categories
and 100 relations from the original GQA dataset, selected by frequency in the
Stacked hybrid-attention and group collaborative learning for unbiased scene graph generation
paper. This dataset has no official test split since it was used
for question answering rather than… See the full description on the dataset page: https://huggingface.co/datasets/maelic/GQA200-coco-format.GQA_llavaGQA-imagesdetails_BEE-spoke-data__smol_llama-101M-GQA
Dataset Card for Evaluation run of BEE-spoke-data/smol_llama-101M-GQA
Dataset Summary
Dataset automatically created during the evaluation run of model BEE-spoke-data/smol_llama-101M-GQA on the Open LLM Leaderboard.
The dataset is composed of 64 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_BEE-spoke-data__smol_llama-101M-GQA.gqaGQA is a new dataset for real-world visual reasoning and compositional question answering,
seeking to address key shortcomings of previous visual question answering (VQA) datasets.GQA200
GQA200
GQA200 is the scene-graph-generation benchmark subset of
GQA. This repository
uses the standard GQA200 taxonomy with 200 foreground object classes
and 100 foreground predicate classes.
Splits
Split
Images
Source
train
57,623
Standard GQA200 Train
validation
8,209
Standard GQA200 Test
The standard GQA200 Test annotations are intentionally exposed as
validation. This repository does not define a separate test split.
Fields… See the full description on the dataset page: https://huggingface.co/datasets/wliafe/GQA200.gqa-testdev-balanced-lance
GQA testdev-balanced (Lance Format)
A Lance-formatted version of the canonical GQA testdev_balanced slice — 12,578 compositional VQA questions joined against the matching 398 images — sourced from lmms-lab/GQA. The original redistribution ships instructions and images as separate parquet configs; here they are pre-joined on image_id, so each row carries the question text, the short answer, the GQA reasoning-program tags, paired CLIP image and question embeddings, and the inline JPEG… See the full description on the dataset page: https://huggingface.co/datasets/lance-format/gqa-testdev-balanced-lance.sealvqa_gqaGQA_for_llava_chunksdetails_8Xqmff94__pooled_gqa_mix_chatmlGQAdetails_8xqmff94__pooled_gqa_rawdetails_8xqmff94__pooled_gqa_mathgqa-inpaint
GQA-Inpaint Dataset
GQA-Inpaint is a real image dataset to train and evaluate models for the instructional image inpainting task. Scene graphs of the GQA dataset are exploited to generate paired training data by utilizing state-of-the-art instance segmentation and inpainting methods. Dataset usage and content details are explained in the Inst-Inpaint GitHub repository.
GQA-ru
GQA-ru
This is a translated version of original GQA dataset and
stored in format supported for lmms-eval pipeline.
For this dataset, we:
Translate the original one with gpt-4-turbo
Filter out unsuccessful translations, i.e. where the model protection was triggered
Manually validate most common errors
Dataset Structure
Dataset includes both train and test splits translated from original train_balanced and testdev_balanced.
Train split includes 27519 images with… See the full description on the dataset page: https://huggingface.co/datasets/deepvk/GQA-ru.details_xformAI__opt-125m-gqa-ub-6-best-for-KV-cache
Dataset Card for Evaluation run of xformAI/opt-125m-gqa-ub-6-best-for-KV-cache
Dataset automatically created during the evaluation run of model xformAI/opt-125m-gqa-ub-6-best-for-KV-cache on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 2 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_xformAI__opt-125m-gqa-ub-6-best-for-KV-cache.details_BEE-spoke-data__smol_llama-220M-GQA
Dataset Card for Evaluation run of BEE-spoke-data/smol_llama-220M-GQA
Dataset automatically created during the evaluation run of model BEE-spoke-data/smol_llama-220M-GQA on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the timestamp of the run.The "train" split is always… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_BEE-spoke-data__smol_llama-220M-GQA.gqa_answerGQAgqa-tracesGQA
Large-scale Multi-modality Models Evaluation Suite
Accelerating the development of large-scale multi-modality models (LMMs) with lmms-eval
🏠 Homepage | 📚 Documentation | 🤗 Huggingface Datasets
This Dataset
This is a formatted version of GQA. It is used in our lmms-eval pipeline to allow for one-click evaluations of large multi-modality models.
@inproceedings{hudson2019gqa,
title={Gqa: A new dataset for real-world visual reasoning and compositional question… See the full description on the dataset page: https://huggingface.co/datasets/alexwww94/GQA.gqa-valdetails_saarvajanik__facebook-opt-6.7b-gqa-ub-16-best-for-KV-cache
Dataset Card for Evaluation run of saarvajanik/facebook-opt-6.7b-gqa-ub-16-best-for-KV-cache
Dataset automatically created during the evaluation run of model saarvajanik/facebook-opt-6.7b-gqa-ub-16-best-for-KV-cache on the Open LLM Leaderboard.
The dataset is composed of 63 configuration, each one coresponding to one of the evaluated task.
The dataset has been created from 1 run(s). Each run can be found as a specific split in each configuration, the split being named using the… See the full description on the dataset page: https://huggingface.co/datasets/open-llm-leaderboard-old/details_saarvajanik__facebook-opt-6.7b-gqa-ub-16-best-for-KV-cache.