weikaih/TaskMeAnything-v1-imageqa-random

Dataset Card for TaskMeAnything-v1-imageqa-random

<h2 align="center"> TaskMeAnything-v1-imageqa-random dataset</h2>

<h2 align="center"> <a href="https://www.task-me-anything.org/">🌐 Website</a> | <a href="https://arxiv.org/abs/2406.11775">📑 Paper</a> | <a href="https://huggingface.co/collections/jieyuz2/taskmeanything-664ebf028ab2524c0380526a">🤗 Huggingface</a> | <a href="https://huggingface.co/spaces/zixianma/TaskMeAnything-UI">💻 Interface</a></h2>

<h5 align="center"> If you like our project, please give us a star ⭐ on GitHub for latest update. </h2>

TaskMeAnything-v1-Random

TaskMeAnything-v1-imageqa-random is a dataset which using randomly sampled questions from TaskMeAnything-v1, including 5,700 ImageQA questions. The dataset contains 19 splits, while each splits contains 300 questions from a specific task generator in TaskMeAnything-v1. For each row of dataset, it includes: image, question, options, answer and its corresponding task plan.

Load TaskMeAnything-v1-Random ImageQA Dataset

import datasets

dataset_name = 'weikaih/TaskMeAnything-v1-imageqa-random'
dataset = datasets.load_dataset(dataset_name, split = TASK_GENERATOR_SPLIT)

where TASK_GENERATOR_SPLIT is one of the task generators, eg, random_2d_how_many.

Evaluation Results

Overall

Breakdown performance on each task types

Out-of-Scope Use

This dataset should not be used for training models.

Disclaimers

TaskMeAnything and its associated resources are provided for research and educational purposes only. The authors and contributors make no warranties regarding the accuracy or reliability of the data and software. Users are responsible for ensuring their use complies with applicable laws and regulations. The project is not liable for any damages or losses resulting from the use of these resources.

Contact

• —Jieyu Zhang: jieyuz2@cs.washington.edu

Citation

BibTeX:

@article{zhang2024task,
  title={Task Me Anything},
  author={Zhang, Jieyu and Huang, Weikai and Ma, Zixian and Michel, Oscar and He, Dong and Gupta, Tanmay and Ma, Wei-Chiu and Farhadi, Ali and Kembhavi, Aniruddha and Krishna, Ranjay},
  journal={arXiv preprint arXiv:2406.11775},
  year={2024}
}