Namronaldo2004/ViInfographicsVQA

Introduction

ViInfographicsVQA is a Vietnamese Visual Question Answering (VQA) dataset constructed from infographics sourced from 26 different news platforms. The dataset is designed to support research in multimodal learning by providing diverse questions and answers based on real-world visual data. The detailed distribution of sources is presented in the table below.

<div style="text-align: center;"> <img src="press.png" alt="Infographics Distribution" style="display: block; margin: auto;"> <div style="font-style: italic;">Figure 1: The number of infographics per news source.</div> </div>

• —Developed by: @Namronaldo2004, @Kiet2302, @Mels22, @JoeCao, @minhthien

• —Dataset Type: Visual Question Answering (VQA) on Vietnamese language.

• —Language: Vietnamese

• —License: Apache 2.0

Data pipeline

Regarding the dataset creation process, we strictly adhere to the following dataset construction workflow:

<div style="text-align: center;"> <img src="pipeline.png" alt="The pipeline of building ViInfographicsVQA dataset" style="display: block; margin: auto;"> <div style="font-style: italic;">Figure 2: The pipeline of building ViInfographicsVQA dataset.</div> </div>

<br>

QA Type Classification

To better analyze and experiment with the scene-text properties of our dataset, we classify each QA into either "Text QA" or "Non-text QA."

• —Text QA refers to questions based on numerical data, textual information, or any text present in the infographic. Questions involving information extracted from the text to answer other specific questions also fall into this category.

• —Non-text QA refers to questions about colors, chart shapes, positions on the map, and objects such as people, trees, animals, vehicles, etc., that do not require reading text to answer.

Rules and Constraints

To restrict the scope as well as the usability of the dataset, we defined rules and constraints before creating QA pairs in the dataset. Those ones are presented as follows:

• —Number of QAs: Generate about 5 QAs per image, including 3 Text QAs and 2 Non-text QAs.

• —QA Length: Questions and answers should not exceed 30 words.

• —Colors:

• —Only use these colors: black, white, red, orange, yellow, green, blue, sky blue, purple, pink, brown, gray. Ignore color if it's not applicable.

• —Only ask about the color of specific objects — not the background.

• —Question Constraints:

• —Avoid yes/no and choice-based questions.

• —Do not ask questions requiring deep analysis or inference beyond the infographic data.

• —Ensure sufficient data is available to answer the question.

• —Avoid questions that can be answered without referencing the infographic.

• —For numerical questions, include a comparison (e.g., greater than, within a range).

• —For counting-based questions, specify criteria like starting or ending with a specific letter, word or phrase.

• —Answer Constraints:

• —Answers should be complete sentences that directly address the question.

• —Include a clear explanation (within 100 words) detailing the reasoning (e.g., counted objects, location on the infographic). Write as a paragraph, not bullet points.

Data Structure

The dataset is structured as follows:

• —image: The input image.

• —question: A question related to the image.

• —answer: The correct answer to the question.

• —explanation: A justification for why the answer is correct.

• —type: The category of the question.

Sample code

To utilize our dataset, you can use the sample code below:

import matplotlib.pyplot as plt
from datasets import load_dataset

# Load dataset in streaming version
train_ds = load_dataset("Namronaldo2004/ViInfographicsVQA", split = "train", streaming = True)

# Get the first record
first_record = next(iter(train_ds))

# Plot the image
plt.imshow(first_record["image"])
plt.axis("off")
plt.title("First Image in Train Dataset")
plt.show()

# Print remaining attributes
print("❓ Question:", first_record["question"])
print("✅ Answer:", first_record["answer"])
print("💡 Explanation:", first_record["explanation"])
print("📌 Type:", first_record["type"])