Datasetpublic

multilingual-vlm-conflict/rpg-conflict

RPG Fantasy Battle Conflict Dataset A dataset of 100 visual RPG combat conflict samples designed to evaluate Vision-Language Models (VLMs) under cross-modal conflicts (discrepancy between battle screenshots and caption text). Derived from the rcannizzaro/rpg_fantasy_battle_counterfactual_v2 dataset. Dataset Statistics This dataset consists of a single train split containing 100 perfectly isolated conflict samples derived from the… See the full description on the dataset page: https://huggingface.co/datasets/multilingual-vlm-conflict/rpg-conflict.

sourceHugging Faceupdated 3mo agoView on Hugging Face

0likes131downloads

Dataset Card

RPG Fantasy Battle Conflict Dataset

A dataset of 100 visual RPG combat conflict samples designed to evaluate Vision-Language Models (VLMs) under cross-modal conflicts (discrepancy between battle screenshots and caption text). Derived from the rcannizzaro/rpg_fantasy_battle_counterfactual_v2 dataset.

Dataset Statistics

This dataset consists of a single train split containing 100 perfectly isolated conflict samples derived from the rcannizzaro/rpg_fantasy_battle_counterfactual_v2 dataset.

The samples are evenly distributed across 5 categories to prevent evaluation bias:

—background: 20 samples
—player_1_type: 20 samples
—player_1_element: 20 samples
—player_2_type: 20 samples
—player_2_element: 20 samples

Dataset Schema

The dataset is structured as an ImageFolder with the following schema:

Column Name	Type	Description
`image`	`Image`	The base battle image. Automatically loaded by Hugging Face from `file_name`.
`file_name`	`string`	The filename of the image in the `train/` directory (e.g. `rpg_conflict_1.png`).
`original_caption`	`string`	The truthful templated caption that accurately describes the image.
`conflicting_caption`	`string`	The counterfactual caption containing a single hallucinated attribute.
`question`	`string`	A neutral question evaluating the specific attribute in conflict.
`image_bias`	`string`	The truthful answer matching the image.
`text_bias`	`string`	The incorrect answer matching the hallucinated text.
`distractor`	`string`	A plausible but incorrect third option.
`serial_no`	`int64`	The row ID.
`conflict_type`	`string`	The category of conflict (e.g., `background`, `player_1_type`).
`language`	`string`	Language of the text (English).

Reproduction

This dataset is a highly filtered subset derived from the original [rcannizzaro/rpg_fantasy_battle_counterfactual_v2](https://huggingface.co/datasets/rcannizzaro/rpg_fantasy_battle_counterfactual_v2) dataset.

To reproduce this dataset from scratch, you can use the build_rpg_dataset.py script included in this repository.

1.Ensure you have the datasets and pillow libraries installed.
2.Run the generation script:

bash

python build_rpg_dataset.py

This script will download the original 2000-image dataset, search for rows containing exactly 1 changed intervention between the base caption and counterfactual caption, and extract 20 perfect samples per category into a local directory with a generated metadata.csv.

[!NOTE] In the source train/metadata.csv file, the column is named file_name and contains the relative path to the image file (e.g. rpg_conflict_1.png). The Hugging Face ImageFolder builder automatically parses the file_name column to load the image objects and exposes them under the image column, discarding the raw file name string.