Datasetpublic

rentianfei122/Gen-Arena

Gen-Arena Gen-Arena is a text-to-image evaluation benchmark for compositional prompt following and reference-conditioned generation. Each sample contains a natural-language generation prompt, optional reference images, named entities, and fine-grained constraints. Gen-Arena is introduced in the paper SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation as a human-annotated benchmark with entity- and constraint-level specifications.… See the full description on the dataset page: https://huggingface.co/datasets/rentianfei122/Gen-Arena.

sourceHugging Faceotherupdated 5mo agoView on Hugging Face

0likes20downloads

Dataset Card

Gen-Arena

Gen-Arena is a text-to-image evaluation benchmark for compositional prompt following and reference-conditioned generation. Each sample contains a natural-language generation prompt, optional reference images, named entities, and fine-grained constraints.

Gen-Arena is introduced in the paper SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation as a human-annotated benchmark with entity- and constraint-level specifications.

This package contains 300 samples across 6 categories:

Category	Samples	Unique image files	Reference slots
`1-cartoon`	50	56	56
`2-game`	50	103	104
`3-sports`	50	50	50
`4-entertainment`	50	50	50
`5-competition`	50	0	0
`6-ceremony`	50	30	50

Files

—data/gen_arena.jsonl: unified metadata file with all 300 samples.
—reference_images/<category>/: reference images grouped by category.

All image paths in data/gen_arena.jsonl are root-relative paths.

Schema

Each row in data/gen_arena.jsonl has the following fields:

—id: unique sample id.
—category: category directory name.
—prompt: generation prompt.
—reference_images: list of objects with id and root-relative path.
—reference_image_paths: list of root-relative image paths.
—num_reference_images: number of reference images for the sample.
—entities: list of named entities and optional reference links.
—constraints: list of fine-grained constraints with id, type, text, and depends_on.
—num_entities: entity count.
—num_constraints: constraint count.
—constraint_type_counts: counts by constraint type.

Constraint types are attribute, layout, and relation.

Loading

python

from datasets import load_dataset

dataset = load_dataset("json", data_files="data/gen_arena.jsonl", split="train")
print(dataset[0]["prompt"])
print(dataset[0]["reference_image_paths"])

Reference images can be loaded by joining each reference_image_paths value with the dataset root. For example, image paths follow the form reference_images/1-cartoon/1-cartoon_001_Tiggy.jpg.

Notes

—5-competition is intentionally text-only and has no reference images.
—Some samples share the same reference image file after exact duplicate cleanup.
—The license and redistribution status of reference images should be verified before public release.

Citation

Citation information will be added after the associated paper is publicly available.