neurips2026-ed443/audit-release

Structural T2I Verifier Protocol Audit Dataset

This artifact contains the generated images, human annotations, prompt metadata, verifier outputs, and detector artifacts needed by the companion code repository. Mount or copy this directory as ./dataset when running the code.

The dataset is intended for research and audit of structural text-to-image evaluation protocols. It should be used with the companion code artifact or with independent analysis code that preserves the same image-level evaluation unit.

Layout

• —release/: generated image corpora and per-image Human-1 annotation JSON files.
• —annotations/human1_annotations.csv: canonical Human-1 labels for the full image release.
• —annotations/human2_annotations.csv: canonical Human-2 labels for the double-audited subset, with human2_audit_subset marking broad-audit and traffic-test membership.
• —prompts/: T2I prompt templates by generator task, prompt family, and target count.
• —results/open_mllm_matrix/: open-verifier raw predictions and scored evaluation tables.
• —results/gpt55_hard_slice/: targeted GPT-5.5 sanity-check outputs.
• —results/claude_opus47_hard_slice/: targeted Claude Opus 4.7 sanity-check outputs.
• —results/tifa_like_qwen27b/: target-query/TIFA-like outputs.
• —results/ranking_flip/: generator-ranking flip tables.
• —results/detector_baselines/: detector predictions, threshold selections, and scored tables.
• —detector/yolo/: YOLO support labels, manifests, and train/validation lists.
• —croissant.json: Croissant metadata with NeurIPS E&D Responsible-AI fields.

The image release contains 6,240 generated images: 3,120 from Qwen-Image and 3,120 from FLUX.2-dev.

License

Human annotations authored for this artifact are released under the Creative Commons Attribution-NonCommercial 4.0 International license (CC BY-NC 4.0). See LICENSE.

Dataset documentation, prompt metadata, detector support metadata, manifests, and analysis tables are provided to support non-commercial research reuse of the annotation and evaluation artifact.

AI-generated images and verifier/model outputs are governed by the licenses and terms of their respective generator, verifier, model, and service providers. They are redistributed for research and audit use subject to those terms. Users are responsible for reviewing and complying with the applicable upstream terms before reuse or redistribution.

Citation

If you use this artifact, cite the companion paper and include attribution for this dataset artifact. Also preserve attribution to the upstream generator and verifier providers where required by their terms.