AIML-TUDA/COCOLogic-v2

COCOLogic-V2

COCOLogic-V2 is an object-centric dataset for visual inductive reasoning on real-world images. Built on MSCOCO, it frames reasoning as a multilabel classification task over 10 compositional first-order-logic rules (object presence/absence, counting, and count comparisons). Samples of each rule are divided into different positive variants, as well as types of near-boundary (NB) negatives, and the typically easy far-from-boundary (FB) negatives. These annotations enable automated insights for model accountability: they reveal whether a model has actually learned a logical rule or is merely exploiting coarse statistical shortcuts.

This dataset contains no images

It contains annotations only. Download MSCOCO 2017 yourself from cocodataset.org and join on file_name:

• —the train splits reference COCO train2017 (file_name = train2017/000000253444.jpg)
• —the test splits reference COCO val2017 (file_name = val2017/...)

import os
from datasets import load_dataset
from PIL import Image

COCO_IMAGES = "/path/to/coco/images"   # contains train2017/ and val2017/

ds = load_dataset("AIML-TUDA/COCOLogic-v2", "full", split="train")
ex = ds[0]
image = Image.open(os.path.join(COCO_IMAGES, ex["file_name"])).convert("RGB")
print(ex["sample_type"])      # per-rule: positive / near_boundary / far_from_boundary
print(ex["object_names"], ex["object_counts"])

Configs and splits

In fewshot, rows outnumber images because an image can be relevant to more than one rule.

full    = load_dataset("AIML-TUDA/COCOLogic-v2", "full")
fewshot = load_dataset("AIML-TUDA/COCOLogic-v2", "fewshot")

The 10 rules

Rule r (1-based) corresponds to index r - 1 in every per-rule vector column (labels, sample_type, boundary_type, rule_variants).

Sample taxonomy: variants, NB, FB

Every sample is categorized per rule into one of three groups:

• —Positive variants — the distinct ways a rule can be satisfied. Each rule is converted to disjunctive normal form (DNF); each disjunct is one positive variant. The full positive set is the union of all variants. Variants are numbered from 1 (rule 8 has 4; an image may satisfy more than one).
• —Near-boundary (NB) negatives — hard negatives derived from a DNF disjunct by flipping a single literal (or, for counting rules, by changing the required object count). These sit just outside the rule and are the key probe of true rule understanding. NB types are numbered from 1 (up to 6).
• —Far-from-boundary (FB) negatives — easy negatives drawn from the remaining MSCOCO images, kept so the overall distribution stays close to MSCOCO's.

Worked example — "Double Serving" (exactly two of bottle, cup, pizza): positive variants are bottle ∧ cup ∧ ¬pizza, bottle ∧ pizza ∧ ¬cup, cup ∧ pizza ∧ ¬bottle; NB types are the three single-category cases plus the all-three case.

sample_type contains a coarse distinction between positive, near_boundary and far_from_boundary, while boundary_type contains the specific near_boundary type of the sample.

Columns

Both configs share this per-image block:

full adds image_id, image_index, file_name, and the 10 boolean label_<rule> columns (the exploded form of labels, so the viewer can filter and sort per rule).

fewshot adds the rule the row belongs to, and the per-image block specialized to that rule:

Few-shot / in-context learning

The fewshot config is the task definition: it states which images are relevant for evaluating each rule. Each rule gets 24 train examples (8 positive + 16 negative) and 40 test examples (20 + 20). Reconstruct one rule's task with a single filter — no join needed, since each row already carries its image's metadata:

fs = load_dataset("AIML-TUDA/COCOLogic-v2", "fewshot", split="train")
rule_8 = fs.filter(lambda x: x["rule_id"] == 8)
positives = rule_8.filter(lambda x: x["rule_sample_type"] == "positive")
negatives = rule_8.filter(lambda x: x["rule_sample_type"] != "positive")

Note rule 4 ("Either Dog or Car") has no far-from-boundary examples — all of its negatives are near-boundary.

The few-shot version assumes a working perception module: images are manually curated so that the relevant objects are clearly visible and the labels are correct. It is intended for few-shot / in-context rule learning, not for training perception from scratch.

Source JSON files

The four original JSONs (cocologic_{train,test}[_fewshot].json) sit at the repo root, byte-for-byte as the code repository reads them. The parquet configs above are generated from them and round-trip back to them exactly.

Note the rules block of the full JSONs is not exported to parquet: its integers are not COCO image ids but positional indices into the source COCO split's candidate pools, recorded before subsampling (and one group, rule_7.near_boundary[2] in train, mixes raw ids into that index space). The per-image labels / rule_variants / boundary_type fields carry the same information, are self-consistent, and are in real COCO id space. The rules block of the fewshot JSONs does use real COCO image ids and is exported as the fewshot config.

Row order

Parquet row order matches the source JSON key order, and image_index records it. This matters only if you reuse the precomputed tensors from the code repository, which are index-aligned to that order.

Licensing

The COCOLogic-V2 annotations are released under CC BY 4.0, consistent with the MSCOCO annotations they are derived from (© COCO Consortium). Images are not redistributed here; they remain subject to the COCO terms of use and the Flickr terms for the individual photographs. Please cite both COCO and COCOLogic-V2.

Citation

@article{steinmann2026cocologic,
  title={COCOLogic-V2: Identifying Logical Inconsistencies via Truly Hard-Negatives},
  author={Steinmann, David and W{\"u}st, Antonia and Kersting, Kristian and Stammer, Wolfgang},
  journal={arXiv preprint arXiv:2606.28194},
  year={2026}
}


@inproceedings{lin2014microsoft,
  title={Microsoft coco: Common objects in context},
  author={Lin, Tsung-Yi and Maire, Michael and Belongie, Serge and Hays, James and Perona, Pietro and Ramanan, Deva and Doll{\'a}r, Piotr and Zitnick, C Lawrence},
  booktitle={European conference on computer vision},
  pages={740--755},
  year={2014},
  organization={Springer}
}