ShyFoo/CountHallu-dataset-ToyShape

CountHalluSet — ToyShape

Synthetic dataset from [Counting Hallucinations in Diffusion Models](https://arxiv.org/abs/2510.13080) (arXiv:2510.13080). Part of CountHalluSet, a suite with well-defined counting criteria used to measure counting hallucination — a diffusion model generating the wrong number of instances, even for patterns absent from its training data.

What's inside

128×128 RGB images of non-overlapping white shapes on a black background. Each image holds 1–3 shapes drawn from {triangle, square, pentagon}, at most one instance per type. This defines the counting criterion: a correct sample has each present shape exactly once; two of any shape, or an empty image, is a hallucination.

ToyShape/
├── images/       # 00000.png, 00001.png, ...
└── labels.csv    # filename, triangle, square, pentagon   (each count ∈ {0, 1})

Default release: 30,000 samples.

Usage

huggingface-cli download ShyFoo/CountHallu-dataset-ToyShape \
    --repo-type dataset --local-dir $DATASET_ROOT/ToyShape

Load with the reference code (counthallu.datasets.ToyShape) or regenerate from scratch — the generator is deterministic given a seed:

python -m counthallu.datasets.toyshape --data_root $DATASET_ROOT --num_samples 30000

See the CountHallu repository for training and the full evaluation protocol.

Citation

@article{fu2025counting,
  title={Counting Hallucinations in Diffusion Models},
  author={Fu, Shuai and Zhou, Jian and Chen, Qi and Jing, Huang and Nguyen, Huy Anh and Liu, Xiaohan and Zeng, Zhixiong and Ma, Lin and Zhang, Quanshi and Wu, Qi},
  journal={arXiv preprint arXiv:2510.13080},
  year={2025}
}