llm-jp/WAON-Bench

WAON-Bench: Japanese Cultural Image Classification Dataset

<div align="center" style="line-height: 1;">

| <a href="https://huggingface.co/collections/llm-jp/waon" target="blank">🤗 HuggingFace</a> &nbsp;| <a href="https://arxiv.org/abs/2510.22276" target="blank">📄 Paper</a> &nbsp;| <a href="https://github.com/llm-jp/WAON" target="_blank">🧑‍💻 Code</a> &nbsp;|

<br/>

</div>

<img src="WAON-Bench.jpg" alt="Overview of WAON-Bench" width="100%"/>

WAON-Bench is a manually curated image classification dataset designed to benchmark Vision-Language models on Japanese culture. The dataset contains 374 classes across 8 categories (animals, buildings, events, everyday life, food, nature, scenery, and traditions), with 5 images per class, totaling 1,870 examples.

How to Use

⚠️ This repository is a mirror of the original dataset hosted at https://gitlab.llm-jp.nii.ac.jp/datasets/WAON-Bench

Due to copyright restrictions, WAON-Bench dataset with images is hosted only on a domestic server and are not included in this mirror.

To use WAON-Bench, first download the dataset from:

git clone https://gitlab.llm-jp.nii.ac.jp/datasets/WAON-Bench
mv WAON-Bench/data .

After placing the dataset directory locally, you can load each dataset using the 🤗 datasets library:

from datasets import load_dataset

ds = load_dataset("data")

Data Collection Pipeline

We followed the pipeline below to construct the dataset:

1. 1.Class Definition: A total of 374 class names were manually defined and grouped into eight top-level categories: animal, building, event, everyday, food, nature, scenery, and tradition.
2. 2.Image Selection: For each class, 5 images were manually retrieved using Google Image Search. \ Images were selected based on the following criteria:
3. 3.The image should clearly represent the intended class.
4. 4.It should not contain elements that could be easily confused with other classes.

Dataset Format

Each sample includes:

• —class: Class name
• —url: Image URL
• —category: Class category

Example:

{'class': '柴犬', 'url': 'https://img.wanqol.com/2020/11/6e489894-main.jpg?auto=format', 'category': 'animal'}

Dataset Statistics

• —Total classes: 374

• —Total images: 1,870

• —Class num per category | class | animal | building | event | everyday | food | nature | scenery | tradition | total | |----------:|-------:|---------:|------:|---------:|-----:|-------:|--------:|----------:|------:| | count | 41 | 40 | 29 | 45 | 55 | 27 | 75 | 62 | 374 |

• —Example Class Names per Category |category | class names| |:-----------|--------:| | animal | '柴犬', 'エゾシカ', 'ニホンカモシカ', 'イノシシ', ...| | building | '鳥居', '茶室', '合掌造り', '町家', '縁側', ...| | event | '花見', '花火大会', '盆踊り', '運動会', '卒業式', '成人式', ...| | everyday | 'カラオケ', '温泉', '屋台', '洗濯物', '敷布団', ...| | food | '茄子', 'しらす', 'ラーメン', '焼き鳥', '焼肉', ...| | nature | '桜', '梅', '藤', '松, '噴火', ...| | scenery | '茶畑', '雪国の街並み', '漁港', '砂防ダム', '石垣', ...| | tradition| '華道', 剣道', '柔道', '弓道', ...|

• —t-SNE Visualization of SigLIP2 Embeddings

The figure below shows a 2D t-SNE projection of image embeddings generated using google/siglip2-base-patch16-256. Each point represents one image in the dataset.

<img src="sigliptsnevisualization.png" alt="t-SNE Visualization" width="50%"/>

LICENSE

This dataset is licensed under the Apache License 2.0.

Citation

@misc{sugiura2025waonlargescalehighqualityjapanese,
      title={WAON: Large-Scale and High-Quality Japanese Image-Text Pair Dataset for Vision-Language Models}, 
      author={Issa Sugiura and Shuhei Kurita and Yusuke Oda and Daisuke Kawahara and Yasuo Okabe and Naoaki Okazaki},
      year={2025},
      eprint={2510.22276},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2510.22276}, 
}