Datasetpublic

sci-m-wang/C4-Eval

C4-Eval C4-Eval is the evaluation set for C4 Bench, a Chengyu-based benchmark for measuring whether multimodal language models can understand cross-concept creativity. The release contains the original images, the corresponding idiom answers, and the complete task-specific questions used for evaluation. 221 base items: 37 human-designed seed figures and 184 bridge-controlled synthetic figures. 1,105 evaluation instances: five task formulations for every base item. Language:… See the full description on the dataset page: https://huggingface.co/datasets/sci-m-wang/C4-Eval.

sourceHugging Faceupdated 1mo agoView on Hugging Face

0likes693downloads

Dataset Card

C4-Eval

C4-Eval is the evaluation set for C4 Bench, a Chengyu-based benchmark for measuring whether multimodal language models can understand cross-concept creativity. The release contains the original images, the corresponding idiom answers, and the complete task-specific questions used for evaluation.

—221 base items: 37 human-designed seed figures and 184 bridge-controlled synthetic figures.
—1,105 evaluation instances: five task formulations for every base item.
—Language: Chinese.
—Code and evaluation tools: github.com/sci-m-wang/C4

Tasks

Task	Evaluation form	Expected response
`H0`	Image-only idiom identification	One Chinese idiom
`H1`	Idiom identification with a generic cross-concept hint	One Chinese idiom
`H4`	Candidate-constrained idiom identification	One idiom from four candidates
`E0`	Free idiom identification and bridge explanation	Structured JSON
`E1`	Bridge explanation with the gold idiom provided	Structured JSON

The primary leaderboard score uses exact answer recovery over H0, H1, H4, and E0, for a total of 884 instances. E1 is retained for explanation analysis and is excluded from the primary score.

Load the evaluation set

python

from datasets import load_dataset

dataset = load_dataset("sci-m-wang/C4-Eval", split="test")
example = dataset[0]

print(example["question"])
print(example["answer"])
print(example["image"])

The image column contains a resolvable Hub URL. image_path gives the corresponding repository-relative path, which is useful when the dataset has been downloaded locally.

Evaluation fields

Field	Description
`instance_id`	Unique task-level identifier
`item_id`	Identifier shared by the five formulations of one image
`origin`	Human seed figure or synthetic bridge-controlled figure
`image`	URL of the original image on the Hub
`image_path`	Repository-relative image path
`task`	One of `H0`, `H1`, `H4`, `E0`, or `E1`
`task_name`	Descriptive task name
`question`	Complete model-facing prompt for this task instance
`answer`	Gold Chinese idiom
`answer_aliases`	Accepted answer aliases, when applicable
`candidates`	Four candidate idioms for `H4`; empty otherwise
`level`	Bridge-construction level for synthetic items
`level_mode`	Bridge configuration for synthetic items
`explanation_reference`	JSON-encoded reference bridge annotation

data/items.jsonl additionally provides one row per base item. data/task_templates.json records the five task definitions. The metadata/ directory contains the construction annotations, reviewed bridge networks, and manual scene specifications used to produce the release.

Repository layout

text

data/eval.jsonl                 # 1,105 model-facing task instances
data/items.jsonl                # 221 base items
data/task_templates.json        # task definitions
images/human/                   # 37 human-designed seed figures
images/synthetic/               # 184 bridge-controlled figures
metadata/                       # construction and provenance metadata
export_report.json              # release counts and schema version

Reproducibility

The public code repository can rebuild the task views, validate image coverage, score normalized exact answers, and export this Hugging Face layout. The published images retain the original files used by the benchmark.

Citation

If you use C4 Bench in your research, please cite the arXiv preprint:

bibtex

@misc{wang2026mllmsdecodecreativeleap,
      title={Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding},
      author={Ming Wang and Yuqing Zhang and Tingna Xie and Xiangju Li and Xiaocui Yang and Daling Wang and Shi Feng and Yifei Zhang},
      year={2026},
      eprint={2608.06501},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2608.06501},
}