busy-pig/ChinaPaint

CCPP

Evaluating and Benchmarking Classical Chinese Poetry-to-Painting for Multimodal Large Language Models

CCPP: Classical Chinese Poetry-to-Painting Project

A comprehensive project supporting the research on Classical Chinese Poetry-to-Painting (CCPP) generation and evaluation, including benchmark datasets, human painting references, model outputs, and auxiliary scripts. This project serves as the official code & data repository for the corresponding academic paper, focusing on assessing Multi-Modal Large Language Models (MLLMs)’ ability to understand traditional Chinese culture and generate context-aligned paintings.

Table of Contents

• —Project Overview
• —Key Data Files (Priority Order)
• —1. CCPP-Bench: `all_imge_meta.xlsx`
• —2. Human Painting References: `human_painting.zip`
• —Folder Structure & Description
• —File Details
• —Quick Start
• —License

Project Overview

The CCPP project is designed to facilitate research on "Classical Chinese Poetry-to-Painting" generation. It provides a high-quality benchmark (CCPP-Bench), human-created painting samples, MLLM-generated painting outputs, and tools for data processing/evaluation. The project aims to enable reproducible research on evaluating MLLMs’ understanding of classical Chinese culture (poetry, idioms, classical prose) and their cross-modal generation capabilities. We provide a partial dataset (300 samples) and human-painting. The full dataset will be released upon paper acceptance.

Key Data Files (Priority Order)

1. CCPP-Bench: all_imge_meta.xlsx

Located in: benchmark/CCPP-Bench/ As the core metadata file of CCPP-Bench (the project’s benchmark dataset), this Excel file contains structured metadata for all samples in the dataset, supporting traceability, categorization, and statistical analysis. Its key contents include:

• —Text Metadata:
• —Text-Style: Classification of textual content (382 Idioms, 466 Shi/Ci Poetry, 231 Wen Yan Wen (Classical Prose)).
• —Text-Dynasty: Historical period of the text (Warring States, Han, Tang, Song, Yuan, Ming, Qing, and Wei/Jin/Southern-Northern Dynasties).
• —Poetry-Theme: Thematic category of the text (Frontier and Warfare, Landscape and Countryside, Lyricism and Self-Expression, etc.).
• —Image Metadata:
• —Painting-Object: Subject of the painting (Taoist and Buddhist, Genre, Flower-and-Bird, Landscape, Portrait, Figure).
• —Painting-Technique: Traditional Chinese painting techniques (Bai Miao, Gongbi Dan Cai, Gongbi Zhong Cai, Xieyi Shuimo).
• —Mapping Relationships: Links each textual sample to its corresponding human/MLLM-generated painting, enabling targeted evaluation (e.g., assessing MLLMs’ performance on Tang Dynasty poetry or Xieyi-style paintings).

This file is the foundation for dataset usage—researchers can filter samples by metadata (e.g., "Tang Dynasty + Landscape theme") to conduct grouped experiments.

2. Human Painting References: human_painting.zip

Located in: human_paintings/ A compressed archive of human-created paintings inspired by classical Chinese poetry, serving as the "gold standard" for evaluating MLLM-generated paintings. Key details:

• —Collection Sources(Human score on six dimensions：`CCPP_image_scoring_annotations.xlsx`):
• —Public platforms (classical poetry websites, educational/cultural portals, Xiaohongshu).
• —Authorized electronic scrolls (via copyright email communication).
• —Chinese textbook illustrations (PEP edition, junior/senior high school).
• —Sample Characteristics:
• —Total samples: ~650 (300 from public poetry sites, 100 from educational platforms, 50 from other sites, 50 from authorized scrolls, 100 from textbooks).
• —Artistic styles: Covers core traditional Chinese painting styles (ink wash, Gongbi, blue-and-green landscape, etc.).
• —Usage: Unzip to the human_paintings/ folder (automatically organized by Painting-Technique or Text-Theme). These paintings are used to compare with MLLM outputs in 6-dimensional human evaluation (Relevance, Accuracy, Aesthetic, etc.).

Folder Structure & Description

The project follows a modular folder structure for clarity and usability:

File Details

Beyond the key files/folders above, the project includes the following root-level files:

Quick Start

1. Clone the Repository

git clone https://github.com/yyy1103/CCPP.git
cd CCPP

### 2. Install Dependencies

pip install -r requirements.txt

3. Access Key Data

·CCPP-Bench Metadata: Open benchmark/allimgemeta.xlsx with Excel/LibreOffice to explore sample metadata. · Human Paintings: Unzip humanpainting.zip to humanpaintings/ using:

unzip human_painting.zip -d human_paintings/
Model Outputs: Navigate to model_outputs/ to access pre-generated paintings from different MLLMs.


This project is licensed under the Apache License 2.0—see the LICENSE file for details. Non-commercial academic use is permitted; for commercial use, please contact the project maintainers.