kelseye/test_data

<div align="center">

RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension

</div>

<p align="center"> 🌐 <a href="https://rpc-bench.github.io/" target="blank">Project Page</a> • 💻 <a href="https://github.com/zai-org/RPC-Bench" target="blank">GitHub</a> • 📖 <a href="https://arxiv.org/abs/2601.14289" target="_blank">Paper</a> </p>

<div align="center"> <img src="assets/pipeline.png" width="100%" /> </div>

RPC-Bench is a fine-grained benchmark for research paper comprehension. It is built from review-rebuttal exchanges of high-quality academic papers and supports both text-only and visual evaluation through complementary paper representations.

Data Structure

RPC-Bench is organized into train, dev, and test subsets. Split assignments are recorded in manifest.jsonl, and the original split JSON files are provided in split_metadata/ (train.json, dev.json, test.json).

md/ contains Markdown files parsed from each paper by MinerU. These files provide the text input for LLM-oriented evaluation.

parse/ contains the full MinerU parsing outputs for each paper, including structured layout and content artifacts.

pdf/ contains the original paper PDFs.

vlm/ contains page images rendered from the PDFs with PyMuPDF at 200 DPI for VLM-oriented evaluation.

RPC-Bench/
├── README.md
├── manifest.jsonl
├── split_metadata/
│   ├── train.json
│   ├── dev.json
│   └── test.json
├── parse/
│   ├── train/
│   │   └── <paper_id>/
│   ├── dev/
│   │   └── <paper_id>/
│   └── test/
│       └── <paper_id>/
├── md/
│   ├── train/
│   │   └── <paper_id>/
│   │       └── <paper_id>.md
│   ├── dev/
│   │   └── <paper_id>/
│   │       └── <paper_id>.md
│   └── test/
│       └── <paper_id>/
│           └── <paper_id>.md
├── pdf/
│   ├── train/
│   │   └── <paper_id>.pdf
│   ├── dev/
│   │   └── <paper_id>.pdf
│   └── test/
│       └── <paper_id>.pdf
└── vlm/
    ├── train/
    │   └── <paper_id>/
    ├── dev/
    │   └── <paper_id>/
    └── test/
        └── <paper_id>/

Practical Uses

RPC-Bench can be used to try paper-centric systems that require broader document understanding rather than local snippet matching.

• —Research paper comprehension: try models on full-paper understanding, including core concepts, methods, and experimental findings.
• —Long-context evaluation: try whether longer context windows or long-context architectures improve document-level reasoning.
• —Multimodal reasoning: try models that combine textual evidence with page-level figures, tables, and diagrams in the original PDF layout.
• —RAG system diagnosis: try retrieval, chunking, and evidence-fusion strategies for paper-centric workflows beyond snippet-level retrieval accuracy.

Citation

@article{chen2026rpc,
  title={RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension},
  author={Chen, Yelin and Zhang, Fanjin and Sun, Suping and Pang, Yunhe and Wang, Yuanchun and Song, Jian and Li, Xiaoyan and Hou, Lei and Zhao, Shu and Tang, Jie and others},
  journal={arXiv preprint arXiv:2601.14289},
  year={2026}
}