HenryExcellent/SciDocBench

SciDocBench

Official data release for SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding. Project and evaluation code: InternLM/SciDocBench.

Dataset Summary

SciDocBench contains 496 evaluation instances derived from 124 expert-authored scientific-document questions. Each question is represented under four matched settings that cross English/Chinese questions with all-images-first/interleaved document representations.

The release references 7,052 images across instances and contains 2,758 unique image files after SHA-256 deduplication.

Files

• —data/test-00000-of-00001.parquet: Hugging Face test split.
• —SciDocBench.tsv: portable VLMEvalKit-compatible source table.
• —scidocbench.tsv: compatibility alias for the original repository path.
• —images/SciDocBench/: content-addressed document images.
• —release_manifest.json: counts and checksums.
• —manifests/images.jsonl: per-image hashes and reference counts.

In the Parquet file, image_path is a list of paths relative to the VLMEvalKit image root, while images contains repository-relative paths. Interleaved segments retain their ordering and refer to the rewritten image_path values.

Usage

Load the table with Hugging Face Datasets:

from datasets import load_dataset

dataset = load_dataset("HenryExcellent/SciDocBench", split="test")

For a local VLMEvalKit checkout, download the repository and place the TSV and image tree under LMUData:

hf download HenryExcellent/SciDocBench   --repo-type dataset   --local-dir SciDocBench

cp SciDocBench/SciDocBench.tsv "$LMUData/SciDocBench.tsv"
mkdir -p "$LMUData/images/SciDocBench"
cp -a SciDocBench/images/SciDocBench/. "$LMUData/images/SciDocBench/"

Evaluation

Use the SciDocBench integration in VLMEvalKit for standardized inference and evaluation.

License and Use

Benchmark annotations and scripts are released for research use under the terms stated by the SciDocBench project. Paper pages and figures originate from heterogeneous scientific sources and may retain source-specific copyright or license terms. Users are responsible for checking the applicable terms before redistribution or commercial use. No blanket license is asserted over third-party document imagery.

Citation

@article{wu2026scidocbench,
  title   = {SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding},
  author  = {Wu, Shenxi and Liu, Yuhong and Zhang, Haosong and Zou, Tongjin and Zhang, Yanxun and Chen, Gaochang and Liang, Dun and Wang, Jiaqi and Wang, Zhecan James and Zang, Yuhang and Lin, Dahua},
  journal = {arXiv preprint arXiv:2609.05141},
  year    = {2026}
}