HenryExcellent/SciDocBench

1---2license: other3language:4- en5- zh6task_categories:7- visual-question-answering8- question-answering9pretty_name: SciDocBench10size_categories:11- n<1K12configs:13- config_name: default14  data_files:15  - split: test16    path: data/test-*.parquet17---18 19# SciDocBench20 21Official data release for [SciDocBench: A Workflow-Centered Benchmark and Data22Pipeline for Scientific Document Understanding](https://arxiv.org/abs/2609.05141).23Project and evaluation code: [InternLM/SciDocBench](https://github.com/InternLM/SciDocBench).24 25## Dataset Summary26 27SciDocBench contains 496 evaluation instances derived from28124 expert-authored scientific-document questions.29Each question is represented under four matched settings that cross English/Chinese30questions with all-images-first/interleaved document representations.31 32| Partition | Instances |33|---|---:|34| EN, all images first | 124 |35| EN, interleaved | 124 |36| ZH, all images first | 124 |37| ZH, interleaved | 124 |38 39The release references 7,052 images across instances and40contains 2,758 unique image files after SHA-256 deduplication.41 42## Files43 44- `data/test-00000-of-00001.parquet`: Hugging Face test split.45- `SciDocBench.tsv`: portable VLMEvalKit-compatible source table.46- `scidocbench.tsv`: compatibility alias for the original repository path.47- `images/SciDocBench/`: content-addressed document images.48- `release_manifest.json`: counts and checksums.49- `manifests/images.jsonl`: per-image hashes and reference counts.50 51In the Parquet file, `image_path` is a list of paths relative to the VLMEvalKit image52root, while `images` contains repository-relative paths. Interleaved `segments` retain53their ordering and refer to the rewritten `image_path` values.54 55## Usage56 57Load the table with Hugging Face Datasets:58 59```python60from datasets import load_dataset61 62dataset = load_dataset("HenryExcellent/SciDocBench", split="test")63```64 65For a local VLMEvalKit checkout, download the repository and place the TSV and image66tree under `LMUData`:67 68```bash69hf download HenryExcellent/SciDocBench   --repo-type dataset   --local-dir SciDocBench70 71cp SciDocBench/SciDocBench.tsv "$LMUData/SciDocBench.tsv"72mkdir -p "$LMUData/images/SciDocBench"73cp -a SciDocBench/images/SciDocBench/. "$LMUData/images/SciDocBench/"74```75 76## Evaluation77 78Use the SciDocBench integration in79[VLMEvalKit](https://github.com/open-compass/VLMEvalKit) for standardized inference80and evaluation.81 82## License and Use83 84Benchmark annotations and scripts are released for research use under the terms stated85by the SciDocBench project. Paper pages and figures originate from heterogeneous86scientific sources and may retain source-specific copyright or license terms. Users87are responsible for checking the applicable terms before redistribution or commercial88use. No blanket license is asserted over third-party document imagery.89 90## Citation91 92```bibtex93@article{wu2026scidocbench,94  title   = {SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding},95  author  = {Wu, Shenxi and Liu, Yuhong and Zhang, Haosong and Zou, Tongjin and Zhang, Yanxun and Chen, Gaochang and Liang, Dun and Wang, Jiaqi and Wang, Zhecan James and Zang, Yuhang and Lin, Dahua},96  journal = {arXiv preprint arXiv:2609.05141},97  year    = {2026}98}99```100