PVIS2027-JT-6597/SciVisCap

<!-- Paste this as the dataset repo's README.md on the Hub (it becomes the Dataset Card). The dataset_info block (features schema + per-split row counts, which powers the Dataset Viewer table) is auto-generated by the Hub / datasets library on push -- don't hand-write it here, it'll be inserted above automatically once uploaded via push_to_hub() or the web uploader. -->

Dataset Card for SciVisCap

Dataset Summary

SciVisCap is a dataset for captioning scientific visualization (SciVis) figures -- generating a caption for a SciVis rendering, optionally using the paragraph(s) in the source paper that reference it. It contains 3,539 SciVis figures from 992 IEEE Vis and SciVis papers, drawn from the VIS30K corpus and paired with their published captions and figure-referencing paragraphs.

Supported Tasks

• —Image captioning (image-to-text): generate a caption from the rendering alone.
• —Context-conditioned captioning: generate a caption from the rendering plus its figure-referencing paragraph(s), or from the paragraph(s) alone (a text-only control condition).

Languages

English (en) -- all captions and referencing text are in English.

Dataset Structure

Data Instances

Each item is one figure from one paper: the rendering image, its published caption, the paragraph(s) that reference it, and paper/VIS30K metadata.

Data Fields

Data Splits

Splits are made at the paper level (by DOI, 80/10/10). Distributed as three folders (train/, validation/, test/).

Repository Structure

SciVisCap/
├── README.md
├── train/
│   ├── metadata.jsonl          (2,790 items)
│   ├── 1996/
│   │   ├── VisC.73.9.png
│   │   └── ...
│   ├── 1997/
│   ├── ...
│   └── 2020/
├── validation/
│   ├── metadata.jsonl          (367 items)
│   ├── 1996/
│   ├── ...
│   └── 2020/
└── test/
    ├── metadata.jsonl          (382 items)
    ├── 1996/
    ├── ...
    └── 2020/

Images are grouped by split, then by publication year, since VIS30K reuses figure filenames across different papers/years -- collisions only ever happen across years, never within the same year, so {year}/{filename} is always unique.

Dataset Creation

Source Data

Built from VIS30K, a corpus of figures and tables from IEEE Visualization publications. Items are restricted to the Vis/SciVis conferences and filtered by vis_type/ encoding_type tags (excluding schematic diagrams, GUI screenshots, bar charts, and unclassified figures), then manually classified as SciVis, InfoVis, both, or none by three annotators; only SciVis/both figures are kept.

Annotations

Published captions are recovered from VIS30K's caption-crop images via OCR. Figure-referencing paragraphs are extracted from the source PDF via Docling and matched to the target figure by number. Recurring OCR errors (e.g. misread figure prefixes) were corrected and re-run; remaining problem cases were manually reviewed and either fixed or excluded.

Considerations for Using the Data

• —Figure images and published captions originate from copyrighted papers (© their original authors / IEEE); this dataset redistributes them for non-commercial research use only, under the license below.
• —A nonzero fraction of candidate figures have no figure-referencing paragraphs at all -- these are excluded from the final dataset, not present with an empty field.
• —OCR- and regex-based extraction may leave residual errors despite automated correction and manual review.

Licensing Information

The SciVisCap annotations, extracted figure-referencing paragraphs, captions, metadata, and splits are released under CC BY-NC-SA 4.0. Figure images and published captions remain © their original authors / IEEE and are redistributed for non-commercial research use only. SciVisCap is derived from VIS30K; use of the figures is also subject to VIS30K's terms.