savoji/minivlm-tablevqa-sft

minivlm-tablevqa-sft

Supervised fine-tuning mixture for improving visual table question answering in small vision-language models — built for the MiniVLMDocEval project to lift Qwen3.5-0.8B on TableVQABench.

Why this mixture (data-driven targeting)

We measured Qwen3.5-0.8B per TableVQABench sub-domain and found the weakness is Wikipedia-style visual-table lookup, not financial tables:

So this dataset is weighted toward table-lookup QA (the measured gap), with a general-document replay portion to prevent catastrophic forgetting. This deliberately departs from the common assumption that financial/numeric tables are the bottleneck — our own evaluation said otherwise.

Composition

• —records: 6211 · by subtype: {'table': 4900, 'replay': 1311} · by source: {'visual_tableqa': 4900, 'InfoVQA_VAL': 437, 'DocVQA_VAL': 437, 'ChartQA_TEST': 437}
• —table sources: preset:visual_tableqa (synthetic / non-Wikipedia by design — see overlap guarantee)
• —replay: held-out rows of DocVQA / ChartQA / InfoVQA / OCRBench
• —build seed: 42

No train/eval overlap (guarantee)

This set is built to be safely disjoint from the TableVQABench evaluation:

• —1262 distinct TableVQABench questions were used as an exclusion set; any training row matching one was dropped (0 dropped).
• —replay rows are drawn only from outside the fixed seed-42 evaluation subset of each protected benchmark (held out: {'DocVQA_VAL': 1000, 'ChartQA_TEST': 1000, 'OCRBench': 1000, 'InfoVQA_VAL': 1000, 'TableVQABench': 901}).
• —TableVQABench itself is never used as a training source.

Schema

Reproduce

Built by `scripts/build_table_sft.py` and pushed with scripts/push_dataset_hf.py. See the bundled manifest.json for the exact counts, sources, and overlap-guard results of this build.