somusan/ist-vqa-zip-full

IST-VQA — Multilingual Scene Text VQA Dataset

Curated dataset for Indic Scene Text Visual Question Answering covering three languages: Bengali (bn) · Hindi (hi) · Tamil (ta)

Summary

Sources: Real-world web-crawled scene photos (Google Image Crawler) · IndicSTR12 benchmark subset · Pipeline-generated synthetic images (synth_scene_text.py)

Annotation types:

• —*_vqa_no_gt.json — crawl images, questions only (no ground-truth answers)
• —*_vqa_single_turn.json — IndicSTR images, complete Q&A pairs
• —*_synth_vqa.json + annotations.json — synth images, Q&A pairs + COCO bounding boxes

Image formats: .jpg (~5,460) · .jpeg (~866, IndicSTR only) · .png (~344) · .webp (~247, crawl only)

Versions: Each language has v2 (initial crawl/synth) and v3/v4 (extended + quality-improved) splits. Synth v3/v4 uses higher minimum font size (40 px), always-on bold outline, no perspective warping, and processes all background images.

Data is organised into three source pillars:

Directory Tree

ist_vqa/
├── org_muli_lingual_crawl_data/          # Crawl + IndicSTR
│   ├── bengali_scene_text_crawl_v2/      # Bengali crawl — v2
│   ├── bangali_v3/                       # Bengali crawl — v3
│   ├── hindi_scene_text_crawl_v2/        # Hindi crawl — v2
│   ├── hindi_v3/                         # Hindi crawl — v3
│   ├── tamil_scene_text_crawl_v2/        # Tamil crawl — v2
│   ├── tamil_v3/                         # Tamil crawl — v3
│   └── indicstr12_multi_lingual/         # IndicSTR12 subset
│       ├── bengali/images/
│       ├── hindi/images/
│       ├── tamil/images/
│       └── single_turn_dataset/          # Merged VQA JSONs only (no images)
└── org_synth_data_multi_lang/            # Synthetic data
    ├── bengali_synth_text/               # Bengali synth — v1
    ├── bengali_synth_v3/                 # Bengali synth — v3
    ├── hindi_synth_text/                 # Hindi synth — v1
    ├── hindi_synth_v4/                   # Hindi synth — v4
    ├── tamil_synth_text/                 # Tamil synth — v1
    ├── tamil_synth_v3/                   # Tamil synth — v3
    └── temp/                             # Staging / background images for synthesis

1 · Crawl Data

Real-world scene text images downloaded via Google Image Crawler, split into v2 (initial) and v3 (extended/cleaned) batches.

Bengali Crawl

Hindi Crawl

Tamil Crawl

Crawl Grand Total

Annotation format: *_vqa_no_gt.json — single-turn VQA without ground-truth answers (questions generated from image metadata; answers require model inference or human labelling).

2 · IndicSTR12 Data

A curated multilingual subset extracted from the IndicSTR12 benchmark. All images are .jpeg.

Annotation format: *_vqa_single_turn.json — fully formed single-turn VQA pairs with ground-truth answers. `single_turn_dataset/` contains merged copies of all three language JSONs — no additional images.

3 · Synthetic Data

Background images sourced from crawl/IndicSTR are used as canvases. The synthesis pipeline (synth_scene_text.py) overlays Indic script text using MiDaS depth + Felzenszwalb segmentation to find flat surfaces.

Bengali Synthetic

Hindi Synthetic

Tamil Synthetic

Synthetic Grand Total

Annotation format: annotations.json (COCO-Text format) + *_synth_vqa.json (single-turn VQA pairs with ground-truth text).

What changed between versions

4 · Dataset Totals

Per Language

Image Extension Breakdown (entire dataset)

The difference between ~6,917 raw files and 6,727 annotated images is accounted for by background staging images in temp/ (805 images) that are used as synthesis inputs and not independently annotated.

5 · File Inventory