Mausul/khondo

Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms

Real Bangla and English government forms assembled into document packets for the packet-splitting task: given a packet of concatenated form pages, recover which pages belong to each source document and restore each document's original page order.

This is the dataset and its schema. The full benchmark pipeline (inference, evaluation, and analysis) lives in the code repository: [github.com/mausulazad/Khondo](https://github.com/mausulazad/Khondo). Paper: arXiv:2607.21780.

Khondo provides 1,950 packets, 390 per variant across 5 concatenation schemes, built from 423 real government forms across 14 administrative domains. It is bilingual (Bangla and English) and vision-native: models read the page images directly, without OCR. Each variant is split train/validation/test at the form level, with no form shared across splits.

Variants

Five packet-construction strategies of increasing difficulty. Each packet concatenates pages from one or more source forms; shuffled variants permute the page order.

Structure

datasets/
  <variant>/ground_truth_json/<train|validation|test>/*.json    # one packet per file (authoritative ground truth)
  <variant>/<train|validation|test>.csv                         # row-per-page flattening (dataset viewer)
  xling/<bn|en>/<variant>/ground_truth_json/test/*.json         # cross-lingual strata (test only, JSON only)
images/
  <domain>/<form>/pNN.jpg                                       # form page images

The five configs above expose the per-variant CSVs to the dataset viewer and load_dataset. The packet ground truth (ground_truth_json/) and the cross-lingual strata (xling/) are JSON files, read directly rather than through load_dataset.

Data formats

Ground-truth JSON

One file per packet, named by doc_id. This is the authoritative ground truth for evaluation.

image_path is relative to the download root, so it resolves directly after snapshot_download.

CSV

A row-per-page flattening of the packets, with columns doc_type, parent_doc_name (the packet UUID), local_doc_id, page, image_path, and group_id. Page order for shuffled variants comes from the ground-truth JSON.

Cross-lingual strata

datasets/xling/ provides size-matched monolingual test packets, Bangla-only (bn) and English-only (en), built with the same construction strategies, 100 packets per language per variant. They hold packet structure fixed while varying language, for measuring how packet-splitting performance shifts across languages.

Load

from huggingface_hub import snapshot_download
import os, glob, json

root = snapshot_download("Mausul/khondo", repo_type="dataset")

gt_dir = os.path.join(root, "datasets", "mono_seq", "ground_truth_json", "test")
packet = json.load(open(glob.glob(os.path.join(gt_dir, "*.json"))[0], encoding="utf-8"))

# image_path is repo-relative and resolves under `root`.
rel = packet["subdocuments"][0]["pages"][0]["image_path"]   # e.g. "images/agriculture/004/p01.jpg"
page_path = os.path.join(root, *rel.split("/"))

The per-variant CSVs load the same way through load_dataset("Mausul/khondo", "mono_seq").

Reproducing the benchmark

Inference, evaluation, and analysis live in the code repository. Download the dataset into the repository root so datasets/ and images/ sit beside the code, or point KHONDO_DATA_ROOT at the download location. Model predictions ship with the code, so the published numbers reproduce without re-running inference. See github.com/mausulazad/Khondo.

Source and attribution

Khondo adapts the packet-splitting methodology of DocSplit (Islam et al., arXiv:2602.15958; `amazon/doc_split`, CC-BY-NC-4.0) for Bangla and English forms, and adds the cross-lingual strata.

License

Code: MIT. Dataset: CC-BY-NC-4.0, following DocSplit and RVL-CDIP-N-MP.

Citation

@misc{azad2026khondomultimodalbenchmarkdocument,
  title={Khondo: A Multimodal Benchmark for Document Packet Splitting of Bangla Forms}, 
  author={Abu Tyeb Azad and Fahim Ahmed and Ishita Sur Apan and Ezharuddin Jubaer and Sumaiya Karim Katha and Armun Alam and Amin Ahsan Ali and Aman Chadha and Md Mofijul Islam and AKM Mahbubur Rahman},
  year={2026},
  eprint={2607.21780},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2607.21780}, 
}