vkehfdl1/banana-merged

Banana-Merged

A synthetic multi-page visual question answering dataset with hard negatives, designed for fine-tuning visual document retrievers like ColFlor and ColPali.

Dataset Summary

Banana-Merged contains 1,100 training samples and 10,054 images (positive pages + hard negative variants). Each sample pairs a multi-page analytical query with a set of document images that collectively contain the answer, plus one or more hard negative documents that look visually and lexically similar but would be the wrong retrieval result.

The dataset was generated by the Nano Banana Pro pipeline using Gemini 3 Pro Image Preview for image generation and editing. The pipeline takes a seed document image (sourced from llamaindex/vdr-multilingual-train and similar corpora), synthesizes a multi-page document around it, generates a query that requires reading across all pages to answer, then produces hard negative variants by deliberately modifying specific facts or attributes while preserving the visual style.

Hard negatives are critical for training retrieval models to distinguish between documents that share surface-level keywords and layout but differ in the specific information needed to answer a query. Without them, models tend to overfit to lexical overlap and fail on visually similar but semantically distinct documents.

Supported Tasks

• —Visual document retrieval fine-tuning (primary use case): Train ColFlor/ColPali-style models using the positive/hard-negative pairs
• —Multi-page VQA: Each query requires synthesizing information across 2-6 document pages
• —Hard negative mining research: Study how free-form modification strategies affect retrieval difficulty

Languages

English only. Source seed documents were English-language PDFs, slides, and scanned pages.

Dataset Structure

Data Fields

Data Splits

There is no eval or test split. The dataset is intended for training only. Users should hold out their own evaluation set from separate document sources.

Example Record

Sample sample_sample_5d0bc43f5f61 (6 pages, 6 HN variants, forensic tools technical report):

{
  "query": "For all software applications analyzed in the 'Forensic Tools Analysis' chapter, extract their identified development languages and host operating system requirements, and determine which development language is the most commonly used across the tools.",
  "positive_pages": [
    "images/sample_sample_5d0bc43f5f61/positive/page_1.png",
    "images/sample_sample_5d0bc43f5f61/positive/page_2.png",
    "images/sample_sample_5d0bc43f5f61/positive/page_3.png",
    "..."
  ],
  "hard_negative_pages": [
    "images/sample_sample_5d0bc43f5f61/hard_negatives/hn_001/page_1.png",
    "images/sample_sample_5d0bc43f5f61/hard_negatives/hn_002/page_2.png",
    "images/sample_sample_5d0bc43f5f61/hard_negatives/hn_003/page_3.png",
    "..."
  ],
  "hard_negative_documents": [
    {
      "variant_id": "hn_001",
      "target_positive_page": 1,
      "paired_positive_path": "images/sample_sample_5d0bc43f5f61/positive/page_1.png",
      "hn_path": "images/sample_sample_5d0bc43f5f61/hard_negatives/hn_001/page_1.png",
      "alt_concept": "Forensic Tools Analysis: Lab Setup and Hardware Requirements",
      "rationale": "This variant is a 'hard' negative because it originates from the same 'Forensic Tools Analysis' chapter and shares the exact same visual layout and entity list. It mentions all five tools by name (MOBILedit!, Oxygen Forensic Suite, etc.), which will trigger high lexical scores in a retriever. However, it fails to answer the query because it focuses on Hardware Prerequisites (RAM, Disk space, CPU) rather than Development Language or Host OS requirements..."
    },
    {
      "variant_id": "hn_002",
      "target_positive_page": 2,
      "paired_positive_path": "images/sample_sample_5d0bc43f5f61/positive/page_2.png",
      "hn_path": "images/sample_sample_5d0bc43f5f61/hard_negatives/hn_002/page_2.png",
      "alt_concept": "Oxygen Forensic Suite Analysis -- Database and Artifact Extraction",
      "rationale": "This document mirrors the visual template of the target page exactly, using the same primary heading and layout. It is lexically dense with query keywords: 'Oxygen Forensic Suite' and lists of 'Operating Systems'. However, it describes internal database schemas and the OS of target mobile devices rather than the host application's development language (VC++ 9.0) or host machine OS requirements..."
    },
    "..."
  ],
  "metadata": {
    "sample_id": "sample_sample_5d0bc43f5f61",
    "original_image": "input/vdr_en_1000_retry2/en_11_3ac5e983d796ccbacf837e9de554e229e42af489.jpg",
    "document_type": "Technical report or software analysis document",
    "answer": "Visual C++ is the most commonly used development language across the five forensic tools. MOBILedit! uses Visual C++ 7.0, Oxygen Forensic Suite uses Visual C++ 9.0, and both Cellebrite UFED and Belkasoft Evidence Center use Visual C++ 10.0. Micro Systemation XRY is the only tool using Microsoft .NET (C#). Host OS requirements trend toward 64-bit Windows 10/11, while older tools retain 32-bit compatibility...",
    "num_positive_pages": 6,
    "num_hard_negative_pages": 6,
    "hard_negative_variants": ["..."]
  }
}

Hard Negative Pairing Explanation

A hard negative in this dataset is a document that would be the wrong retrieval result for a given query, but is difficult to reject because it shares the same visual layout, topic domain, and many surface-level keywords as the correct (positive) document. The key difference is that the HN omits or changes the specific facts needed to answer the query.

For example: a query asking for "development languages and host OS requirements" of forensic tools gets a hard negative that lists the same tools and mentions "operating systems" -- but refers to the target mobile device OS rather than the host machine OS. A retriever relying on keyword overlap would rank it highly; a well-trained model should reject it.

Using hard_negative_documents[]

Each entry in hard_negative_documents pairs one positive page with its corresponding HN page:

• —paired_positive_path -- the positive page image this HN was generated from
• —hn_path -- the edited HN page image
• —rationale -- a detailed explanation of what the HN shares with the positive and what it changes

The mapping is 1-to-1: hard_negative_documents[i] corresponds to positive_pages[i] (via target_positive_page).

from datasets import load_dataset

ds = load_dataset("vkehfdl1/banana-merged", split="train")
sample = ds[0]

for doc in sample["hard_negative_documents"]:
    print(f"HN variant {doc['variant_id']}:")
    print(f"  mimics positive page {doc['target_positive_page']}: {doc['paired_positive_path']}")
    print(f"  HN image: {doc['hn_path']}")
    print(f"  alt concept: {doc['alt_concept']}")
    print(f"  why hard: {doc['rationale'][:200]}...")

Dataset Creation

Pipeline Overview

Seed Image (page_1)
    |
    v
Gemini 3 Flash -- Visual Description + Document Type
    |
    v
Gemini 3.1 Pro -- Task Planner (query + answer + page plan)
    |
    v
Gemini 3 Flash -- Page Planner (per-page content descriptions)
    |
    v
Gemini 3 Flash -- Page Augmenter (detailed content, run in parallel)
    |
    v
Gemini 3 Pro Image Preview -- Image Generation (pages 2..N)
    |
    v
Gemini 3 Flash -- Hard Negative Variant Generator
    |             (free-form modification specs per page)
    v
Gemini 3 Pro Image Preview -- Hard Negative Image Editor
    |             (edits each page to match the HN spec)
    v
ColFlor Export (train.jsonl + images/)

Modification Strategy

The Hard Negative Variant Generator does not use a fixed taxonomy of modification types. It sees the query, the expected answer, and the page descriptions, then invents whatever modification is most effective at breaking the answer for that specific document. Common patterns include entity swaps, temporal shifts, numeric changes, scope swaps (e.g., "target device OS" vs. "host OS"), and partial deletions -- but the generator is free to use any approach. Each variant is described by a free-form rationale string.

Source Seeds

Seed images (page 1 of each sample) come primarily from llamaindex/vdr-multilingual-train and similar public document-image corpora. Pages 2..N are fully synthetic, generated by Gemini 3 Pro Image Preview to match the style and content plan derived from the seed.

Source Batches

1,209 samples were generated across 7 batches; 109 were removed during deduplication, leaving 1,100.

Considerations for Using the Data

Synthetic data biases. All page images beyond page 1 are AI-generated. Text rendering quality varies; some pages have minor OCR-unfriendly artifacts. Answer text is generated by an LLM and may contain hallucinated facts not actually visible in the images.

Hard negative rationale quality varies. The rationale strings are LLM-generated and occasionally over-explain or mischaracterize the modification. Treat them as useful hints, not ground truth.

Not suitable as a benchmark. This dataset is synthetic and was generated by the same model family used in many retrieval baselines. Use it for training only. Evaluate on held-out real-document benchmarks (e.g., ViDoRe).

Post-hoc repairs. Three samples had true image-generation failures and were regenerated in Wave 2/3: sample_sample_58145db80e59, sample_sample_7812fe47f68d, and sample_sample_a49b3b9295b0. Their pages are marked with regenerated_at timestamps in hard_negative_documents. The regenerated pages are visually consistent but may differ slightly from what the original pipeline would have produced.

No PII audit. Source seed images are from public document corpora, but no systematic PII scan was performed. If you find sensitive content, please open an issue.

License

Apache 2.0. See LICENSE.

Citation

@misc{banana-merged,
  title = {Banana-Merged: Multi-page VQA with Hard Negatives},
  author = {Kim, Jeffrey (Dongkyu)},
  year = {2026},
  url = {https://huggingface.co/datasets/vkehfdl1/banana-merged}
}

Changelog

v2 (current)

• —Repaired 40 merge-drop victims: pages that were silently truncated during the initial merge deduplication step were recovered from source batches
• —Regenerated 3 samples with true image-generation failures: sample_sample_58145db80e59, sample_sample_7812fe47f68d, sample_sample_a49b3b9295b0
• —Normalized schema: added top-level hard_negative_documents[] field with explicit paired_positive_path and hn_path per variant (replaces the denormalized metadata.hard_negative_variants list, which is retained for backward compatibility)

v1

• —Initial release: 1,100 samples, but 40 had silently truncated page lists due to a merge deduplication bug, and 3 had missing page images