Chrisyichuan/screenshot-training-naive-top2-hn-ablation

Chrisyichuan/screenshot-training-naive-top2-hn-ablation

Ablation variant of `Chrisyichuan/screenshot-training-natural-filtered-v2`.

Same queries, same positives. Only `neg_chunk_paths` differ.

The filtered-v2 dataset applies a Gemini VLM judge to filter false negatives out of the retrieved candidates. This ablation set skips that filter entirely: for every (query, chunkpath), we hit the text-retrieval search API for the top-10 results and keep the first two non-positive hits as `negchunk_paths`.

Use this dataset side-by-side with the filtered version to measure how much the Gemini false-negative filter helps.

Contents

• —train_hn.jsonl
• —eval_hn.jsonl
• —test_hn.jsonl
• —image_shards/ — tar-packed images (same scheme as the filtered dataset)

Row schema

{
  "query": "...",
  "chunk_path": "images/shard_123/shard_00001/123456.png.tiles/chunk_0000_00.png",
  "neg_chunk_paths": [
    "images/shard_234/shard_00002/234567.png.tiles/chunk_0000_01.png",
    "images/shard_345/shard_00003/345678.png.tiles/chunk_0000_02.png"
  ],
  "split": "train"
}

Exactly 2 hard negatives per row. Rows with fewer than 2 non-positive candidates in the top-10 are dropped (none in this release).

Split sizes

• —train: 104033
• —eval: 5779
• —test: 5781

Mining config

• —search API: localhost:30888/search
• —n_docs: 10
• —nprobe: 128
• —num_hard_negatives: 2
• —no VLM / LLM false-negative filter
• —no source_positive_rank / source_positive_score metadata (raw ablation input)

Image Storage

Images are packed as tar shards under image_shards/ to keep the file count manageable. To materialize them after download:

python extract_hf_image_shards.py --dataset-dir .