monica-sekoyan/TickTockVQA-segmented

TickTockVQA Segmented (SAM3 crops)

Real world analog clock images from jaeha-choi/TickTockVQA, each cropped to the clock face by a SAM3 segmentation pass, packaged for grounded visual reasoning experiments with vision language models.

Every image is paired with a single fixed instruction and a ground truth time label. All answers are in H:MM format: this corpus contains no second hand, so it exercises real world hour and minute reading only and carries no H:MM:SS signal.

The task

Each row shares one prompt:

Read the analog clock. If a second hand is visible, return the time as H:MM:SS. If no second hand is visible, return the nearest minute as H:MM. Return only the time.

The answer field holds the ground truth, for example 10:40. Because no clock in this set shows a second hand, every answer resolves to the H:MM branch of the prompt.

Splits

test_small is a fixed, seeded, source and clocktype stratified subset of `test`. Its rows are duplicated across the two splits by design, so `testsmall is a strict subset of test and the two are not independent draws. Numbers on testsmall` are directly comparable to `test` because the samples are shared. Do not tune on `testsmall: selection is what dev` is for.

Provenance and the out of distribution split

The original TickTockVQA split reserves COCO, Open Images, and Clock Movies for testing and uses all other sources for training, so evaluation happens on sources never seen in training. That property is preserved here. Nothing from the source test pool is ever moved into train. The dev, test, and test_small splits are all carved from within the source test pool alone.

The dev and test division is stratified by source and clocktype and is group disjoint on `imagehash, so no original image contributes crops to both dev and test. test_small inherits that disjointness from test`.

Seeds: dev split 42, test_small subset 43 (base seed 42).

Source composition

Clock type composition

Segmentation, cropping, and attrition

Clock faces were localised with SAM3 (facebook/sam3, transformers backend) using the text prompt analog clock. The pass was configured as follows:

• —one detection kept per image (max_detections: 1), the highest scoring one, so each row corresponds to a single clock
• —detection and mask thresholds both 0.5 (threshold, mask_threshold)
• —the crop box is taken from the segmentation mask rather than the raw detection box (box_source: mask)
• —no post hoc score filtering (score_threshold: null), so lower confidence detections are retained rather than removed; the sam3_score column lets you filter by confidence downstream if you wish
• —images with no detection are skipped (on_no_detection: skip), which accounts for the attrition below
• —crops saved as png, batch size 8

Padding and the crop area cap

Before cropping, the mask derived box is expanded by a padding fraction of 0.25 on each side (padding: 0.25), so every crop keeps a quarter of the box width and height of surrounding context on each edge rather than cutting tight to the dial. This leaves a margin around the face, which helps preserve the outer tick ring and any bezel context.

If the padded crop would cover more than 0.7 of the original image area (max_crop_area_ratio: 0.7), the full original image is kept instead of cropping, on the grounds that such a crop would add little over the original. In that case the row's image is the original frame rather than a tight crop, which explains the presence of some large images in the crop size distribution below.

Attrition

Rows whose segmentation produced no crop were dropped:

The image column holds the segmented crop. Segmentation provenance is retained per row (sam3_score, crop_box, segmented_width, segmented_height) so the crop can be traced back to the detection that produced it. Original outputs were preserved as segmented; no truncated or missing detections were reconstructed.

Resolution distribution

The figure below shows image resolution before and after cropping, width on the x axis against height on the y axis, with train and test distinguished and opacity on the points. Both panels share the same axes, so the leftward and downward shift in the right panel shows directly how much cropping reduces resolution.

[image]

Two differences are worth noting. First, the segmented crops are much smaller than the source images: the median crop is roughly 250 pixels on a side, against original medians of about 512 pixels for train and about 770 pixels for test, and a share of crops fall below 100 pixels on the shorter side. Effective spatial resolution available to the encoder is therefore far lower than the raw images suggest, and a small crop tail is present. Second, the raw resolution gap between train and test, which reflects the different underlying sources (the held out test sources are higher resolution than the train sources), narrows substantially after cropping, so the crops are more comparable across the out of distribution boundary than the originals are.

Columns