rollingcoconut/rav4-retrieval-assignment2

Image-to-Video Semantic Retrieval via Object Detection

Retrieve matching car exterior parts to video clips where those parts appear for RAV4.

Notes

• —Two parquet files provided
• —YouTube clips URI include start time, end time can be determined by start time + clip duration noted in row

Source Video

• —Video: YouTube - YcvECxtXoxQ
• —Segment: 18:50 to 24:00 (310 frames @ 1 fps)

Query Images

• —Dataset: aegean-ai/rav4-exterior-images

Detector

• —Model: YOLOv8n-seg fine-tuned on Ultralytics Car Parts Segmentation Dataset
• —Training: 15 epochs, 90/10 train/validation split
• —Classes: 23 car exterior parts (frontbumper, hood, wheel, frontglass, back_light ...)

Schema

Parquet 1: detections — Frame Detections

One row per frame. Each row includes the video frame image and a list of all detected car parts.

Example detections value:

[{"class_label": "front_bumper", "confidence": 0.82, "bbox": [553.5, 595.4, 2780.9, 1574.2]},
 {"class_label": "wheel", "confidence": 0.71, "bbox": [2534.9, 878.5, 2917.5, 1673.7]}]

Parquet 2: clips — Retrieval Results

One row per query-part-clip match. For each query image, car parts are detected and matched to ~3 second video clips.