maximilian-franz/basil-instances-archive-3

maximilian-franz/basil-instances

Per-plant-instance segmented crops derived from ['maximilian-franz/basil', 'maximilian-franz/basil-2'], one row per (original frame × confirmed plant instance).

Layout

ImageFolder-style dataset: metadata.csv at the repo root, with a file_name column pointing to each row's masked crop under images/<plant_instance_id>/<NNNN>.png, and a bbox_file_name column pointing to the same row's plain rectangular crop under images_bbox/<plant_instance_id>/<NNNN>.png. Load with:

from datasets import load_dataset
ds = load_dataset("maximilian-franz/basil-instances")

Columns

• —file_name — relative path to the masked crop (see Layout above)
• —bbox_file_name — relative path to the unmasked rectangular crop of the same padded box
• —camera_id, tower_id, timestamp — carried over from the source frame
• —plant_instance_id — {tower_id}_{camera_id}_pos{n}, n assigned by angle-sorting confirmed t0 centroids clockwise from 12 o'clock around their group's centroid
• —bbox_x, bbox_y, bbox_w, bbox_h — instance bounding box in the original (uncropped) frame's pixel coordinates, before the 20px padding applied to the crops
• —mask_iou_score — sigmoid of the SAM 3 tracker's per-frame object-score logit for this instance; a stability/confidence proxy, not IoU against ground truth
• —flagged_for_review — True if this (frame, instance) tripped the merge-detection heuristics below

Segmentation & tracking

• —Model: SAM 3 (facebook/sam3), via transformers (Sam3Model/Sam3Processor for candidate discovery, Sam3TrackerVideoModel/ Sam3TrackerVideoProcessor for video propagation).
• —Candidate text prompt: "basil seedling" (pcs_score_threshold=0.5, pcs_mask_threshold=0.5); heuristic greenness-threshold + connected-components fallback when unavailable.
• —Manual review: instances confirmed via an in-notebook widget at each of the 8 (tower_id, camera_id) groups' earliest frame; 557/3584 (15.5%) of output rows were flagged for review by the merge-detection heuristics below.
• —Merge-detection thresholds: an age-indexed growth budget fit per plant instance from its own area trajectory, flagged as a spike above cfg.merge_growth_upper_ratio (1.75×) or a collapse below cfg.merge_growth_lower_ratio (0.4×) the fitted expected area at that frame's age; pairwise mask IoU threshold 0.05. Independently, video propagation (Section 4) clips every propagated mask each frame to its own last-known bounding box expanded by cfg.propagation_clip_margin_px (40px) as a preventative safety valve, so an absorbed neighboring-plant region is stopped from entering the propagated mask in the first place rather than only being caught after the fact by the thresholds above.
• —Crop format: bounding box padded by 20px, shipped as two crops per row: the mask-zeroed crop (file_name), with pixels outside the instance mask zero-filled to avoid leaking neighboring-plant pixels into the image branch of the downstream anomaly-detection model, and the plain rectangular crop (bbox_file_name), with no masking applied, useful wherever the raw unmasked appearance inside the box matters (e.g. visual QA, or any downstream use that doesn't want the mask's zero-fill artifact).

Known limitations

• —Stereo cross-validation between a tower's two cameras is out of scope for this pipeline — each (tower_id, camera_id) sequence is segmented and propagated fully independently.
• —Degenerate/empty bounding boxes in the source data: 0 row(s).