ECCV26-Tomato-Phenotyping/SYNTOM

SYNTOM: Synthetic Tomato Greenhouse Segmentation

[image]

77,217 photorealistic renders of greenhouse tomato plants (68,328 train / 8,889 val, 1920x1080) with pixel perfect ground truth for two tasks:

• —Semantic segmentation: 4 organ classes plus background, single channel PNG masks
• —Instance segmentation: whole plant instances in COCO format

Released with Text-conditioned Segmentation for Tomato Phenotyping via Procedural Synthetic Data, where it is used to fine-tune SAM 3 for greenhouse crop organs.

Fine-tuned weights

The SAM 3 model trained on this dataset is released alongside it:

[ECCV26-Tomato-Phenotyping/SYNTOM-SAM3](https://huggingface.co/ECCV26-Tomato-Phenotyping/SYNTOM-SAM3)

It reaches 0.6562 macro fruit IoU on three real tomato datasets against 0.5269 for zero-shot SAM 3, trained only on the synthetic frames published here.

The weights are a derivative of Meta's SAM 3 and carry the SAM License, not the CC BY 4.0 licence that covers this dataset.

Labels are generated by the renderer rather than drawn by annotators, so every mask is exact and complete, including thin stems, occluded fruit and distant plants. Frames average 185 plants and reach 849, which makes this a dense and heavily occluded benchmark.

At a glance

Layout

The release is packed as WebDataset tar shards. One shard holds a few hundred frames, and each frame is three members sharing a key:

SYNTOM/
├── data/
│   ├── train/  train-000000-of-000140.tar ...   140 shards, 289.2 GiB
│   └── val/    val-000000-of-000019.tar ...      19 shards, 38.1 GiB
├── annotations/
│   └── instances_val.json               COCO val instances, pycocotools ready
├── splits/train.txt, splits/val.txt     one stem per line
├── classes.json                         class ids, names and palette
├── dataset_stats.json                   counts and per-class pixel statistics
├── preview/contact_sheet.png            rendered label examples
└── visualize_labels.py                  colorize and overlay the masks

Inside a shard:

train_012345.png        the raw render, 1920x1080 RGBA
train_012345.mask.png   the semantic mask, uint8, pixel value = class id
train_012345.json       {"image": <coco image record>, "annotations": [...]}

Frames are shuffled at a fixed seed before being assigned to shards, so any single shard is a diverse sample rather than one contiguous camera run.

Loading

from datasets import load_dataset

ds = load_dataset("ECCV26-Tomato-Phenotyping/SYNTOM", split="val", streaming=True)
sample = next(iter(ds))
image = sample["png"]              
mask = sample["mask.png"]          
anns = sample["json"]["annotations"]

Or with the webdataset library directly:

import io, json
import numpy as np
import webdataset as wds
from PIL import Image

url = ("https://huggingface.co/datasets/ECCV26-Tomato-Phenotyping/SYNTOM/"
       "resolve/main/data/val/val-{000000..000018}-of-000019.tar")

def decode(sample):
    return {
        "image": Image.open(io.BytesIO(sample["png"])).convert("RGB"),
        "mask": np.array(Image.open(io.BytesIO(sample["mask.png"]))),   
        "annotations": json.loads(sample["json"])["annotations"],
    }

ds = wds.WebDataset(url).map(decode)

Download only what you need:

from huggingface_hub import snapshot_download

# val split only, enough to evaluate
snapshot_download("ECCV26-Tomato-Phenotyping/SYNTOM", repo_type="dataset",
                  local_dir="SYNTOM",
                  allow_patterns=["data/val/*", "annotations/instances_val.json",
                                  "splits/*", "*.json", "*.py", "preview/*"])

Semantic segmentation

Single channel uint8 PNG, pixel value = class id. There is no ignore index: every pixel is labeled and background is a real class, so use reduce_zero_label=False in mmseg terms and do not subtract 1.

stem covers stems, peduncles and petioles; fruit covers fruit and sepals. The palette is the one in classes.json and is a display convention only, with no effect on the stored labels.

flower is the rarest class at 0.04% of pixels, so report per-class IoU alongside mIoU and weight the loss accordingly.

Label PNGs store the class id directly in the pixel value, following the same convention as Cityscapes labelTrainIds, ADE20K and COCO-Stuff. Since the ids are small numbers, the masks appear dark in an image viewer; use the palette below, or preview/contact_sheet.png, to look at them. Storing plain ids rather than a palette means PIL, OpenCV, mmcv and scikit-image all read back the same 0..4 values.

mmsegmentation: classes=["background","leaf","stem","flower","fruit"], reduce_zero_label=False, num_classes=5. Metric: mIoU.

Viewing the labels

preview/contact_sheet.png has ready made examples. To render one yourself from a shard:

import io, glob
import numpy as np, webdataset as wds
from PIL import Image

PALETTE = np.array([[0, 0, 0], [0, 255, 0], [255, 0, 0],
                    [255, 255, 0], [0, 0, 255]], np.uint8)   

s = next(iter(wds.WebDataset(glob.glob("data/val/*.tar")[0], shardshuffle=False)))
image = np.array(Image.open(io.BytesIO(s["png"])).convert("RGB"))
mask = np.array(Image.open(io.BytesIO(s["mask.png"])))
overlay = (0.45 * image + 0.55 * PALETTE[mask]).astype(np.uint8)
Image.fromarray(np.concatenate([image, PALETTE[mask], overlay], axis=1)).save("preview.png")

visualize_labels.py does the same for an unpacked images/<split> plus labels/<split> tree:

python visualize_labels.py val_001889              
python visualize_labels.py --contact-sheet 8       

Instance segmentation

COCO format with a single category, plant (id 1), evaluated with pycocotools and COCOeval on mask AP. Per frame annotations are in each sample's .json inside the shards. annotations/instances_val.json additionally provides the val split as one standard COCO file so that COCOeval works out of the box.

Each annotation carries segmentation as RLE, bbox in absolute pixels, area, iscrowd: 0, plus two extra fields:

• —plant_id: the individual plant, 900 distinct values. Stable across frames of the same scene, so it can also be used for tracking or re-identification.
• —model_id: which procedural plant asset the plant was grown from, 10 distinct values, near uniformly distributed. Many plants share one, so it is a variant tag rather than an instance id.

RLE is used rather than polygons so that concave, occlusion split silhouettes are represented exactly. Masks are visible region, as in COCO, and within an image the plant masks are disjoint and together cover exactly the non background pixels of the semantic mask, so the two tasks stay consistent. Object sizes span a wide range, so filter on area if your task needs a minimum.

from pycocotools.coco import COCO
from pycocotools import mask as maskutil

coco = COCO("annotations/instances_val.json")
img = coco.loadImgs(coco.getImgIds()[0])[0]
anns = coco.loadAnns(coco.getAnnIds(imgIds=img["id"]))
plants = [maskutil.decode(a["segmentation"]) for a in anns]  

Citation

Accepted at the 11th Workshop on Computer Vision in Plant Phenotyping and Agriculture (CVPPA), in conjunction with the European Conference on Computer Vision (ECCV), Malmo, Sweden, September 2026.

@inproceedings{mounir2026syntom,
  title         = {Text-conditioned Segmentation for Tomato Phenotyping via
                   Procedural Synthetic Data},
  author        = {Mounir Samy, Cieslak Mikolaj, Dhieb Najmeddine,
                   Ghazzai Hakim, Klein Jonathan, Froehlich Katja,
                   Pirk Soeren, Palubicki Wojciech, Setti Gianluca,
                   Eltawil Ahmed M., Michels Dominik L.},
  booktitle     = {11th Workshop on Computer Vision in Plant Phenotyping and
                   Agriculture (CVPPA), in conjunction with the European
                   Conference on Computer Vision (ECCV)},
  address       = {Malmo, Sweden},
  month         = sep,
  year          = {2026},
  eprint        = {2607.18576},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}

Acknowledgment

This work was supported by funding from King Abdullah University of Science and Technology (KAUST) - Center of Excellence on Sustainable Food Security, under award number 5934.

Contact

• —Samy Mounir, <samy.mounir@kaust.edu.sa>
• —Mikolaj Cieslak, <mikolaj.cieslak@kaust.edu.sa>
• —Najmeddine Dhieb, <najmeddine.dhieb@kaust.edu.sa>