finedet/openimages

FineOpenimages — Open Images V7 boxed subset in the unified detection format

Source: official Open Images bbox CSVs + CVDF-hosted image tars (open-images-dataset S3 bucket).

Converted by the finedet project into a unified, AutoTrain-compatible layout: image / width / height / objects{bbox, category} with COCO-format [x, y, w, h] boxes in absolute pixels. Boxes are clipped to the image and empty boxes dropped; category ids are densified per the category tables below.

Box format

objects.bbox follows the COCO convention: [x, y, w, h] in absolute pixels, origin at the image's top-left corner.

<img src="assets/bbox_format.png" width="480"/>

License

Annotations: CC BY 4.0 (Google LLC). Images: listed as CC BY 2.0 individually; Google does not warrant the license status of each image. https://storage.googleapis.com/openimages/web/factsfigures_v7.html

Example images

Boxes are colored by category: near-transparent fill, opaque outline.

<table> <tr><td><img src="assets/sample0.jpg" width="360"/></td><td><img src="assets/sample1.jpg" width="360"/></td></tr> <tr><td><img src="assets/sample2.jpg" width="360"/></td><td><img src="assets/sample3.jpg" width="360"/></td></tr> </table>

Conversion notes

Only images with box annotations are included. Normalized XMin/XMax/YMin/YMax converted to absolute COCO xywh using the decoded image size. IsGroupOf boxes are kept. Attribute flags (occluded/truncated/depiction/inside) are not carried over.

Splits

• —test: 112194 images
• —train: 1743042 images
• —validation: 37306 images

Categories

This dataset has 601 categories. The full category table has moved to categories.csv (columns: id, original_id, name).

Training with transformers

The boxes are already in the absolute-pixel COCO [x, y, w, h] format that AutoImageProcessor expects, so fine-tuning a detector needs no bbox conversion:

import torch
from datasets import load_dataset
from transformers import (AutoImageProcessor, AutoModelForObjectDetection,
                          Trainer, TrainingArguments)

ds = load_dataset("finedet/openimages")
obj_feat = ds["train"].features["objects"]
if hasattr(obj_feat, "feature"):
    obj_feat = obj_feat.feature
cat_feat = obj_feat["category"]
names = (cat_feat.feature if hasattr(cat_feat, "feature") else cat_feat).names

checkpoint = "facebook/detr-resnet-50"
processor = AutoImageProcessor.from_pretrained(checkpoint)
model = AutoModelForObjectDetection.from_pretrained(
    checkpoint,
    id2label=dict(enumerate(names)),
    label2id={n: i for i, n in enumerate(names)},
    ignore_mismatched_sizes=True,
)


def transform(batch):
    images = [img.convert("RGB") for img in batch["image"]]
    annotations = [
        {"image_id": i,
         "annotations": [
             {"bbox": box, "category_id": cat, "area": box[2] * box[3], "iscrowd": 0}
             for box, cat in zip(objs["bbox"], objs["category"])
         ]}
        for i, objs in enumerate(batch["objects"])
    ]
    return processor(images=images, annotations=annotations, return_tensors="pt")


def collate(batch):
    return {"pixel_values": torch.stack([x["pixel_values"] for x in batch]),
            "labels": [x["labels"] for x in batch]}


trainer = Trainer(
    model=model,
    args=TrainingArguments(output_dir="out", per_device_train_batch_size=4,
                           num_train_epochs=10, learning_rate=1e-5,
                           remove_unused_columns=False),
    train_dataset=ds["train"].with_transform(transform),
    data_collator=collate,
)
trainer.train()