k3999/multilabel-imagenet-1k

MultiLabel ImageNet-1K Train Annotations with Selected Masks

This dataset contains automated multi-label annotations for the ImageNet-1K training split, together with spatial masks for the selected object-level labels.

The release is designed to make the annotations easy to inspect and reuse. It does not include the original ImageNet images. Users need access to the ImageNet-1K training images separately; image paths are stored relative to the ImageNet train root, for example:

n01440764/n01440764_10026.JPEG

What Is Included

• —image_labels.parquet: one row per ImageNet-1K training image.
• —mask_labels.parquet: at most one selected mask per (image, positive label).
• —metadata.json: source/config metadata and the selection rule.

Dataset size:

• —Images: 1,281,167
• —Selected mask-label rows: 2,193,717

Label And Mask Selection

The image-level labels come from the compressed annotation TSV used in the project release. For each image, the label vector is sparse: only positive class indices and their probabilities are stored.

For each positive label in an image, we select at most one mask:

1. 1.Collect candidate masks from four MaskCut proposal configurations.
2. 2.Keep candidates whose localized labeler prediction matches the positive label.
3. 3.Select the mask with the highest probability for that label.
4. 4.Store that selected mask's full top-5 class predictions.

This means mask_labels.parquet has no duplicate (filename, label) pairs.

The exported masks are generated from MaskCut proposals and labeled by a trained region classifier.

Files

image_labels.parquet

mask_labels.parquet

Loading

Read the two tables directly with datasets, pandas, or pyarrow.

from datasets import load_dataset

repo_id = "YOUR_USERNAME/YOUR_DATASET_NAME"

image_labels = load_dataset(
    "parquet",
    data_files=f"hf://datasets/{repo_id}/image_labels.parquet",
    split="train",
)

mask_labels = load_dataset(
    "parquet",
    data_files=f"hf://datasets/{repo_id}/mask_labels.parquet",
    split="train",
)

Or with pandas:

import pandas as pd

image_labels = pd.read_parquet("image_labels.parquet")
mask_labels = pd.read_parquet("mask_labels.parquet")

Decoding A Mask

from pathlib import Path

import numpy as np
from PIL import Image
from pycocotools import mask as mask_utils

imagenet_train_root = Path("/path/to/imagenet/train")

row = mask_labels[0]
image = Image.open(imagenet_train_root / row["filename"]).convert("RGB")

rle = {
    "size": row["segmentation_rle_size"],
    "counts": row["segmentation_rle_counts"].encode("ascii"),
}
mask = mask_utils.decode(rle).astype(bool)

Reconstructing Sparse Image Labels From Masks

For mask-grounded labels, the image-level probabilities can be reconstructed by taking the maximum selected-mask probability per (filename, label) and rounding to four decimals. Some image-level labels may be present without a selected mask, so use image_labels.parquet as the authoritative image-level annotation table.

from collections import defaultdict

by_image = defaultdict(dict)
for row in mask_labels:
    by_image[row["filename"]][row["label"]] = max(
        by_image[row["filename"]].get(row["label"], 0.0),
        row["prob"],
    )

Source Configurations

The selected masks come from four MaskCut proposal configurations:

• —dinov2_vitg_s672
• —dinov3_vitb16_v_768
• —dinov1_vits_s480
• —dinov2_vitl_s448

See metadata.json for the exact local source paths used during export.

Intended Use

This dataset is intended for research on multi-label ImageNet training, region-grounded labels, object-centric supervision, and analysis of ImageNet's multi-object nature.

Limitations

• —The annotations are automatically generated and may contain mistakes.
• —Masks are proposal masks, not human-annotated segmentation masks.
• —ImageNet access and use are governed by ImageNet's own terms.

Citation

If you use this dataset, please cite:

@article{chen2026multilabel_imagenet,
  title   = {Unlocking ImageNet's Multi-Object Nature: Automated Large-Scale Multilabel Annotation},
  author  = {Chen, Junyu and Harun, Md Yousuf and Kanan, Christopher},
  journal = {arXiv preprint arXiv:2603.05729},
  year    = {2026},
  url     = {https://arxiv.org/abs/2603.05729}
}