Datasetpublic

vrachit/imagenet-1k-webdataset

ImageNet-1k WebDataset This dataset contains ImageNet-1k in WebDataset format (tar files) for efficient streaming. Dataset Structure Training: 129 shards (train-*.tar) Validation: 5 shards (validation-*.tar) Total size: 147.82 GB Format Each tar file contains samples with: *.jpg: Image bytes *.cls: Label (class ID as text) Usage import webdataset as wds # Training dataset train_url = "train-{000000..000000000}.tar" dataset =… See the full description on the dataset page: https://huggingface.co/datasets/vrachit/imagenet-1k-webdataset.

sourceHugging Faceupdated 9mo agoView on Hugging Face

0likes38downloads

Dataset Card

ImageNet-1k WebDataset

This dataset contains ImageNet-1k in WebDataset format (tar files) for efficient streaming.

Dataset Structure

—Training: 129 shards (train-*.tar)
—Validation: 5 shards (validation-*.tar)
—Total size: 147.82 GB

Format

Each tar file contains samples with:

—*.jpg: Image bytes
—*.cls: Label (class ID as text)

Usage

python

import webdataset as wds

# Training dataset
train_url = "train-{000000..000000000}.tar"
dataset = wds.WebDataset(train_url).shuffle(1000).decode("rgb")

for sample in dataset:
    image = sample["jpg"]  # PIL Image
    label = int(sample["cls"])
    # ... your training code

Source

Converted from ILSVRC/imagenet-1k parquet format.

Generated using MetaDistil ImageNet Pipeline