datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
wds-dataset-viewer-testRead from the webdataset (after saving it somewhere on your disk) like this:
from webdataset import WebDataset
from typing import TypedDict, Iterable
from PIL import Image
from PIL.PngImagePlugin import PngImageFile
from io import BytesIO
from os import makedirs
Example = TypedDict('Example', {
'__key__': str,
'__url__': str,
'img.png': bytes,
})
dataset = WebDataset('./wds-dataset-viewer-test/{00000..00001}.tar')
out_root = 'out'
makedirs(out_root, exist_ok=True)
it:… See the full description on the dataset page: https://huggingface.co/datasets/Birchlabs/wds-dataset-viewer-test.test-dataset-viewer