davanstrien/viewer-test-15-large-images-ok
Test Dataset: viewer-test-15-large-images-ok Purpose: Demonstrate dataset viewer behavior with large images and row group sizing. Dataset Details Number of images: 15 Total size: ~188MB Average per image: ~12.6MB Expected Behavior 15 large images (~14MB each, ~210MB total). Under the 300MB limit, so the viewer should work. Prediction: SHOULD WORK - Under 300MB limit Technical Context The dataset viewer converts imagefolder datasets… See the full description on the dataset page: https://huggingface.co/datasets/davanstrien/viewer-test-15-large-images-ok.
Test Dataset: viewer-test-15-large-images-ok
Purpose: Demonstrate dataset viewer behavior with large images and row group sizing.
Dataset Details
- Number of images: 15
- Total size: ~188MB
- Average per image: ~12.6MB
Expected Behavior
15 large images (~14MB each, ~210MB total). Under the 300MB limit, so the viewer should work.
Prediction: SHOULD WORK - Under 300MB limit
Technical Context
The dataset viewer converts imagefolder datasets to parquet with:
- Row group size: 100 rows (for image datasets)
- Scan size limit: 300MB
If all images end up in one row group and exceed 300MB, the viewer fails with:
Scan size limit exceeded: attempted to read X bytes, limit is 300000000 bytesRelated Issue
This dataset was created to help debug row group sizing for large images. See: https://github.com/huggingface/dataset-viewer for context.