CoolFace
Datasetpublic

cleanorlabs/cleanor-storage-lab

pretty_name: "Cleanor Storage Lab" license: cc-by-4.0 language: - en tags: - image-compression - avif - webp - jpeg-xl - heic - cloud-storage - benchmark - open-data size_categories: - n<1K configs: - config_name: compression-benchmark data_files: compression-benchmark.csv - config_name: heic-tax data_files: heic-tax-benchmark.csv - config_name: nextgen-formats data_files: nextgen-formats-benchmark.csv - config_name: cloud-price-index… See the full description on the dataset page: https://huggingface.co/datasets/cleanorlabs/cleanor-storage-lab.

sourceHugging Faceupdated 2mo agoView on Hugging Face
1likes115downloads
Dataset Card

<!-- Ready-to-use Hugging Face dataset card.

How to publish:

  1. 1.Create a dataset at https://huggingface.co/new-dataset (owner: cleanor, name: storage-lab)
  2. 2.Upload the CSVs from this repo's data/ folder.
  3. 3.Paste EVERYTHING below (including the --- YAML block) as the dataset's README.md. Keep this file in sync with the GitHub README when the data updates. --> --- pretty_name: "Cleanor Storage Lab" license: cc-by-4.0 language:
  4. 4.en tags:
  5. 5.image-compression
  6. 6.avif
  7. 7.webp
  8. 8.jpeg-xl
  9. 9.heic
  10. 10.cloud-storage
  11. 11.benchmark
  12. 12.open-data size_categories:
  13. 13.n<1K configs:
  14. 14.configname: compression-benchmark datafiles: compression-benchmark.csv
  15. 15.configname: heic-tax datafiles: heic-tax-benchmark.csv
  16. 16.configname: nextgen-formats datafiles: nextgen-formats-benchmark.csv
  17. 17.configname: cloud-price-index datafiles: cloud-price-index.csv
  18. 18.configname: photo-storage-capacity datafiles: photo-storage-capacity.csv
  19. 19.configname: video-codec-benchmark datafiles: video-codec-benchmark.csv
  20. 20.configname: video-codec-summary datafiles: video-codec-summary.csv
  21. 21.configname: app-size-index datafiles: app-size-index.csv
  22. 22.configname: storage-upgrade-tax datafiles: storage-upgrade-tax.csv
  23. 23.configname: gif-format-benchmark datafiles: gif-format-benchmark.csv
  24. 24.configname: lossless-recompression datafiles: lossless-recompression.csv
  25. 25.configname: font-format-weight datafiles: font-format-weight.csv
  26. 26.configname: svg-minify datafiles: svg-minify.csv
  27. 27.configname: pdf-compression datafiles: pdf-compression.csv ---

Cleanor Storage Lab

Reproducible benchmarks and open datasets on image compression (JPEG / WebP / AVIF / JPEG XL), the HEIC-to-JPG size tax, cloud storage price per GB, and phone storage capacity.

Every number is generated by a script you can re-run yourself. Image benchmarks run on the Kodak and public high-resolution phone-photo corpora, encoded with libvips/sharp, libjxl and libheif, and scored with SSIM + PSNR against the lossless master.

Highlights (quick answers)

  • —AVIF vs JPEG: at identical quality (SSIM 0.95), AVIF is ~41% smaller than JPEG; WebP ~13% smaller.
  • —HEIC to JPG at matched quality makes a photo corpus ~2.5x bigger (to PNG, ~5.5x) — the "HEIC tax".
  • —Xvid/DivX vs modern video codecs: Xvid needs ~4x the bitrate of H.264 and ~8.9x AV1 at the same quality.
  • —GIF vs MP4: the same clip is 44-91x smaller as an MP4 than a GIF.
  • —App sizes: the median popular iOS app is ~400 MB to download; TikTok is 936 MB.
  • —Storage upgrade tax: makers charge ~$0.78/GB to add storage, ~13x the ~$0.06/GB flash cost.
  • —Lossless recompression: oxipng shrinks a PNG ~10%, jpegtran a JPEG ~6%, bit-identical.
  • —Fonts: a WOFF2 is ~40% of the TTF; subset to Latin, ~11%.
  • —PDF: Ghostscript /ebook cuts ~70% (lossy, text kept); qpdf ~13% lossless.

Files

FileWhat it is
compression-benchmark.csvPer-image JPEG/WebP/AVIF size + SSIM/PSNR across quality levels
heic-tax-benchmark.csvHEIC vs matched JPG/PNG size at equal quality
nextgen-formats-benchmark.csvJPEG XL vs AVIF/WebP
cloud-price-index.csvConsumer cloud storage $/GB by provider and tier
photo-storage-capacity.csvHow many photos fit per GB at common resolutions
video-codec-benchmark.csv / video-codec-summary.csvXvid/DivX vs H.264/HEVC/AV1/VP9 bitrate at matched SSIM
app-size-index.csvApp Store download size of 40 popular iOS apps (date-stamped)
storage-upgrade-tax.csvWhat makers charge to add storage vs flash cost (5-13x markup)
gif-format-benchmark.csvSame clip as GIF/APNG/MP4/WebM: bytes + ratio vs GIF
lossless-recompression.csvLossless PNG/JPEG recompression savings (oxipng, jpegtran)
font-format-weight.csvTTF vs WOFF vs WOFF2 vs Latin-subset
svg-minify.csvSVGO + gzip savings on SVGs
pdf-compression.csvGhostscript (lossy) vs qpdf (lossless), with text-preservation check

Sources & docs

  • —Cite: DOI 10.5281/zenodo.21217372 (concept DOI, all versions)
  • —Code + harness: https://github.com/cleanor-app/cleanor-storage-lab
  • —Open datasets catalog: https://cleanor.app/research/open-datasets
  • —Human-readable studies with charts: https://cleanor.app/research

License

Data: CC BY 4.0 (attribution required). Code in the source repo: MIT.

Citation

Cleanor Labs. Storage Lab: Open Image-Compression & Cloud-Storage Datasets. 2026. https://cleanor.app/research