anonymous-neurips2026-0383/example_difficulty_scores

Precomputed scores from Dataset Difficulty and the Role of Inductive Bias

• —checkpoints end with .pt
• —all scores are saved in .npz files
• —scores with variances computed over training are stored under the keys 'mean' and 'var', otherwise they are under the key 'arr_0'
• —precomputed contains scores from Carlini, Erlingsson & Papernot (2019) [1], Feldman & Zhang (2020) [2], and Sorscher et al. (2022) [3]. This directory can be placed in src/difficulty if using code from nn_example_difficulty to read them
• —precomputed scores from Feldman & Zhang (2020) [2] are downloaded from https://pluskid.github.io/influence-memorization/
• —the SwAV scores from Sorscher et al. (2022) are reproduced using https://github.com/facebookresearch/ and the pretrained checkpoint at https://dl.fbaipublicfiles.com/deepcluster/swav_800ep_pretrain.pth.tar.
• —precomputed/imagenet_tr_labels.npz stores the data order and class order for ImageNet. ImageNet images are sorted by filename, and classes are sorted by wordnet ID
• —CIFAR-10 and CIFAR-100 images and labels are in the originally published order
• —The checkpoints used to compute cifar10_vit, cifar100_vit, and imagenet_vit are listed in selected_vit_models.csv, and are downloaded from https://huggingface.co/laion/scaling-laws-for-comparison, which is described in Nezhurina et al. (2025) [4]

[1] Carlini, N., Erlingsson, U., & Papernot, N. (2019). Distribution density, tails, and outliers in machine learning: Metrics and applications. arXiv preprint arXiv:1910.13427.

[2] Feldman, V., & Zhang, C. (2020). What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in neural information processing systems, 33, 2881-2891.

[3] Sorscher, B., Geirhos, R., Shekhar, S., Ganguli, S., & Morcos, A. (2022). Beyond neural scaling laws: beating power law scaling via data pruning. Advances in Neural Information Processing Systems, 35, 19523-19536.

[4] Nezhurina, M., Porian, T., Puccetti, G., Kerssies, T., Beaumont, R., Cherti, M., & Jitsev, J. Scaling Laws for Robust Comparison of Open Foundation Language-Vision Models and Datasets. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.