twistshan/realistic-niah-count-empirical-law

Realistic NIAH Count Empirical Law V3.1 This private dataset is the frozen stimulus bank for the preregistered V3.1 realistic needle-in-a-haystack counting experiment. Registered grid Passage lengths: 1K, 2K, 3K, 5K, 8K, 10K, 15K, and 20K canonical tokens. Needle counts: 1-10, 12, 15, 18, and 20. Paired seeds: 1234-1263 (30 seeds per cell). Total: 112 N x L cells and 3,360 unique stimuli. The stimulus bank is shared across all registered models and prompt modes.… See the full description on the dataset page: https://huggingface.co/datasets/twistshan/realistic-niah-count-empirical-law.