Datasetpublic

khashazad/amc-ruler-qwen35-32k

AMC RULER 32k This dataset contains frozen inputs for the RULER benchmark. Generation metadata Benchmark: RULER Sequence length: 32,768 tokens Tokenizer: Qwen/Qwen3.5-9B Tokenizer revision: c202236 lm-eval version: 0.4.12 Task configurations: 13 Samples per configuration: 500 Deterministic generation: Yes. Each configuration resets Python, NumPy, and task random state to seed 42. Task configurations niah_single_1 niah_single_2 niah_single_3… See the full description on the dataset page: https://huggingface.co/datasets/khashazad/amc-ruler-qwen35-32k.

sourceHugging Faceupdated 1mo agoView on Hugging Face

0likes227downloads

Dataset Card

AMC RULER 32k

This dataset contains frozen inputs for the RULER benchmark.

Generation metadata

—Benchmark: RULER
—Sequence length: 32,768 tokens
—Tokenizer: Qwen/Qwen3.5-9B
—Tokenizer revision: c202236
—lm-eval version: 0.4.12
—Task configurations: 13
—Samples per configuration: 500
—Deterministic generation: Yes. Each configuration resets Python, NumPy, and task random state to seed 42.

Task configurations

—niah_single_1
—niah_single_2
—niah_single_3
—niah_multikey_1
—niah_multikey_2
—niah_multikey_3
—niah_multiquery
—niah_multivalue
—ruler_vt
—ruler_cwe
—ruler_fwe
—ruler_qa_squad
—ruler_qa_hotpot

[!WARNING] This dataset is tokenizer-specific. Sequence boundaries and generated samples are coupled to the linked tokenizer and exact revision. Use that tokenizer revision for evaluation.