fxevangelinenyu/rl-forgetting-math-benchmarks

RL-Forgetting math benchmarks

Train and test/benchmark sets used in the RL-Forgetting-Exp study of replay-buffer freshness. All parquets share the verl RL schema (data_source, prompt, ability, reward_model, extra_info).

Layout

polaris_full/
    train.parquet    # 52,309 prompts (Polaris-full training set)
    test.parquet     #    800 prompts (held-out test, 100/difficulty)
deepscaler/
    train.parquet    #  8,192 prompts (skywork_deepscaler_easy_8192, fixed ground_truth)
    test/            # external math benchmarks used for pass@k eval
        aime2024.parquet      #  30
        aime2025.parquet      #  30
        amc_test.parquet      #  83
        math500.parquet       # 500
        minerva_math.parquet  # 272

Notes

• —polaris_full is the Polaris-53k dataset; train.parquet is the RL training split and test.parquet is the sampled held-out test set (100 prompts per difficulty bucket).
• —deepscaler/train.parquet is the fixed-ground-truth variant of skywork_deepscaler_easy_8192 used for RL training.
• —The DeepScaleR "test set" is the standard suite of external math benchmarks (AIME 2024/2025, AMC, MATH-500, Minerva-Math) rendered in the same prompt template as training, used to measure pass@k.