RL-Forgetting-Experiments-3/mbpp-code-rl

MBPP for code RL (deduplicated against MBPP+)

MBPP prepared for RLVR training in verl, with two independent hold-outs so both MBPP+ and MBPP's own canonical test split stay reportable after training on this data.

Prompt

Each row is a two-turn chat asking for reasoning inside an explicit <think></think> region, then a fenced Python block:

system: Please reason step by step inside a <think> </think> block, then put
        your final solution in a single ```python code block.
user:   <task>
        Your implementation should pass tests like the following ...
        First reason step by step inside <think> and </think>. Then write your
        complete solution in a single markdown fenced code block ...

The delimiter is deliberate. Without it, reasoning is unmarked prose and the only signal about it is total response length, which conflates "stopped reasoning" with "wrote shorter code". Measured on Qwen2.5-3B during RL on this data, mean response length fell from ~446 tokens at step 9 to 54 by step 95 while training reward held at 0.83: MBPP is easy enough that reasoning earns no extra reward, so GRPO strips it. The tag does not prevent that collapse, but it makes the think region separately measurable.

Half the tests (floor(N/2)) are shown so the model can infer the function name and signature; all tests are used for reward.

Why 320 and not 974

MBPP full is 974 problems across four canonical splits (prompt 10, test 500, validation 90, train 374). Two things are removed from the training pool:

1. 1.Everything in MBPP+ (378). MBPP+ is derived from MBPP-sanitized and consumes 378 of that config's 427 problems.
2. 2.MBPP's canonical test split. Deduplicating against MBPP+ alone would leave 596 training problems, but 276 of those are canonical MBPP test -- fine if MBPP+ is your only benchmark, fatal if you ever want standard MBPP numbers.

Deduplicating MBPP-sanitized against MBPP+ leaves only 49 problems, which is why the sanitized config alone is not viable for training. Each problem here uses its sanitized text and corrected tests where one exists (427 of 974), keeping the other 547 -- sanitized quality at usable size.

Leakage is asserted at build time: no train task_id appears in MBPP+, and no train row comes from a withheld split.

Format

verl RLHF layout: data_source, prompt, ability, reward_model, extra_info.

reward_model.ground_truth is a JSON string:

{"assert_case": ["assert floor_Min(10,20,30) == 15", "..."]}

The verifier dispatches on the keys of that dict, so one reward path handles MBPP asserts and competitive-programming stdin/stdout with no dataset-specific routing. Setup imports (test_setup_code / test_imports) are folded into each assert string, because every test runs as a standalone script: solution + "\n" + assert_case[i].

Validation

All 974 MBPP reference solutions score 1.0 through the verifier.

Sources

• —`google-research-datasets/mbpp` (configs full and sanitized)
• —`evalplus/mbppplus`