RyanYr/dapo-math-17k-drop-never-solved-qwen3-1.7b-base-k16

DAPO-Math-17k, never-solved problems dropped (Qwen3-1.7B-Base, K=16)

The 7,434 problems of the DAPO-Math-17k train set that Qwen3-1.7B-Base solved at least once in K=16 samples — the other 9,964 of 17,398 problems (57.27%), solved 0/16 times, are dropped.

In group-relative RL a prompt group whose K responses all score the same has a reward advantage of exactly zero and contributes no gradient. A problem the starting policy solves 0/16 times can only produce such groups, so every rollout spent on it is wasted. This dataset removes exactly those problems and nothing else. There was nothing to drop on the easy side: not one problem was solved 16/16 (the best is 14/16).

Contents

One parquet, train-00000-of-00001.parquet, 7,434 rows, two string columns — the same schema as the training taskset it replaces:

Row order is the (shuffled) order of the taskset the training runs read (train_ds_shuf), with dropped rows removed — filtering preserved order, so this is a drop-in replacement for that taskset path.

How it was built

Per-problem difficulty comes from the sibling dataset `RyanYr/dapo-math-17k-difficulty-qwen3-1.7b-base-k16` (K=16 samples per problem, temperature 1.0, max 4096 new tokens, scored with the same compute_score the training reward uses; see that repo for the full probe setup). The join key is

hashlib.sha1(problem.encode("utf-8")).hexdigest()[:16]

computed from the problem column. The join was verified total before filtering: all 17,398 taskset rows matched exactly one difficulty record and all 17,398 difficulty keys matched exactly one row (the probe's source parquet and the shuffled taskset were confirmed to hold the same (problem, answer) multiset, differing only in row order). Kept: n_correct >= 1.

Built by analysis/difficulty/build_trainset_drop_never_solved.py in the async-rl repo, which re-runs all of these checks and refuses to write on any mismatch.

Difficulty of what remains

Every problem here can produce a within-group reward difference at the probe's K=16; the set remains hard (mean base-model accuracy over the kept problems is ~0.176 at K=16, temperature 1.0).

Caveats

• —The filter is model- and probe-specific: "never solved" means by Qwen3-1.7B-Base under that one sampling configuration. For a different starting policy, re-probe rather than reuse.
• —K=16 estimates each problem's solve rate with a standard error of up to 0.125, so some kept problems are near-zero and some dropped ones would occasionally be solved. The 0/16 boundary is sharp; the underlying difficulty is not.
• —Problems are from BytedTsinghua-SIA/DAPO-Math-17k (Apache-2.0), redistributed here in filtered form.