xxccho/gsm8k_rmbench_style

GSM8K-RMBench-Style — Style-controlled (correct, incorrect) variants on GSM8K

Per GSM8K problem this dataset provides 6 response surfaces — correct and incorrect each rendered in markdown / normal / concise styles — to support RM-Bench-style 3 × 3 (chosen × rejected) pair-grid evaluation and forget-LoRA training for Reward Model debiasing.

{ question, gold }
  ├── correct   : { markdown, normal, concise }
  └── incorrect : { markdown, normal, concise }

These 6 surfaces yield 9 pairwise comparisons (one per cell of the 3 × 3 grid), classified as Easy (chosen fancier), Normal (same fanciness), or Hard (rejected fancier) — the same convention as RM-Bench.

Splits

Post-filter (clean) versions only. Filter drops ~2.5% of records that exhibit unit-conversion-equivalent incorrect answers or hidden dual-path narration. See "Filtering" below.

Schema

Each row:

{
  "id": "gsm8k_train_03104",            // gsm8k_<split>_<source-index>
  "source_index": 3104,                  // index into the upstream GSM8K split
  "question": "...",                     // GSM8K problem text
  "gold_answer": 18.0,                   // numeric gold answer
  "gold_answer_text": "She has ...\n#### 18",
  "correct": {
    "markdown": "...",                   // Markdown-formatted correct solution
    "normal":   "...",                   // Plain-prose correct solution
    "concise":  "..."                    // 1-2 sentence correct solution
  },
  "incorrect": {
    "markdown": "...",                   // Markdown-formatted plausible wrong solution
    "normal":   "...",
    "concise":  "..."
  },
  "meta": {
    "incorrect_source": "fallback",      // 'main_filter' (natural) or 'fallback'
    "correct_boxed": 18.0,
    "incorrect_boxed": 24.0,
    "k_samples": 3,                       // K for natural-failure attempt
    "n_correct_samples": 3,
    "n_incorrect_samples": 1,
    "lengths": {
      "correct":   {"markdown": 260, "normal": 198, "concise": 65},
      "incorrect": {"markdown": 270, "normal": 195, "concise": 70}
    },
    "generator_model": "openai/gpt-5-nano-2025-08-07"
  }
}

Every text field contains exactly one \boxed{N} final answer.

Construction

Generator: `gpt-5-nano-2025-08-07` (LightLLM gateway), K=3 main samples per problem with a fallback prompt for cases where all K were correct (typical since gpt-5-nano solves GSM8K-easy with ~96% accuracy).

Pipeline stages per problem:

1. 1.K markdown solutions via MARKDOWN_CORRECT_PROMPT → split by \boxed{} extraction + gold match.
2. 2.If no natural incorrect → fallback MARKDOWN_WRONG_PROMPT (up to 4 retries) with strict narration-word filter.
3. 3.Length-matched correct selection: among K candidates, pick the one whose markdown length is closest to the incorrect markdown — removes length-confound between chosen and rejected.
4. 4.Variant transforms with up to 3 retries each (temperature 0.2 → 0.1 → 0.0):
5. 5.MARKDOWN_TO_NORMAL (correct & incorrect)
6. 6.NORMAL_TO_CONCISE_CORRECT and NORMAL_TO_CONCISE_INCORRECT (different rules to preserve flawed reasoning on the incorrect side).
7. 7.Validation gates: single \boxed{} per surface, factuality preserved, length order concise < normal ≤ markdown (±15 chars tolerance), no dual-path narration.

Filtering (post-hoc)

Two patterns that pass runtime gates but contaminate the bias signal are removed:

Dropped IDs are not included here. The raw pre-filter JSONL is preserved locally by the generator for reproducibility.

Usage example

from datasets import load_dataset

ds = load_dataset("xxcho/gsm8k_rmbench_style")
print(ds["train"][0]["correct"]["markdown"][:200])

# Extract a Hard-cell pair: chosen=plain correct, rejected=stylish incorrect
def hard_pair(r):
    return {
        "question": r["question"],
        "chosen":   r["correct"]["concise"],     # plain
        "rejected": r["incorrect"]["markdown"],  # stylish
        "gold":     r["gold_answer"],
    }

hard_pairs = ds["train"].map(hard_pair, remove_columns=ds["train"].column_names)

Intended use

• —Reward Model evaluation under RM-Bench's Easy/Normal/Hard convention.
• —Forget-LoRA / task arithmetic debiasing — supports style-only (same correctness, different style), correctness-only (same style, different correctness), and cross-axis (style ↔ correctness conflict) pair constructions.
• —Style-bias probing of RM internals (factuality vs style direction decomposition).

Limitations

• —Generator is a single model (gpt-5-nano). Reasoning patterns reflect one model's failure mode distribution.
• —GSM8K-easy domain only. Generalization to MATH / AIME / non-numeric tasks not validated.
• —~2% residual noise (unit-conv / alt-framing) acknowledged above; for high-purity downstream needs, additional manual review recommended.

Citation

If you use this dataset, please cite both GSM8K and RM-Bench:

@article{cobbe2021gsm8k,
  title={Training Verifiers to Solve Math Word Problems},
  author={Cobbe, Karl and Kosaraju, Vineet and Bavarian, Mohammad and others},
  journal={arXiv preprint arXiv:2110.14168}, year={2021}
}
@article{liu2024rmbench,
  title={RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Style},
  author={Liu, Yantao and Yao, Zijun and others},
  journal={arXiv preprint arXiv:2410.16184}, year={2024}
}