yuyangbai/graphdancer-dpo-iter1-seed167

GraphDancer-DPO Iter-1 (seed 167) — best single-seed Rouge-L

This is the highest-Rouge-L single-seed result from the GraphDancer iter-1 DPO refinement of yuyangbai/GraphDancer-Qwen2.5-3B-Instruct-Curriculum-PPO.

Headline numbers (full GRBench eval, 890 queries, T=0.7 single-sample)

vs paper PPO baseline 0.4062 → +0.0377 R-L AVG (single seed).

Distribution context (across 11 seeds with the same recipe)

• —All-11 mean R-L AVG: 0.4265 ± 0.0079
• —This seed (167) is the maximum (0.4439); seed 109 is the minimum (0.4156).
• —1-sample t-test of all-11 mean vs paper-PPO 0.4062: t=8.48, df=10, p<0.001.

Recipe

• —Base = π_ref = yuyangbai/GraphDancer-Qwen2.5-3B-Instruct-Curriculum-PPO
• —Phase 0: K=8 trajectories sampled from πref (verl + vLLM, T=1.0, topp=0.95, max_turns=10)
• —Pair builder: lex tuple v1 = (em, eh, vf, -loop, -invalid_tool, -n_graph_rounds), strategy extreme
• —Trainer: TRL 0.19.1 DPOTrainer (subclass with agent-token mask), β=0.1, lr=2e-7, 100 steps, global batch 64, warmup 0.05
• —Curriculum: paper-faithful E2HGaussian (q=[0.5,0.5,0], η 0.2→0.8, β=3, σ=0.75)
• —Seed: 167

See dpo_iter1_full_config.md and dpo_iter1_top5_summary.md in the GraphDancer-Codebases repo for full details and the 11-seed distribution.