CooperBench/qwen35-9b-leader-follower-coop

What this is

Cooperative two-agent coding dataset: 39 task pairs across 14 repos (random-50 subset), generated with mini_swe_agent on Qwen/Qwen3.5-9B in coop setting using a leader-follower prompt variant — one agent is designated as leader and sets the coordination strategy; the other acts as follower and adapts its implementation plan accordingly. All 39 pairs were successfully evaluated.

Notable: this variant produced the lowest merge conflict rate (17.9%) of all random-50 prompt variants, suggesting the asymmetric coordination structure reduces patch overlaps.

At a glance

How it was generated

cooperbench run --setting coop -a mini_swe_agent -c 30 qwen35-9b-leader-follower-coop

Model served via vLLM OpenAI-compatible endpoint (openai/Qwen/Qwen3.5-9B). Leader-follower variant: agent1 is prompted as leader (sets the coordination approach); agent2 is prompted as follower (adapts to the leader's strategy).

File layout

• —index.csv — one row per task pair; HF Dataset Viewer entry point
• —qwen35-9b-leader-follower-coop/coop/<repo>/<task_id>/<features>/ — raw per-pair artifacts: result.json, eval.json, agent1_traj.json, agent2_traj.json, agent{1,2}.patch, conversation.json

log_dir column in index.csv points to the per-pair subdirectory.

Schema highlights for mid-training

Filter on: both_passed=true, model, agent_framework.

metadata JSON carries per-agent statuses, steps, merge outcome, per-feature pass — use json.loads(row["metadata"]) without following the pointer.

Note: total_tokens is 0 for this run — token counts are in agent_full_traj.json under messages[*].extra.response.usage (~24.9M total in+out).

Caveats

• —random-50 subset — 39 tasks completed
• —15.4% LimitsExceeded exits (12/78 agent slots); 0 Error exits
• —17.9% merge conflict rate (7/39) — lowest of all random-50 prompt variants
• —Token fields in result.json are 0; aggregate from agent_full_traj.json if needed

Citation

@dataset{qwen35_9b_leader_follower_coop,
  title  = {qwen35-9b-leader-follower-coop},
  author = {Arya Prabhudesai},
  year   = {2026},
  url    = {https://huggingface.co/datasets/CooperBench/qwen35-9b-leader-follower-coop}
}