shawnmkim/qwen2.5-coder-7b-rlvr-compete-collab

Qwen2.5-Coder-7B — RLVR (GRPO) LoRA Adapter — Compete-then-Collaborate

LoRA adapter for Qwen/Qwen2.5-Coder-7B, trained with GRPO (verifiable-reward RL) on execution-verified competition problems. This is the final 1000-step checkpoint reported in the paper:

Compete then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation — Kim, 2026.

• —📄 Paper / code / harness: https://github.com/shawnkim678/compete-then-collaborate
• —📊 Companion dataset: `shawnmkim/compete-collab-taskbank`

Headline result (paper §5.3)

Held-out code_contests d6–9, 68 problems, execution pass@1:

Same data, opposite direction from SFT.

Learning curve:

Differences within the 250–750 plateau are ±1 problem out of 68 (sampling noise); the robust signal is base → RLVR (4 → 5–6 / 68). We recommend checkpoint selection over final-step training.

Ethics / ToS statement (important)

This RLVR adapter is trained with NO teacher-output distillation. The reward signal is deterministic execution against hidden tests on publicly-licensed competition problems (code_contests, CC-BY-4.0). No raw outputs from Claude / GPT / Grok / Gemini were used as training targets.

This places the artifact outside the competition-scope restrictions of Anthropic / OpenAI / Google-Gemini / xAI (the paper only ranks those teachers as a scientific benchmark; it does not distill them to build a competitor).

Quick start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE  = "Qwen/Qwen2.5-Coder-7B"
LORA  = "shawnmkim/qwen2.5-coder-7b-rlvr-compete-collab"

tok   = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, LORA)
model.eval()

prompt = "Write a Python function that returns the n-th Fibonacci number."
inputs = tok(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0], skip_special_tokens=True))

Training details

• —Base model: Qwen/Qwen2.5-Coder-7B
• —Method: GRPO (Group Relative Policy Optimization, DeepSeek-Math)
• —RL library: TRL 0.24.0 (GRPOTrainer), PEFT 0.19.1
• —PEFT config (LoRA):
• —rank r=32, alpha=64, dropout 0
• —target modules: q_proj, k_proj, v_proj, o_proj, up_proj, down_proj, gate_proj
• —bias none, task type CAUSAL_LM
• —Reward: test_pass_fraction + small_format_bonus
• —Rollout: HF-generate (vLLM incompatible on this stack)
• —Steps: 1000 (peak plateau 250–750)
• —Hardware: NVIDIA DGX Spark GB10 (128 GB unified memory)
• —Frameworks: transformers 5.5 · torch 2.11 · cu130

Reproducing the results

git clone https://github.com/shawnkim678/compete-then-collaborate
cd compete-then-collaborate
python reproduce.py --check-banks --selftest
python scripts/eval_code_students.py \
    --base "Qwen/Qwen2.5-Coder-7B" \
    --adapter "shawnmkim/qwen2.5-coder-7b-rlvr-compete-collab" \
    --bank data/taskbank_contests_heldout.jsonl \
    --label "grpo-7B [contests]"

Files in this repository

• —adapter_config.json — PEFT LoRA config (rank 32)
• —adapter_model.safetensors — LoRA weights (~323 MB)
• —chat_template.jinja — chat template inherited from base
• —tokenizer.json / tokenizer_config.json — tokenizer copies

Citation

@misc{kim2026compete,
  title  = {Compete then Collaborate: Frontier AI Teachers Build a Verifiable Curriculum to Improve a Coding Student Beyond Imitation},
  author = {Kim, Miseong (Shawn)},
  year   = {2026},
  note   = {Preprint, Genesis Cortex AI Inc.},
  url    = {https://github.com/shawnkim678/compete-then-collaborate}
}

Cite GRPO (method) and TRL (implementation):

@article{shao2024deepseekmath,
  title  = {DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models},
  author = {Shao, Zhihong and Wang, Peiyi and Zhu, Qihao and others},
  year   = {2024},
  eprint = {arXiv:2402.03300}
}

@misc{vonwerra2022trl,
  title  = {{TRL: Transformer Reinforcement Learning}},
  author = {von Werra, Leandro and Belkada, Younes and others},
  year   = {2020},
  url    = {https://github.com/huggingface/trl}
}

Acknowledgments

Following the accompanying paper, we thank Dr. Min Jang (POSTECH) — whose 1999 dissertation on ensemble learning with virtual data provided the conceptual lineage for the verifiable-curriculum framing — for reviewing an earlier draft and highlighting the engineering contributions of the reported system as its principal strengths (GB10 stack debugging, four-provider CLI orchestration, execution sandbox).