AlexChen1997/fedagent-webshop-grpo-hardness-std1-qwen2.5-1.5b
FedAgent WebShop agent — Qwen2.5-1.5B · GRPO · hardness-partition std-1 (federated, 70 rounds)
WebShop shopping-agent policy trained with federated GRPO on the FedAgent verl-0.8 stack. This repo holds the final aggregated global model (round 70) at the root, and the best-validation aggregate (round 68) under checkpoints/round_68/.
Training setup
- Base model: Qwen/Qwen2.5-1.5B-Instruct; weights here are full fp32 HF exports (actor only — GRPO has no critic)
- Algorithm: GRPO,
rollout.n=8, train_batch 8, prompt/response 4096/512, fused kernels (triton) - Federation: 100 clients, 2 sampled per round, 70 rounds, 3 local epochs/round, FedAvg over actor weights,
base_seed 42(fully deterministic schedule) - Task heterogeneity: hardness partition, success_std = 1 — each client's WebShop goal pool is stratified by a trained-reference success signal (≥100 goals/client)
- Environment: vendored WebShop engine, BM25 search (
search_return_n=50), 16-env pools per client, unperturbed shared val service - Hardware / wall-clock: 4× H100-80GB single node, ~16.5 min/round, ~19 h total
- Config: run copy of
config/paper_accelerated/task_heterogeneity/grpo/webshop/fed_webshop_grpo_total-100_cl-per-rd-2_rd-70_ep-per-cl-3_min-goals-per-cl-100_p-hardness_success_std-1.yaml(science knobs untouched)
The full run record — per-round validation dumps, training logs, curves, and the official federated_summary.json (71 global val points + 140 per-client circles) — is published separately as a dataset: `canyuchen/fedagent-webshop-grpo-hardness-std1`.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "canyuchen/fedagent-webshop-grpo-hardness-std1-qwen2.5-1.5b"
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained(repo)
# best-val checkpoint instead of the final one:
# model = AutoModelForCausalLM.from_pretrained(repo, subfolder="checkpoints/round_68", ...)The agent expects WebShop-style observation/action prompting (as produced by the FedAgent rollout loop); it is not a general chat model, though the chat template is included.
Note on `chat_template`: verl's modelmerger drops the tokenizer's `chattemplate when exporting; it has been **re-injected from the base Qwen tokenizer** here, so tok.applychattemplate(...)` works out of the box. If you re-export checkpoints from the raw run artifacts yourself, remember to do the same.
Provenance
- Trained 2026-07-23 → 2026-07-24 (fresh run, no prior checkpoints), FedAgent verl-0.8 overlay
- Companion PPO twin (same partition/protocol) reached only 14.1 % best success by round 34 — GRPO dominates at every matched round
- fp32 shards, ~5.8 GB per checkpoint;
model.safetensors.index.jsonincluded