while-ai/course-refunds-grpo-1.5b

course-refunds-grpo-1.5b

Recipe: [recipes/04-train/grpo](https://github.com/whilehq/whileai-sdk/tree/main/recipes/04-train/grpo) · Collection: [Course and community runs](https://huggingface.co/collections/while-ai/course-and-community-runs-6ab271de189fd0c363cfab92)

The course's GRPO run: a 1.5B refunds agent trained on Modal with the dashboard watching. The reward is the recipe's executable refund environment; the holdout is split by scenario. holdout_before.jsonl and holdout_after.jsonl are every sampled row on both arms, so the before/after is recomputable from this repo alone.

What this is

The adapter the 40-step run usersim-grpo-40 left on the whileai-grpo-runs volume, with both holdout row files beside it. The recipe README explains the environment, why the reward is hackable and how the --balance flag closed the split it was hiding. Numbers for this exact run are in the two row files; the README's tables are from the runs it names.

Arms in this repo

The root holds the arm the recipe README's headline number reports. Every other arm is a subfolder named after it. checkpoints/ never ships.

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base, "while-ai/course-refunds-grpo-1.5b")  # the headline arm

Reproduce

git clone https://github.com/whilehq/whileai-sdk && cd whileai-sdk/recipes/04-train/grpo
modal run train_modal.py --steps 40

The recipe README pins the seed, the library versions and the GPU, and its Checks table says what the eval verified. Read the Learned section before quoting a number from this card.