CoolFace
Modelpublic

budget-internalization-iclr2027/qwen3.5-4b-4k-grpo-livecodebench-crackgannet-s240

sourceHugging Faceapache-2.0updated 2d agoView on Hugging Face
0likes9downloads
Model Card

Qwen3.5-4B · 4k-token budget · grpo-livecodebench · crackgannet

RL-finetuned Qwen/Qwen3.5-4B trained with GRPO on LiveCodeBench (v1) coding problems under a 4,096-token generation budget. Released as part of an anonymous ICLR 2027 submission.

Run ID (petname): `crackgannet` · checkpoint step 240

Training

Base modelQwen/Qwen3.5-4B
AlgorithmGRPO (leave-one-out baseline, group reward normalization, token-level loss). Responses that hit the budget receive zero reward.
Generation budget (max_new_tokens)4,096
DataLiveCodeBench v1 problems, 10 epochs max
Batch8 prompts × 8 rollouts per step
OptimizerAdam, cosine LR schedule, peak LR 5e-7, 8 warmup steps
Steps240
Rewardgenerated program passes the problem's tests
Weights dtypeBF16

Prompts are the LiveCodeBench (v1) coding problems task prompts, rendered with the base model's chat template.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "budget-internalization-iclr2027/qwen3.5-4b-4k-grpo-livecodebench-crackgannet-s240"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")

With vLLM: vllm serve budget-internalization-iclr2027/qwen3.5-4b-4k-grpo-livecodebench-crackgannet-s240

License

Inherits the license of the base model (Qwen/Qwen3.5-4B).