CoolFace
Modelpublic

budget-internalization-iclr2027/gemma4-e2b-2k-grpo-reasoninggym-smartcamel-s300

sourceHugging Faceapache-2.0updated 2d agoView on Hugging Face
0likes11downloads
Model Card

Gemma-4-E2B-it · 2k-token budget · grpo-reasoninggym · smartcamel

RL-finetuned google/gemma-4-E2B-it trained with GRPO on Reasoning Gym tasks under a 2,048-token generation budget. Released as part of an anonymous ICLR 2027 submission.

Run ID (petname): `smartcamel` · checkpoint step 300

Training

Base modelgoogle/gemma-4-E2B-it
AlgorithmGRPO (leave-one-out baseline, group reward normalization, token-level loss). Responses that hit the budget receive zero reward.
Generation budget (max_new_tokens)2,048
DataReasoning Gym tasks (procedurally generated reasoning problems, \boxed{} answers), 3 epochs max
Batch32 prompts × 8 rollouts per step
OptimizerAdam, cosine LR schedule, peak LR 3e-6, 10 warmup steps
Steps300
RewardReasoning Gym task verifier on the \boxed{} answer
Weights dtypeF32

Prompts are the Reasoning Gym task prompts, rendered with the base model's chat template.

Usage

python
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "budget-internalization-iclr2027/gemma4-e2b-2k-grpo-reasoninggym-smartcamel-s300"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="auto", device_map="auto")

With vLLM: vllm serve budget-internalization-iclr2027/gemma4-e2b-2k-grpo-reasoninggym-smartcamel-s300

License

Inherits the license of the base model (google/gemma-4-E2B-it).