Modelpublic

shrango/qwen3-4b-base-taco-grpo-lorem-shaping

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes15downloads

Model Card

Qwen3-4B-Base — TACO code RL (lorem_shaping)

RL-finetuned from Qwen/Qwen3-4B-Base with GRPO on the taco subset of agentica-org/DeepCoder-Preview-Dataset, using an execution-based reward (generated programs are run against the problem's stdin/stdout test cases).

Method: GRPO + LoPE lorem resampling + policy/advantage shaping

Results (pass@1, %)

Benchmark	This model	Qwen3-4B-Base (before RL)
HumanEval	81.71	76.83
MBPP	66.20	49.00
Average	73.96	62.91

All four method variants trained under identical settings:

Method	HumanEval	MBPP	Average
baseline (GRPO)	84.15	60.60	72.38
naive resample	84.76	57.40	71.08
lorem	85.98	66.00	75.99
lorem + shaping	81.71	66.20	73.96

Evaluated with evalscope, pass@1, single sample, temperature=0, served via vLLM. Note: training prompts asked the model to reason inside <think></think> tags, while the evaluation used the benchmarks' default (non-think) prompts.

Training setup

—Framework: verl / EasyR1 fork
—Steps: 174 (3 epochs over 7432 problems)
—Rollout: n=8 per prompt, batch size 128
—Max length: 2048 prompt / 8192 response
—KL: disabled
—Hardware: 8×A100 (single node)

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "shrango/qwen3-4b-base-taco-grpo-lorem-shaping"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")