Modelpublic

arvindcr4/tinker-rl-scale_gsm8k_qwen3-8b-qwen3-8b

sourceHugging Faceapache-2.0updated 5mo agoView on Hugging Face

0likes

Model Card

tinker-rl-scalegsm8kqwen3-8b-qwen3-8b

LoRA adapters trained with GRPO on top of Qwen/Qwen3-8B using the Tinker cloud training service. Part of the TinkerRL-Bench release for our NeurIPS submission "A Unified Benchmark for RL Post-Training of Language Models" (repo).

Training configuration


Base model	`Qwen/Qwen3-8B`
Experiment tag	`scale_gsm8k_qwen3-8b`
Campaign	`None`
Task	`gsm8k`
Seed	`42`
LoRA rank	`32`
Learning rate	`3e-05`
Group size	`8`
Training steps	`30`
Platform	Tinker (`tinker`)
Training run ID	`3154dee5-4fd7-59b6-ba3b-721eef675bfc`

Metrics

Metric	Value
First-5 reward avg	0.3125
Last-10 reward avg	0.34375
Peak reward	0.625
Peak accuracy	0.625
Last-10 accuracy	0.34375

Checkpoints in this repo

Step	Original Tinker URI	Local path
`sampler_weights/final`	tinker://3154dee5-4fd7-59b6-ba3b-721eef675bfc:train:0/sampler_weights/final	`final`

How to load

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3-8B"
adapter = "arvindcr4/tinker-rl-scale_gsm8k_qwen3-8b-qwen3-8b"

tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, adapter, subfolder="final")  # or "<step>"

Companion releases

—Dataset: `arvindcr4/tinker-rl-bench-wandb` — all 334 W&B runs + 9,255 history rows
—Manifest: `arvindcr4/tinker-rl-bench-checkpoints` — full catalogue of every Tinker URI
—Code: `pes-llm-research/tinker-rl-lab`

Citation

bibtex

@misc{tinkerrlbench2026,
  title   = {A Unified Benchmark for RL Post-Training of Language Models},
  author  = {Arvind, C. R. and Jeyaraj, Sandhya},
  year    = {2026},
  note    = {NeurIPS submission, https://github.com/pes-llm-research/tinker-rl-lab}
}

License

Apache 2.0. The underlying base model retains its original license — please check Qwen/Qwen3-8B for any usage restrictions.