Modelpublic

ermiaazarkhalili/Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes9downloads

Model Card

Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K

GRPO (Group Relative Policy Optimization) LoRA fine-tune of `meta-llama/Llama-3.2-3B-Instruct` on the `AI-MO/NuminaMath-CoT` dataset (10K training samples).

Field	Value
Base model	`meta-llama/Llama-3.2-3B-Instruct`
Method	GRPO (RL) with LoRA/PEFT adapters
Dataset	`AI-MO/NuminaMath-CoT`
Training samples	10K
Adapter	LoRA — load on top of the base model with PEFT

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B-Instruct", torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(base, "ermiaazarkhalili/Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K")
tok = AutoTokenizer.from_pretrained("ermiaazarkhalili/Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K")

msgs = [{"role": "user", "content": "Solve: what is 17 * 24?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=512)
print(tok.decode(out[0], skip_special_tokens=True))

Intended use & limitations

Research / non-commercial experimentation on mathematical reasoning. As a LoRA adapter it inherits every limitation of the base model; verify outputs before use. A GGUF build (adapter merged onto the base) is available at ermiaazarkhalili/Llama-3.2-3B-Instruct-GRPO-NuminaMath-10K-GGUF.