Modelpublic

just1nseo/olmo3-7b-think-if-rlvr-anchor-pyx01-pp0-8k

sourceHugging Faceupdated 2mo agoView on Hugging Face

0likes

Model Card

OLMo 3 7B Think — IF-RLVR anchor checkpoints

BF16 inference checkpoints for allenai/Olmo-3-7B-Think-DPO, trained with GRPO on instruction-following constraints and a p(y|x) anchor reward coefficient of 0.1.

Each checkpoint is a complete Transformers model in its own subfolder:

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "just1nseo/olmo3-7b-think-if-rlvr-anchor-pyx01-pp0-8k"
subfolder = "global_step_364"

tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=subfolder)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    subfolder=subfolder,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

OLMo 3 requires a Transformers version with native olmo3 support (the training runtime used Transformers 4.57.1).

Checkpoints

Subfolder	Completed epoch
`global_step_91`	1
`global_step_182`	2
`global_step_273`	3
`global_step_364`	4

Training configuration

Setting	Value
Experiment	`olmo3_7b_grpo_think_emptythink_anchor_pyx01_b1024_c1_pp0_8k_invalidsafe_r4`
Base and reference model	`allenai/Olmo-3-7B-Think-DPO`
Training epochs / total steps	4 / 364
Train batch size	1024
Rollouts per prompt	8
Maximum prompt / response length	2048 / 8192
Rollout sampling	temperature 0.6, top-p 0.95, presence penalty 0.0
Actor learning rate	`1e-6`
KL loss	low-variance KL, coefficient `0.001`
IF reward	fraction of satisfied IFEval constraints
Anchor reward	`p(y	x)`, coefficient` 0.1`, interval mode` both`
PPL scoring prefix	empty think block; final-answer tokens only

The full FSDP trainer checkpoints, including optimizer and data-loader state, are retained separately. This repository contains merged BF16 model weights intended for inference and evaluation.