Modelpublic

SeongryongJung/Qwen3-8B-Materials-SRPO-TR

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes6downloads

Model Card

Qwen3-8B-Materials-SRPO-TR

This repository contains the Qwen3-8B Materials SRPO-TR run trained with batch size 32.

The repository root contains the best validation checkpoint (global_step_20), selected by validation mean@16. checkpoints/last/ contains the final checkpoint.

Performance

Dataset	Method	Base model	Train batch size	Best val mean@16	Best checkpoint	Final val mean@16	Final checkpoint
Materials	SRPO-TR	Qwen3-8B	32	69.28%	20	69.28%	100

[image]

Raw result files:

—results/validation_mean16.csv
—results/training_scores.csv
—artifacts/config.yaml
—artifacts/wandb-summary.json

Training Hyperparameters

Hyperparameter	Value
Base model	`Qwen/Qwen3-8B`
Method	`SRPO-TR`
Train batch size	`32`
Train max samples	`3200`
Total training steps	`100`
Save frequency	`10`
Test frequency	`10`
Teacher regularization	`trust-region`
Teacher update rate	`0.1`
Rollout samples per prompt	`8`
Validation samples per prompt	`16`
Learning rate	`5e-6`
vLLM GPU memory utilization	`0.8`

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "SeongryongJung/Qwen3-8B-Materials-SRPO-TR"
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    torch_dtype="auto",
    device_map="auto",
    trust_remote_code=True,
)