Modelpublic

WMT26Anon/qwen3-4b-sft-cpo

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes5downloads

Model Card

Qwen3-4B-Instruct — SFT + CPO LoRA adapter

Contrastive Preference Optimization (CPO) on top of the SFT LoRA adapter for Qwen/Qwen3-4B-Instruct-2507. Anonymous submission to the WMT26 research track.

Training

—SFT stage: 2 epochs, learning rate 1e-4, glossary in prompt (same adapter as `WMT26Anon/qwen3-4b-sft`)
—CPO stage: 1 epoch, learning rate 5e-6, β = 0.1, over 10,000 (source, chosen, rejected) triplets. Rejected candidates are generated by the base model in zero-shot, no-glossary mode, targeting its own failure modes.
—LoRA rank 64, α = 128, dropout 0.05
—Target modules: qproj, kproj, vproj, oproj, gateproj, upproj, down_proj

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained('Qwen/Qwen3-4B-Instruct-2507')
model = PeftModel.from_pretrained(base, 'WMT26Anon/qwen3-4b-sft-cpo')
tok = AutoTokenizer.from_pretrained('Qwen/Qwen3-4B-Instruct-2507')

Code

Inference pipeline, KB, test sets, and evaluation scripts: https://anonymous.4open.science/r/RAG_System_for_Specialized_Terms-18BB/