Modelpublic

MooreMuaMu/ancient-rl-grpo-pilot3-step50

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes5downloads

Model Card

Ancient Language GRPO — Pilot 3 Step 50

最均衡的 RL checkpoint：10 个 bucket 中 8 个涨点。

模型信息

—基座模型: Qwen3-14B (zisuh/round2-10lang-passk-15k-sft-0608 checkpoint-195-merged)
—训练方法: GRPO (Group Relative Policy Optimization) + LoRA rank=64
—Reward: BLEU-2 + BERTScore (batched)
—训练步数: 50 / 300 (此 checkpoint 在 step 50 保存)
—超参数: gen=16, temp=1.0, β=0.01, lr=2e-6, constant scheduler

训练 Bucket (10 个)

Bucket	任务	训练样本数	test_final_new Δ
tibetan\\annotation	注释	5,273	+0.03pp
traditional\mongolian\\_annotation	注释	2,403	+1.54pp ✅
manchu\\annotation	注释	5,068	+0.61pp ✅
tangut\\segmentation	断句	3,305	+0.43pp ✅
zhuang\\segmentation	断句	222	-0.92pp
uyghur\\restoration	修复	4,335	-1.81pp
classical\chinese\\_translation	翻译	10,000	+0.35pp ✅
zhuang\\translation	翻译	10,000	-0.58pp
traditional\mongolian\\_translation	翻译	10,000	+1.06pp ✅
tangut\\translation	翻译	10,000	-0.98pp

总训练数据: 60,606 samples (每 bucket ≤ 10,000，均衡采样)

评测结果 (sacrebleu sentence-BLEU-2, 0-100)

在 testfinalnew 上评测 (50 samples/bucket, n=8, temp=0.7):

Bucket	Baseline	This Model	Δ
traditional\_mongolian / annotation	57.04	58.58	+1.54pp
traditional\_mongolian / translation	29.42	30.48	+1.06pp
manchu / annotation	65.44	66.05	+0.61pp
tangut / translation	42.09	41.11	-0.98pp
tangut / segmentation	81.74	82.17	+0.43pp
classical\_chinese / translation	42.04	42.39	+0.35pp
tibetan / annotation	57.73	57.25	-0.48pp
zhuang / translation	51.23	50.65	-0.58pp
zhuang / segmentation	66.67	65.75	-0.92pp
uyghur / restoration	87.58	85.77	-1.81pp

使用方式

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("MooreMuaMu/ancient-rl-grpo-pilot3-step50", torch_dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("MooreMuaMu/ancient-rl-grpo-pilot3-step50")

关键发现

—β=0.01 让翻译任务获得了涨点（β=0.02 时翻译无改善）
—Step 50 是多数 bucket 的最优 checkpoint，说明早期学习最有效
—数据均衡（每 bucket ≤ 10k）防止了大 bucket 主导训练