Modelpublic

tatasauce4life/qwen3-4b-en2zh-orpo_beta0.2

sourceHugging Faceapache-2.0updated 9d agoView on Hugging Face

0likes

Model Card

Qwen3-4B 英翻中小說翻譯模型 (ORPO)

概述

—模型類型: LoRA Adapter (SFT + ORPO 累積更新)
—基礎模型: unsloth/Qwen3-4B-Base-unsloth-bnb-4bit
—SFT 模型來源: tatasauce4life/qwen3-4b-en2zh-lora
—訓練方法: ORPO (Odds Ratio Preference Optimization)
—任務: 英文到中文小說翻譯

訓練流程

1.SFT 階段: 使用英中平行小說語料做 LoRA SFT
2.ORPO 階段: 使用人工譯文 (chosen) vs. 機器翻譯 (rejected) 做偏好優化

ORPO 資料格式

json

{
  "prompt": "英文原文",
  "chosen": "人工譯文（高品質）",
  "rejected": "機器翻譯（低品質）"
}

翻譯模板

### English
{english_text}
### Chinese
{chinese_translation}

主要訓練參數

參數	值
ORPO beta	0.1
Learning Rate	0.0001
Epochs	5
Batch Size	1
Gradient Accumulation	16
Max Sequence Length	1024
LoRA r	16
LoRA alpha	32

Epoch	Training Loss	Validation Loss	Runtime	Samples Per Second	Steps Per Second	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen	Nll Loss	Log Odds Ratio	Log Odds Chosen
1	1.844135	1.854190	87.344000	4.167000	4.167000	-0.131862	-0.143242	0.450549	0.011379	-1.432417	-1.318622	7.127260	7.001682	1.778190	-0.759993	0.166458
2	1.552538	1.815805	87.031400	4.182000	4.182000	-0.129877	-0.175067	0.620879	0.045190	-1.750668	-1.298767	5.576954	5.433165	1.753491	-0.623141	0.613916
3	1.331949	1.891158	87.251700	4.172000	4.172000	-0.141172	-0.201667	0.642857	0.060495	-2.016670	-1.411715	4.119596	3.943425	1.829881	-0.612768	0.802193
4	1.110539	2.014756	86.803400	4.193000	4.193000	-0.155560	-0.229186	0.670330	0.073625	-2.291857	-1.555604	3.257812	3.058837	1.955649	-0.591075	0.954718
5	1.028039	2.102785	86.433800	4.211000	4.211000	-0.164843	-0.241751	0.673077	0.076909	-2.417511	-1.648425	2.883676	2.680442	2.043374	-0.594112	0.977506

如何載入模型

此模型是 LoRA adapter，包含 SFT + ORPO 的累積更新。

使用 Unsloth（推薦）

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="tatasauce4life/qwen3-4b-en2zh-orpo_beta0.2",
    max_seq_length=1024,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

使用 PEFT

python

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained(
    "tatasauce4life/qwen3-4b-en2zh-orpo_beta0.2",
    load_in_4bit=True,
)
tokenizer = AutoTokenizer.from_pretrained("tatasauce4life/qwen3-4b-en2zh-orpo_beta0.2")

注意: 此 adapter 的 base model 為 unsloth/Qwen3-4B-Base-unsloth-bnb-4bit。 adapter 包含 SFT 階段與 ORPO 階段的累積權重更新。

使用範例

python

prompt = "### English\n{your_english_text}\n### Chinese\n"
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
result = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(result)