Modelpublic

tatasauce4life/qwen3-4b-en2zh-orpo_beta0.3

sourceHugging Faceapache-2.0updated 8d agoView on Hugging Face

0likes

Model Card

Qwen3-4B 英翻中小說翻譯模型 (ORPO)

概述

—模型類型: LoRA Adapter (SFT + ORPO 累積更新)
—基礎模型: unsloth/Qwen3-4B-Base-unsloth-bnb-4bit
—SFT 模型來源: tatasauce4life/qwen3-4b-en2zh-lora
—訓練方法: ORPO (Odds Ratio Preference Optimization)
—任務: 英文到中文小說翻譯

訓練流程

1.SFT 階段: 使用英中平行小說語料做 LoRA SFT
2.ORPO 階段: 使用人工譯文 (chosen) vs. 機器翻譯 (rejected) 做偏好優化

ORPO 資料格式

json

{
  "prompt": "英文原文",
  "chosen": "人工譯文（高品質）",
  "rejected": "機器翻譯（低品質）"
}

翻譯模板

### English
{english_text}
### Chinese
{chinese_translation}

主要訓練參數

參數	值
ORPO beta	0.05
Learning Rate	5e-05
Epochs	3
Batch Size	1
Gradient Accumulation	16
Max Sequence Length	1024
LoRA r	16
LoRA alpha	32

Epoch	Training Loss	Validation Loss	Runtime	Samples Per Second	Steps Per Second	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen	Nll Loss	Log Odds Ratio	Log Odds Chosen
1	1.832741	1.839185	89.071000	4.087000	4.087000	-0.066546	-0.058645	0.329670	-0.007902	-1.172890	-1.330927	7.650677	7.531346	1.794205	-0.899610	-0.191442
2	1.647559	1.800575	87.179100	4.175000	4.175000	-0.065111	-0.066198	0.423077	0.001087	-1.323966	-1.302224	6.627131	6.495617	1.760711	-0.797270	0.070304
3	1.594447	1.808474	88.543700	4.111000	4.111000	-0.065936	-0.070564	0.450549	0.004629	-1.411288	-1.318715	6.002413	5.855793	1.770063	-0.768214	0.175277

如何載入模型

此模型是 LoRA adapter，包含 SFT + ORPO 的累積更新。

使用 Unsloth（推薦）

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="tatasauce4life/qwen3-4b-en2zh-orpo_beta0.3",
    max_seq_length=1024,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

使用 PEFT

python

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained(
    "tatasauce4life/qwen3-4b-en2zh-orpo_beta0.3",
    load_in_4bit=True,
)
tokenizer = AutoTokenizer.from_pretrained("tatasauce4life/qwen3-4b-en2zh-orpo_beta0.3")

注意: 此 adapter 的 base model 為 unsloth/Qwen3-4B-Base-unsloth-bnb-4bit。 adapter 包含 SFT 階段與 ORPO 階段的累積權重更新。

使用範例

python

prompt = "### English\n{your_english_text}\n### Chinese\n"
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
result = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(result)