Modelpublic

tatasauce4life/qwen3-4b-en2zh-orpo_beta0.4

sourceHugging Faceapache-2.0updated 6d agoView on Hugging Face

0likes

Model Card

Qwen3-4B 英翻中小說翻譯模型 (ORPO)

概述

—模型類型: LoRA Adapter (SFT + ORPO 累積更新)
—基礎模型: unsloth/Qwen3-4B-Base-unsloth-bnb-4bit
—SFT 模型來源: tatasauce4life/qwen3-4b-en2zh-lora
—訓練方法: ORPO (Odds Ratio Preference Optimization)
—任務: 英文到中文小說翻譯

訓練流程

1.SFT 階段: 使用英中平行小說語料做 LoRA SFT
2.ORPO 階段: 使用人工譯文 (chosen) vs. 機器翻譯 (rejected) 做偏好優化

ORPO 資料格式

json

{
  "prompt": "英文原文",
  "chosen": "人工譯文（高品質）",
  "rejected": "機器翻譯（低品質）"
}

翻譯模板

### English
{english_text}
### Chinese
{chinese_translation}

主要訓練參數

參數	值
ORPO beta	0.08
Learning Rate	5e-05
Epochs	4
Batch Size	1
Gradient Accumulation	16
Max Sequence Length	1024
LoRA r	16
LoRA alpha	32

Epoch	Training Loss	Validation Loss	Runtime	Samples Per Second	Steps Per Second	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen	Nll Loss	Log Odds Ratio	Log Odds Chosen
1	1.858179	1.864919	93.603900	3.889000	3.889000	-0.106697	-0.099357	0.354396	-0.007340	-1.241965	-1.333712	7.605855	7.485754	1.795982	-0.861723	-0.100940
2	1.666296	1.817527	92.809200	3.922000	3.922000	-0.103970	-0.114902	0.489011	0.010932	-1.436270	-1.299626	6.519071	6.387166	1.757984	-0.744286	0.218549
3	1.562287	1.833609	94.846200	3.838000	3.838000	-0.107248	-0.130436	0.543956	0.023189	-1.630455	-1.340596	5.412767	5.248291	1.778783	-0.685330	0.427241
4	1.482364	1.859507	93.639700	3.887000	3.887000	-0.110568	-0.136780	0.552198	0.026211	-1.709749	-1.382105	4.884839	4.700651	1.805095	-0.680145	0.471212

如何載入模型

此模型是 LoRA adapter，包含 SFT + ORPO 的累積更新。

使用 Unsloth（推薦）

python

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="tatasauce4life/qwen3-4b-en2zh-orpo_beta0.4",
    max_seq_length=1024,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

使用 PEFT

python

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained(
    "tatasauce4life/qwen3-4b-en2zh-orpo_beta0.4",
    load_in_4bit=True,
)
tokenizer = AutoTokenizer.from_pretrained("tatasauce4life/qwen3-4b-en2zh-orpo_beta0.4")

注意: 此 adapter 的 base model 為 unsloth/Qwen3-4B-Base-unsloth-bnb-4bit。 adapter 包含 SFT 階段與 ORPO 階段的累積權重更新。

使用範例

python

prompt = "### English\n{your_english_text}\n### Chinese\n"
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
result = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(result)