xingqiwang/tang-song-poet-lora

Tang-Song Poet LoRA

PEFT LoRA adapter for Qwen/Qwen2.5-0.5B-Instruct, fine-tuned on Tang poetry and Song ci from the public chinese-poetry corpus as packaged in the tiny-poet project.

Project links

• —Training source
• —Online demo

Training configuration

• —Base architecture: Qwen2 decoder-only Transformer, approximately 494M parameters
• —Max sequence length: 256
• —Steps: 300
• —Batch size: 1
• —Gradient accumulation: 2
• —LoRA rank: 8
• —LoRA alpha: 16
• —LoRA dropout: 0.05
• —LoRA targets: q_proj, v_proj
• —Trainable parameters: 540,672 (0.1093%)

The base weights are frozen. Causal language-modeling loss is computed only on assistant tokens; system and user tokens are masked with -100.

Generated poetry may contain factual, tonal, metrical, or attribution errors.

Usage

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model_id = "YOUR_USERNAME/tang-song-poet-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoPeftModelForCausalLM.from_pretrained(model_id)