Modelpublic

lmxxf/little-pink-lora

sourceHugging Faceotherupdated 7mo agoView on Hugging Face

0likes6downloads

Model Card

Little Pink LoRA (小粉红 LoRA)

LoRA adapter for Qwen2.5-72B-Instruct-AWQ, fine-tuned on 1973 Chinese political commentary Q&A pairs to simulate a "小粉红" (Little Pink / Chinese nationalist) persona.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-72B-Instruct-AWQ",
    torch_dtype="float16",
    device_map="cuda:0"
)
model = PeftModel.from_pretrained(base_model, "lmxxf/little-pink-lora")
tokenizer = AutoTokenizer.from_pretrained("lmxxf/little-pink-lora")

Full project & interactive chat script: github.com/lmxxf/little-pink-ai

Training

Parameter	Value
Base Model	Qwen2.5-72B-Instruct-AWQ
Method	LoRA (r=16, alpha=32)
Training Data	1973 Q&A pairs
Epochs	3
Final Loss	0.2638
Training Time	18 hours
Hardware	NVIDIA DGX Spark (GB10, 128GB)
Framework	LLaMA-Factory

Training Hyperparameters

—learning_rate: 1e-4
—trainbatchsize: 1
—gradientaccumulationsteps: 8 (effective batch size = 8)
—lrschedulertype: cosine
—warmup_ratio: 0.1
—num_epochs: 3
—mixed_precision: fp16

Framework Versions

—PEFT 0.17.1
—Transformers 4.51.3
—PyTorch 2.10.0a0+b558c986e8.nv25.11