Modelpublic

thc1006/cyberpuppy-v6-pinyin-lora

sourceHugging Facecc-by-nc-sa-4.0updated 5mo agoView on Hugging Face

0likes

Model Card

CyberPuppy v6 — Pinyin LoRA (LoRA-B) | Strengthened Homophone Defense

拼音分支 v6 · Qwen3-8B + LoRA r=64 + 同音字攻擊強化防禦 Companion to v6-bilingual. Doubled rank from r=32 → r=64 for stronger phonetic pattern recognition.

What changed from v5

—LoRA rank: 32 → 64 (doubled)
—LoRA alpha: 64 → 128
—Epochs: 3 → 5 (best at epoch 2)
—Max length: 128 → 192
—Consistency loss: λ=0 → λ=0.5

Performance impact (in v6 dual-LoRA ensemble)

Metric	v5.1	v6
Pinyin-only dev F1	0.7979	0.7983 (≈ same)
Ensemble HED-COLD	0.9126	0.9317 (+1.91pt)
Ensemble TC homo	0.8496	0.8510 (+0.14pt)

The pinyin LoRA's standalone dev F1 plateaus around 0.80, but the r=64 capacity helps the ensemble especially on systematic homophone perturbations (HED-COLD).

Usage

⚠️ Must be paired with [v6-bilingual text LoRA](https://huggingface.co/thc1006/cyberpuppy-v6-bilingual). See the companion repo for full ensemble code.

Training Details

Parameter	Value
Base model	Qwen/Qwen3-8B-Base
LoRA rank	64
LoRA alpha	128
Training data	179,186 samples (pinyin-converted v5 bilingual)
Epochs	5 (best at epoch 2, step 9954)
Learning rate	3e-5
Max length	192
Precision	bf16
Loss	Focal γ=2.5 + uncertainty + consistency λ=0.5
Hardware	1× NVIDIA RTX 5090 (32GB, 590W OC)

License

CC BY-NC-SA 4.0.

Citation

See v6-bilingual.

—thc1006/cyberpuppy-v6-bilingual — required companion text LoRA-A
—thc1006/cyberpuppy-v5-pinyin-lora — v5 version (r=32)

Contact

—Author: Hung-Che Tsai (hctsai1006@cs.nctu.edu.tw)
—Takedown: Email above — removed within 7 days