astom-M/lora-sft-v5

Qwen3-4B Structured Output LoRA (v5)

LoRA adapter fine-tuned from Qwen/Qwen3-4B-Instruct-2507 using QLoRA (4-bit, Unsloth). LoRA adapter weights only - base model must be loaded separately.

Training Objective

Improve structured output accuracy (JSON / YAML / XML / TOML / CSV). Loss applied only to final assistant output (CoT masked).

Training Configuration

• —Base model: Qwen/Qwen3-4B-Instruct-2507
• —Method: QLoRA (4-bit) + rsLoRA
• —Max sequence length: 1024
• —Epochs: 2
• —Learning rate: 2e-05
• —LoRA: r=64, alpha=128
• —Effective batch size: 16

Sources & Terms

Training data: u-10bei/structureddatawithcotdataset512v2 Dataset License: MIT License.