yamaTK/qwen3-4b-lora-y-v40

qwen3-4b-lora-y-v40

LoRA adapter fine-tuned for structured output tasks (JSON / YAML / XML / TOML / CSV).

This repository contains adapter weights only. Load with the base model separately.

Training Summary

• —Base model: Qwen/Qwen3-4B-Instruct-2507
• —Dataset file: merged_dataset_final_clean_v41.jsonl
• —Max sequence length: 512
• —Epochs: 2
• —Learning rate: 3.896385201823815e-06
• —Warmup ratio: 0.06105633616739185
• —Weight decay: 0.0
• —LoRA rank/alpha: 128 / 128
• —LoRA dropout: 0.0
• —Gradient accumulation: 4
• —LoRA target modules: q_proj,k_proj,v_proj,o_proj,gate_proj,up_proj,down_proj
• —Seed: 73259
• —System policy mode: off

Local Reference Evaluation

Local evaluation on public_150.json (for offline comparison only):

• —Parse rate: 141/150 (94.0%)
• —Proxy score: 0.6440
• —Key coverage avg: 55.2%
• —Format parse rates: JSON 100%, YAML 100%, TOML 68%, XML 95%, CSV 100%

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_id = "Qwen/Qwen3-4B-Instruct-2507"
adapter_model_id = "yamaTK/qwen3-4b-lora-y-v40"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter_model_id)

Notes

• —Training data follows competition constraints (provided datasets only, no LLM-generated data).
• —public_150.json was not used for training data generation.