Modelpublic

Alogotron/Qwen2.5-3B-JSON-StructuredOutput

sourceHugging Facemitupdated 2mo agoView on Hugging Face

0likes11downloads

Model Card

Qwen2.5-3B JSON Structured Output (LoRA)

A LoRA adapter that enhances Qwen2.5-3B-Instruct for extracting structured JSON from unstructured text.

Training Details

Parameter	Value
Base Model	Qwen/Qwen2.5-3B-Instruct
Method	QLoRA (4-bit NF4)
LoRA Rank	16
LoRA Alpha	32
Target Modules	q,k,v,o,gate,up,down
Trainable Params	29.9M (1.76%)
Training Examples	20 curated
Epochs	5
Learning Rate	5e-5
Final Loss	1.506
Hardware	NVIDIA RTX 3090 x2

Quick Start

python

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch, json

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-3B-Instruct")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", quantization_config=bnb, device_map="auto")
model = PeftModel.from_pretrained(model, "2reb/Qwen2.5-3B-JSON-StructuredOutput")

License

MIT