yimn-Aghosh/zegrate-ai

Zegrate 14B Phase 2 - LoRA Checkpoint

This is a LoRA adapter checkpoint from Phase 2 training of the Zegrate 14B model.

Training Details

• —Base Model: Qwen2.5-14B-Instruct
• —Method: QLoRA (4-bit quantization + LoRA)
• —Steps: 400
• —Learning Rate: 2e-4
• —LoRA Rank: 32

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-14B-Instruct")
model = PeftModel.from_pretrained(model, "yimn-Aghosh/zegrate-ai/checkpoint-400")