Modelpublic

kangsyahrul/dendriva-qwen2.5-coder-3b-instruct-lora

sourceHugging Faceotherupdated 1mo agoView on Hugging Face

0likes9downloads

Model Card

Dendriva Qwen2.5-Coder 3B Instruct — LoRA

PEFT LoRA adapter trained from Qwen/Qwen2.5-Coder-3B-Instruct. This is the lightweight, trainable-format export for Unsloth or Transformers. It requires the base model at load time.

The ready-to-run LM Studio quantization is available in kangsyahrul/dendriva-qwen2.5-coder-3b-instruct-gguf.

Training provenance

—Selected checkpoint: checkpoint-69
—Epochs: 3
—Steps: 69
—Context length: 32,768
—LoRA rank / alpha / dropout: 16 / 16 / 0
—Learning rate: 2e-4
—Batch size: 2
—Optimizer: AdamW 8-bit
—Warmup steps: 3
—Training tokens reported by Unsloth Studio: 8,927,658

The repository intentionally excludes optimizer, scheduler, RNG, and trainer state because those are not required for local inference.

Load with Transformers and PEFT

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen2.5-Coder-3B-Instruct"
adapter_id = "kangsyahrul/dendriva-qwen2.5-coder-3b-instruct-lora"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, adapter_id)

In Unsloth Desktop, use the Hugging Face model source and enter the full adapter repository ID. Authenticate with a Hugging Face token because the repository is private.

Evaluation status

The adapter files and tokenizer were verified after upload. The training run completed successfully, but no comprehensive held-out coding or Manim benchmark is published with this repository. Compile, render, and test generated code before use.

License

This derivative follows the Qwen Research License of the base model. Review the base-model license before redistribution or commercial use.