dghf77/Qwen2.5-3B-Instruct.Q5_spec_Dot_code_generation

Qwen2.5-3B-Instruct (LoRA DOT Fine-Tune, GGUF Q5)

A parameter-efficient fine-tune of Qwen2.5-3B-Instruct specialized for Graphviz DOT code generation. This model was trained with LoRA adapters and exported in GGUF Q5_K_M quantization for lightweight, portable inference.

📌 Model Summary

• —Base Model: Qwen2.5-3B-Instruct (bnb-4bit)
• —Fine-tuning Method: LoRA (Low-Rank Adaptation)
• —Export Format: GGUF (llama.cpp-compatible)
• —Quantization: Q5KM (5-bit, medium precision)
• —File Size: 2.22 GB
• —License: Apache 2.0
• —Task: Natural language → Graphviz DOT code
• —Hosting: Hugging Face Hub

🚀 Motivation

The base Qwen2.5-3B-Instruct model often produced:

• —❌ Syntax errors (malformed nodes/edges, unclosed brackets, invalid attributes)
• —❌ Hallucinated content (extra nodes/edges not in the prompt)

Fine-tuning with LoRA adapters on a curated dataset of 671 compiler-validated DOT examples resolved these issues, ensuring structurally valid DOT syntax generation.

📂 Training Details

• —Dataset Source: Graphviz Gallery (paired natural-language prompts + DOT code)
• —Validation: Every DOT sample compiled successfully before inclusion
• —Final Corpus: ~671 bug-free instruction/output pairs
• —Format: Alpaca-style JSON (train.jsonl, val.jsonl)
• —Trainer: Hugging Face trl.SFTTrainer
• —Environment: Google Colab GPU runtime

⚙️ Technical Stack

• —Unsloth – optimized fine-tuning framework
• —Unsloth Zoo – pretrained configs + utilities
• —Tokenizer Utils – fixes for DOT-specific symbols
• —TRL – supervised fine-tuning trainer
• —Causal Convid + Mamba SSM – efficient sequence modeling for long DOT scripts
• —Ninja + shutil – build + file management utilities

📦 Repository Contents

• —Qwen2.5-3B-Instruct.Q5_K_M.gguf (2.22 GB) – quantized model weights
• —README.md – model card
• —.gitattributes – LFS configuration

🖥️ Inference & Deployment

Compatible runtimes:

• —llama.cpp (CLI / server)
• —llama-cpp-python
• —Ollama
• —LM Studio, Jan, Docker runners
• —Unsloth Studio, Lemonade

Example (llama.cpp CLI):

./main -m Qwen2.5-3B-Instruct.Q5_K_M.gguf -p "Generate a DOT diagram for a binary tree"