JupiterJil/cyberspace-qwen25-7b-lora

Cyberspace Presales Co-Pilot — Qwen 2.5 7B LoRA Adapter

LoRA adapter for Cyberspace Limited's Vertical AI Presales Co-Pilot.

Training Summary

• —Base model: Qwen/Qwen2.5-7B-Instruct
• —Method: LoRA (QLoRA 4-bit) via Unsloth
• —Dataset: 132 Cyberspace proposals (ChatML format, zero template bleed)
• —Final loss: 1.4795 (target: 1.2–1.6 ✅)
• —Training time: ~4.9 min on A100 80GB

LoRA Config

• —Rank: 16
• —Alpha: 16
• —Dropout: 0.05
• —Target modules: qproj, kproj, vproj, oproj, gateproj, upproj, down_proj
• —Epochs: 3
• —LR: 2e-4 (cosine scheduler)
• —train_on_responses_only: True (ChatML markers)

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base, "JupiterJil/cyberspace-qwen25-7b-lora")

Recommended Inference Settings

• —Temperature: 0.4
• —Top-p: 0.9
• —Repeat penalty: 1.1
• —Use detailed category-specific system prompts

Production Artifact

For inference, use the GGUF Q4KM version: JupiterJil/cyberspace-qwen25-7b-gguf