CoolFace
Modelpublic

mojoz/qwen2.5-1.5b-instruct-lora-finetuned

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes
Model Card

Qwen2.5-1.5B-Instruct LoRA Fine-tuned for Tool Calling

Model Description

This is a LoRA fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct trained for application-layer tool calling capabilities in the Dongshan knowledge domain.

Architecture: Application-Layer Tool Calling

This model uses a unique application-layer tool calling architecture where:

  • —The application layer wraps the model and handles tool execution internally
  • —The model outputs tool-CALL-COMPLETED results (not tool call requests)
  • —Tools include: calculator, clock, search, lookup, and composite reasoning

Supported Tool Tags

ToolTagDescription
Calculator<calculator>expr</calculator>Mathematical computation
Clock<clock/>Current date/time
Search<search_1/> ... </search_1>Web search
Lookup<lookup/>Knowledge base lookup
Analyze<analyze/>Problem analysis
Evaluate<evaluate_1/>Quality evaluation
Synthesize<synthesize/>Information synthesis
Focus<focus/>Focus attention
Think<think/>Step-by-step reasoning
Verify<verify/>Result verification
Reflect<reflect/>Self-reflection
Correct<correct/>Error correction

Training Details

  • —Base Model: Qwen/Qwen2.5-1.5B-Instruct
  • —Method: LoRA (PEFT)
  • —LoRA Config: r=16, alpha=32, dropout=0.05
  • —Target Modules: qproj, vproj, kproj, oproj, gateproj, upproj, down_proj
  • —Training Data: 5000 samples (mojoz/dongshan-tool-calling-5k)
  • —Epochs: 3
  • —Learning Rate: 2e-4
  • —Batch Size: 2 (gradient_accumulation=4)
  • —Quantization: INT4 (during training)

Tool Distribution in Training Data

Tool TypePercentage
Calculator33.1%
Search21.5%
Lookup20.1%
Clock14.4%
No-tool18.4%
Multi-tool7.5%

Usage

With Application Layer

python
from tools.app_layer import DongshanAppLayer

app = DongshanAppLayer(model_name="mojoz/qwen2.5-1.5b-instruct-lora-finetuned")
result = app.query("今天星期几？现在几点了？")
print(result)

Direct Model Usage

python
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model and LoRA adapter
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "mojoz/qwen2.5-1.5b-instruct-lora-finetuned")
tokenizer = AutoTokenizer.from_pretrained("mojoz/qwen2.5-1.5b-instruct-lora-finetuned")

# Chat with tool calling
messages = [
    {"role": "system", "content": "你是东山论知识助手..."},
    {"role": "user", "content": "帮我计算 123 × 456"}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0]))

Related