CoolFace
Modelpublic

akugel/qwen2.5-7b-instruct-toolace-qlora

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face
0likes8downloads
Model Card

Qwen2.5-7B-Instruct · ToolACE (QLoRA) — Function Calling

A QLoRA adapter for Qwen/Qwen2.5-7B-Instruct, fine-tuned on Team-ACE/ToolACE to improve function / tool calling, evaluated on the Python subset of BFCL (Berkeley Function Calling Leaderboard).

Full training/eval/serving code and the complete model comparison: 👉 https://github.com/aokugel/ACE-finetuning

Results — BFCL Python subset (macro accuracy, prompt mode)

ModelPython macro-acc
Qwen2.5-7B-Instruct (base)79.52
+ ToolACE (this adapter, QLoRA)81.56

Gains concentrate in irrelevance detection (knowing when not to call a function) and multiple-call accuracy. QLoRA matched LoRA accuracy at ~3–4× lower training memory.

Usage

python
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

repo = "<your-username>/qwen2.5-7b-instruct-toolace-qlora"   # this repo
model = AutoPeftModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained(repo)
# the base model loads automatically from adapter_config.json

For production, merge to bf16 and serve with vLLM (optionally FP8):

python
from peft import PeftModel
from transformers import AutoModelForCausalLM
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", torch_dtype="bfloat16")
merged = PeftModel.from_pretrained(base, repo).merge_and_unload()
merged.save_pretrained("Qwen2.5-7B-Instruct-ToolACE")
# vllm serve Qwen2.5-7B-Instruct-ToolACE --quantization fp8 ...

Training

  • —Method: QLoRA — frozen base in 4-bit NF4 (double-quant, bf16 compute).
  • —LoRA: r=16, α=32, dropout=0.05 on q,k,v,o,gate,up,down (~0.53% params trainable).
  • —Schedule: 2 epochs, lr 1e-4 cosine + 5% warmup, grad-clip 1.0, effective batch 32, max-len 4096.
  • —Loss: assistant-only masking (multi-turn); over-length examples dropped (not truncated).
  • —Data: ToolACE — 11,074 train / 226 val.
  • —Hardware: 1× H100 80GB.

Evaluation

BFCL Python subset (11 categories: simple / multiple / parallel / parallel-multiple + 6 live + relevance/irrelevance), prompt mode, served via vLLM, temperature 0.001. Per-category tables and latency benchmarks are in the GitHub repo.

Limitations

  • —The base model is already strong at function calling, so the absolute lift is modest (+~2 pts macro); the win is mainly robustness (irrelevance handling).
  • —Evaluated in BFCL prompt mode (bracket [func(arg=val)] format); behavior in native function-calling APIs may differ.

License & attribution

Adapter released under Apache-2.0, inheriting the base model's license (Qwen/Qwen2.5-7B-Instruct). Trained on Team-ACE/ToolACE — see its dataset card for terms.

Framework versions

  • —PEFT 0.19.1