Modelpublic

koushikkb12/Qwen-2.5-14B-Function-Calling-Specialist

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes

Model Card

🛠️ Qwen-2.5-14B-Function-Calling-Specialist

A budget-optimized, high-performance function-calling and tool-use adapter for Qwen2.5-14B-Instruct.

This model is fine-tuned using Unsloth and QLoRA on the Glaive Function-Calling v2 dataset. It is specialized to reliably output structured JSON function arguments and coordinate complex multi-step tool calls without hallucinations.

![Unsloth](https://github.com/unslothai/unsloth) ![License: Apache 2.0](https://opensource.org/licenses/Apache-2.0) ![Base Model: Qwen 2.5 14B](https://huggingface.co/unsloth/Qwen2.5-14B-Instruct)

🚀 Key Features

—Zero-Hallucination JSON: Optimized specifically to match strict API specifications and parameters.
—Multi-Turn Tool Orchestration: Efficiently handles the System (Tool Setup) -> User Query -> Tool Call -> Tool Output -> Final Answer loop.
—Unsloth Accelerated: Built using memory-efficient kernels, maximizing training speed and throughput.
—Apache 2.0 Licensed: Fully permissive and suitable for commercial enterprise integration.

📈 Training Logs & Loss Curve

Below is the training log captured from our Blackwell GPU run.

Step	Epoch	Training Loss	Learning Rate	VRAM Usage
1	0.003	1.2420	0.0000	18.2 GB
100	0.320	0.1850	1.36e-4	18.4 GB
200	0.640	0.0864	7.40e-5	18.4 GB
300	0.960	0.0663	9.09e-6	18.4 GB
313 (Final)	1.000	0.0691	0.0000	18.4 GB

Loss Curve Summary: The loss started at 1.24 and converged exceptionally smoothly down to a final step loss of 0.0691 (overall average training loss: 0.09403) in just 7 minutes and 12 seconds of compute. This reflects flawless learning of structured function-calling syntax with zero instability.

⚙️ Hyperparameters

Hyperparameter	Value
Base Model	`unsloth/Qwen2.5-14B-Instruct`
Quantization	4-bit (NF4)
LoRA Rank (r)	64
LoRA Alpha (α)	128
LoRA Target Modules	`q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
Batch Size (Per Device)	4
Gradient Accumulation Steps	4 (Effective batch size = 16)
Optimizer	`AdamW 8-bit`
Learning Rate	`2e-4` (Linear schedule)
Epochs	1
Context Length	2048

💻 How to Use

1. Fast Inference with Unsloth

To load and run this adapter 2x faster with Unsloth:

python

from unsloth import FastLanguageModel
import torch

max_seq_length = 2048
dtype = None # Auto detection
load_in_4bit = True

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "koushikkb12/Qwen-3.5-14B-Function-Calling-Specialist",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
)

# Enable fast inference
FastLanguageModel.for_inference(model)

# Define system prompt with available tools
messages = [
    {"role": "system", "value": "You are a helpful assistant with access to weather tools.\nTools: get_weather(location: str)"},
    {"role": "user", "value": "What's the weather in Seattle?"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize = True,
    add_generation_prompt = True,
    return_tensors = "pt"
).to("cuda")

outputs = model.generate(input_ids = inputs, max_new_tokens = 128, use_cache = True)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

2. Standard Transformers Inference

If you are not using Unsloth for inference, you can load the model with HuggingFace PEFT:

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/Qwen2.5-14B-Instruct",
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, "koushikkb12/Qwen-3.5-14B-Function-Calling-Specialist")
tokenizer = AutoTokenizer.from_pretrained("unsloth/Qwen2.5-14B-Instruct")

🏆 Acknowledgements

—Huge thanks to Unsloth for making fast LLM fine-tuning accessible and memory-efficient.
—Dataset provided by Glaive AI.