CoolFace
Modelpublic

GatekeeperZA/xLAM-1b-fc-r-RKLLM-v1.2.3

sourceHugging Facecc-by-nc-4.0updated 1mo agoView on Hugging Face
0likes10downloads
Model Card

xLAM-1b-fc-r for RKLLM v1.2.3 (RK3588 NPU)

Pre-converted Salesforce/xLAM-1b-fc-r for the Rockchip RK3588 NPU using rknn-llm runtime v1.2.3.

Runs on Orange Pi 5 Plus, Rock 5B, Radxa NX5, and other RK3588-based SBCs with 8GB+ RAM.

xLAM-1b-fc-r achieves 78.94% on BFCL v3 — the highest function-calling accuracy of any model under 2B parameters. Based on DeepSeek-Coder-1.3B with Salesforce's xLAM function-calling fine-tune.

Files

FileSizeDescription
xLAM-1b-fc-r-rk3588-v123-w8a8.rkllm1.4 GBLLM (W8A8 quantized, 8192 token context)

Why xLAM-1b-fc-r?

This model is the best option for on-device tool calling on RK3588 hardware:

ModelParamsBFCL v3 AccuracyNotes
xLAM-1b-fc-r1.1B78.94%Best sub-2B for function calling
Qwen2.5-1.5B-Instruct1.5B~40-50%General instruct, not FC-tuned
Llama-3.2-1B1.2B~30-40%General purpose

For general chat and reasoning, pair it with GatekeeperZA/Qwen2.5-1.5B-Instruct-RKLLM-v1.2.3.

Quick Start

Directory Structure

~/models/xLAM-1b-fc-r/
    xLAM-1b-fc-r-rk3588-v123-w8a8.rkllm

Using with RKLLM API Server

This model is designed for use with the RKLLM API Server, which provides an OpenAI-compatible API for RK3588 NPU inference.

bash
mkdir -p ~/models/xLAM-1b-fc-r
# Copy .rkllm file here — the API server will find it automatically
sudo systemctl restart rkllm-api

The model will appear as xlam-1b-fc-r in the OpenAI-compatible model list.

Function Calling Format

xLAM uses a specific prompt format for tool calling. Refer to the original model card for the system prompt template and JSON tool schema format.

Export Details

ParameterValue
SourceSalesforce/xLAM-1b-fc-r
Base architectureDeepSeek-Coder-1.3B
Toolrkllm-toolkit v1.2.3
QuantizationW8A8 (8-bit weights, 8-bit activations)
Optimization level1
Target platformrk3588
NPU cores3
Max context8192 tokens

Performance Benchmarks

Tested on Orange Pi 5 Plus (16GB RAM), RK3588 SoC, RKNPU driver 0.9.8:

MetricValue
Decode speed~20 tok/s (estimated)
Model load time~2 s
Peak RAM~1.6 GB

Known Limitations

  • —General reasoning: xLAM-1b-fc-r is fine-tuned for function calling, not general chat. Use Qwen2.5-1.5B for open-ended tasks.
  • —Context: Compiled at 8192 tokens. The runtime will truncate inputs that exceed this limit.
  • —Version lock: This file requires RKLLM runtime v1.2.3 and NPU driver v0.9.8. It is not compatible with v1.3.0 or later runtimes.
  • —License: The original xLAM-1b-fc-r is CC-BY-NC-4.0 (non-commercial). This converted file inherits that license.

License

CC-BY-NC-4.0, inherited from Salesforce/xLAM-1b-fc-r. Non-commercial use only.

Credits