GatekeeperZA/Qwen3.5-0.8B-RKLLM-v1.2.3

Qwen3.5-0.8B — RKLLM v1.2.3 (w8a8, RK3588)

RKLLM conversion of Qwen/Qwen3.5-0.8B for Rockchip RK3588 NPU inference.

Converted with RKLLM Toolkit v1.2.3. This is the base model variant — suitable for custom fine-tunes or applications that supply their own system prompting.

Key Details

Why This Model?

Qwen3.5-0.8B is the successor to Qwen3-0.6B, offering improved architecture and training. At under 1GB after quantisation it is the smallest model in the RK3588 lineup — ideal for low-latency responses, always-on assistants, or edge deployments where memory is tight.

Hardware Tested

• —Orange Pi 5 Plus — RK3588, 16GB RAM, Armbian Linux
• —RKNPU driver 0.9.8
• —RKLLM Runtime v1.2.3

Usage

With the RKLLM API Server

mkdir -p ~/models/Qwen3.5-0.8B
cd ~/models/Qwen3.5-0.8B
git lfs install && git clone https://huggingface.co/GatekeeperZA/Qwen3.5-0.8B-RKLLM-v1.2.3 .

Use with GatekeeperZA/RKLLM-API-Server.

With the official RKLLM API demo

git clone https://github.com/airockchip/rknn-llm.git
cd rknn-llm/examples/rkllm_api_demo
./build/rkllm_api_demo /path/to/Qwen3.5-0.8B-rk3588-w8a8.rkllm 4096 8192

File Listing

Compatibility Notes

• —Minimum runtime: RKLLM Runtime v1.2.1. v1.2.3 recommended.
• —RKNPU driver: ≥ 0.9.6
• —SoCs: RK3588 / RK3588S. Not compatible with RK3576 without reconversion.
• —RAM: ~1.5GB loaded. Runs on any RK3588 board including 8GB variants.

Acknowledgements

• —Alibaba Qwen Team for Qwen3.5
• —Rockchip / airockchip for the RKLLM toolkit and runtime
• —Converted by GatekeeperZA