GatekeeperZA/Qwen3-Embedding-0.6B-RKLLM-v1.2.3

Qwen3-Embedding-0.6B — RKLLM v1.2.3 (w8a8, RK3588)

RKLLM conversion of Qwen/Qwen3-Embedding-0.6B for Rockchip RK3588 NPU inference.

Converted with RKLLM Toolkit v1.2.3. This model generates dense vector embeddings for semantic search, RAG pipelines, and similarity tasks — all running on the NPU without a GPU.

Key Details

Why This Model?

Running a dedicated embedding model on the RK3588 NPU allows the main LLM to use the full NPU without context-switching. Qwen3-Embedding-0.6B achieves strong retrieval performance at a compact size, making it ideal for local RAG pipelines on edge hardware.

Pair with the Qwen3-Reranker-0.6B for a complete retrieval stack.

Hardware Tested

• —Orange Pi 5 Plus — RK3588, 16GB RAM, Armbian Linux
• —RKNPU driver 0.9.8
• —RKLLM Runtime v1.2.3

Usage

With the RKLLM Embedding Service

mkdir -p ~/models/Qwen3-Embedding-0.6B
cd ~/models/Qwen3-Embedding-0.6B
git lfs install && git clone https://huggingface.co/GatekeeperZA/Qwen3-Embedding-0.6B-RKLLM-v1.2.3 .

Use with GatekeeperZA/RKLLM-API-Server embedding endpoint.

File Listing

Compatibility Notes

• —Minimum runtime: RKLLM Runtime v1.2.1. v1.2.3 recommended.
• —RKNPU driver: ≥ 0.9.6
• —SoCs: RK3588 / RK3588S. Not compatible with RK3576 without reconversion.
• —RAM: ~1GB loaded.

Acknowledgements

• —Alibaba Qwen Team for Qwen3-Embedding
• —Rockchip / airockchip for the RKLLM toolkit and runtime
• —Converted by GatekeeperZA