Modelpublic

GatekeeperZA/InternVL3.5-4B-Instruct-RKLLM-v1.2.3

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes36downloads

Model Card

InternVL3.5-4B-Instruct — RKLLM v1.2.3 (w8a8, RK3588)

RKLLM/RKNN conversion of OpenGVLab/InternVL3-4B for Rockchip RK3588 NPU inference.

Converted with RKLLM Toolkit v1.2.3 (language model) and RKNN Toolkit (vision encoder). This is a multimodal vision-language model — it accepts both images and text as input.

Key Details

Property	Value
Base Model	OpenGVLab/InternVL3-4B
Toolkit Version	RKLLM Toolkit v1.2.3 / RKNN Toolkit
Runtime Version	RKLLM Runtime ≥ v1.2.1 + RKNN Runtime
Quantization	w8a8 (8-bit weights, 8-bit activations)
Target Platform	RK3588
NPU Cores	3
Thinking Mode	❌ Not applicable
Model Type	Vision-Language (VLM)
Languages	English, Chinese (multilingual)

Why This Model?

InternVL3.5-4B is Shanghai AI Lab's compact vision-language model. It provides image understanding, visual question answering, and OCR capabilities at 4B parameters — all running on the RK3588 NPU without a GPU.

Compared to the Qwen3-VL-4B, InternVL has a different training lineage and excels at dense image analysis and chart/document understanding.

Hardware Tested

—Orange Pi 5 Plus — RK3588, 16GB RAM, Armbian Linux
—RKNPU driver 0.9.8
—RKLLM Runtime v1.2.3

Usage

With the RKLLM API Server (VLM mode)

bash

mkdir -p ~/models/internvl3.5-4b
cd ~/models/internvl3.5-4b
git lfs install && git clone https://huggingface.co/GatekeeperZA/InternVL3.5-4B-Instruct-RKLLM-v1.2.3 .

Use with GatekeeperZA/RKLLM-API-Server — the server loads both the .rkllm and .rknn files automatically when placed in the same directory.

File Listing

File	Description
`internvl3_5-4b-instruct_w8a8_rk3588.rkllm`	Language model weights for RK3588 NPU
`internvl3_5-4b_vision_rk3588.rknn`	Vision encoder for RK3588 NPU

Compatibility Notes

—Minimum runtime: RKLLM Runtime v1.2.1 + RKNN Runtime v2.x. v1.2.3 recommended.
—RKNPU driver: ≥ 0.9.6
—SoCs: RK3588 / RK3588S. Not compatible with RK3576 without reconversion.
—RAM: ~5.5GB loaded. Requires 8GB+ board (16GB recommended).

Acknowledgements

—Shanghai AI Lab / OpenGVLab for InternVL3
—Rockchip / airockchip for the RKLLM and RKNN toolkits
—Converted by GatekeeperZA