Advantech-EIOT/intel_qwen-qwen3.5-9b-int4

Intel Qwen3.5 9B INT4 OpenVINO

This repository contains a Qwen3.5 9B OpenVINO GenAI INT4 model package for local edge inference on Intel hardware.

Model Details

• —Organization: Advantech-EIOT
• —Base model: Qwen/Qwen3.5-9B
• —OpenVINO source package: OrinVoss/qwen3.5-9b-ov
• —Architecture: Qwen3.5 multimodal / vision-language model
• —Precision: INT4
• —Runtime: OpenVINO GenAI
• —Tested devices: CPU and Intel GPU

Files

This is not a single-XML OpenVINO model. It is an OpenVINO GenAI model folder. Load the whole folder with VLMPipeline.

Important files include:

openvino_language_model.xml
openvino_language_model.bin
openvino_tokenizer.xml
openvino_tokenizer.bin
openvino_detokenizer.xml
openvino_detokenizer.bin
openvino_vision_embeddings_model.xml
openvino_vision_embeddings_model.bin
openvino_vision_embeddings_merger_model.xml
openvino_vision_embeddings_merger_model.bin
openvino_vision_embeddings_pos_model.xml
openvino_vision_embeddings_pos_model.bin
openvino_text_embeddings_model.xml
openvino_text_embeddings_model.bin
embed_tokens.npy
config.json
processor_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja

Usage

import openvino_genai as genai

model_dir = "intel_qwen-qwen3.5-9b-int4"
pipe = genai.VLMPipeline(model_dir, "GPU")

result = pipe.generate(
    "Answer in one short sentence: what is OpenVINO?",
    max_new_tokens=64,
)
print(result)

Local Test Results

Tested on an Intel Core Ultra platform with OpenVINO 2026.3.0.

CPU and Intel GPU are recommended for this package.

Notes

Direct export through optimum-intel did not complete locally because the current Optimum exporter did not natively support model_type: qwen3_5 in this environment. This package was prepared from the pre-converted OpenVINO INT4 source package listed above and validated locally with OpenVINO GenAI.

License

This package follows the license metadata from the source OpenVINO package. Please also review the base Qwen model terms before use or redistribution.