Modelpublic

dylanneve1/Qwen3.5-0.8B_int4_sym_group-1_dyn_stateful

sourceHugging Faceapache-2.0updated 6mo agoView on Hugging Face

0likes6downloads

Model Card

Qwen3.5-0.8B — INT4 Symmetric Channel-wise (OpenVINO)

OpenVINO stateful model exported from Qwen/Qwen3.5-0.8B with INT4 symmetric channel-wise weight compression.

Model Details

Property	Value
Base model	Qwen/Qwen3.5-0.8B
Architecture	Qwen3.5 (hybrid linear-attention + full-attention)
Weight format	INT4 (4-bit integer)
Quantization	Symmetric, channel-wise (group_size=-1)
Shapes	Dynamic
State management	Stateful (KV cache + conv/recurrent states)

Architecture

Qwen3.5 is a hybrid model with 24 layers: 18 linear-attention (GatedDeltaNet) + 6 full-attention layers in a 3:1 repeating pattern. Linear-attention layers use fixed-size conv and recurrent states (cache_params.past.{conv,ssm}.N).

Conversion

bash

pip install git+https://github.com/rkazants/optimum-intel.git@support_qwen3_5
pip install --pre -U openvino openvino-tokenizers nncf --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
pip install transformers==5.2.0

optimum-cli export openvino \
  --model Qwen/Qwen3.5-0.8B \
  --weight-format int4 \
  --group-size -1 \
  --sym \
  Qwen3.5-0.8B_int4_sym_group-1_dyn_stateful