drivetechodyssey/Qwen-Drive-1.0-4B-VLM-mlx-8bit

Qwen-Drive-1.0-4B VLM · MLX 8-bit

The vision-language model inside Qwen/Qwen-Drive-1.0-4B, converted to MLX and quantized to 8 bits for Apple-silicon Macs. It answers questions about driving images, which is the VQA mode of Qwen-Drive-1.0.

Also available: bf16 (8.5 GB), which reproduces the original weights exactly, and 4-bit (2.8 GB, feature extraction only).

Only the VLM. Qwen-Drive-1.0 has three parts: the Qwen3.5-4B VLM, a Planning Expert (trajectories) and a BEV perception head (3D boxes, occupancy, map). This repository contains the VLM only. The other two run on a Mac either through PyTorch's MPS backend (`apple-silicon` branch) or in MLX, which reads this model for its features and is about twice as fast (`mlx-port` branch). Not for driving decisions. Research and demonstration only. Answers can be wrong.

Use

pip install "mlx-vlm==0.6.0"
mlx_vlm.generate --model drivetechodyssey/Qwen-Drive-1.0-4B-VLM-mlx-8bit \
    --image front.jpg --prompt "What should the ego vehicle do next?" \
    --max-tokens 400 --temperature 0

The CLI prints an empty thinking block (</think>) before the answer.

import re
from mlx_vlm import apply_chat_template, generate, load
from mlx_vlm.utils import load_config

path = "drivetechodyssey/Qwen-Drive-1.0-4B-VLM-mlx-8bit"
model, processor = load(path)
config = load_config(path)
prompt = apply_chat_template(processor, config, "What should the ego vehicle do next?",
                             num_images=1, enable_thinking=False)
result = generate(model, processor, prompt, image=["front.jpg"], max_tokens=400, temperature=0.0)
# the answer starts with an empty <think></think> block
print(re.sub(r"^\s*<think>.*?</think>\s*", "", result.text, flags=re.S))

How it was made

1. 1.Extract the VLM. Every VLM tensor in the original model.safetensors starts with vlm.; the prefix is removed (723 tensors; lm_head is tied to the embeddings). config.json is the original vlm_config, with vision_config.model_type set to qwen3_5 because mlx-vlm 0.6.0 does not accept qwen3_5_vision. Tokenizer, processor and chat-template files are copied unchanged.
2. 2.Convert. mlx_vlm.convert --dtype bfloat16 -q --q-bits 8 --q-group-size 64 (affine).
3. 3.Restore float32 norms. The original keeps each linear-attention layer's A_log and norm.weight in float32 (48 tensors). The converter kept A_log but cast the 24 norm.weight tensors to bf16, which made long greedy answers drift from the original after roughly 700 characters. Those 24 tensors are copied back from the original checkpoint.

Checks

Measured on an M5 Max (64 GB) with mlx 0.31.2 and mlx-vlm 0.6.0, greedy decoding, nothing else using the GPU. 16 images from the original repository's demo data (the current front, front-left and front-right frames of the four planning scenes, plus four front frames from the perception demo) × 3 questions: a long English scene description (400 tokens), a one-sentence hazard, and two or three sentences in Korean.

Where the 8-bit one-sentence hazards differ from bf16, they were checked against the images: the differences are rewordings, with no invented objects or wrong positions.

A 4-bit version (2.8 GB, 147 tok/s) is published for feature extraction only, not for answering questions: none of its 48 answers matched bf16 (mean similarity 0.44), and on three of the sixteen images it reported a pedestrian crossing that is not there.

Behaviour of the model itself, in bf16 as well: a Korean answer occasionally contains a Chinese word (1 of 16 here, 신호灯的).

Perception and planning also read this model

The perception head and the Planning Expert do not read the model's words. They read the pre-merge vision patches, the image-token hidden states and the full-attention K/V cache. Scored against this repository's demo ground truth, the choice of precision makes no difference there, which is not true of the answers above:

Prompt processing runs at the same speed at every precision (1.9 to 3.0 k tok/s), so a smaller conversion saves memory there rather than time. The MLX implementation of both heads is in the `mlx-port` branch.

License and citation

Apache License 2.0, the same as the original (see LICENSE). The weights are the Qwen team's (Alibaba Group); this repository only changes their format as described above.

@misc{zhou2026qwendrive10initialstepvisionlanguage,
      title={Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving},
      author={Xin Zhou and Zongchuang Zhao and Zhibo Yang and Mingsheng Li and Humen Zhong and Shuai Bai and Du Chu and Ruizhe Chen and Zhaohai Li and Jun Tang and Qiuyue Wang and Mingkun Yang and Jiazhao Zhang and Dayiheng Liu and Dingkang Liang and Xiang Bai},
      year={2026},
      eprint={2609.00111},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.00111},
}