Modelpublic

simaai/Qwen3-VL-4B-Instruct-Autoround-Safetensors

sourceHugging Faceapache-2.0updated 26d agoView on Hugging Face

0likes48downloads

Model Card

Qwen3-VL-4B-Instruct Autoround

Local pre-LLiMa checkpoint for Sima.ai. Do not upload this artifact.

Source revision: ebb281ec70b05090aa6165b016eac8ec08e71b17 (upstream Hugging Face commit; the shared local source directory is flattened).

Quantization

Component	Method	Weight format	Targets
Decoder Linear layers	AutoRound	symmetric INT4, G256	252
`lm_head`	GPTQ	symmetric INT4, G256, static act-order	1
Vision encoder and merger Linear layers	GPTQ	symmetric INT8, per-channel, static act-order	104
Projectors / exceptions	—	No AutoRound target	See `recipe.yaml`

Stage 1 uses deterministic NeelNanda/pile-10k text calibration (dataset revision not captured; 512 samples, sequence length 1024, AutoRound 200 iterations, batch size 1). Stage 2 uses deterministic lmms-lab/flickr30k (dataset revision not captured), test[:512], for 512 image-text samples (sequence length 2048, batch size 1) used by the head and vision GPTQ stages.

Evaluation

Checkpoint	MMStar overall accuracy	Status
Source	36.0667%	Matched full run, 2026-07-17 (`T20260717-105421`; exact-match local judge)
Existing GPTQ decoder baseline	29.5333%	Matched full run, 2026-07-17 (`T20260717-114055`; exact-match local judge; 0/1500 inference and judge failures)
This hybrid checkpoint	39.4667%	Matched full run, 2026-07-17; exact-match local judge; 0/1500 inference and judge failures

Full matched WikiText-2 word perplexity (EleutherAI/wikitext_document_level, wikitext-2-raw-v1, no example limit; 2026-07-18): source 14.6065; this checkpoint 15.9454; absolute degradation +1.3389; relative degradation +9.17%. Raw JSON evidence: perplexity_results/vlm_full_wikitext/qwen3_vl_4b_{source,autoround_w4g256_headgptq_w4_visiongptq_w8}_v3.json.

Reproduction

bash

python quantize.py --model-path /path/to/source-model --output-dir /path/to/new-output

The directory contains quantize.py, recipe.yaml, and versions.txt.

Environment

Exact Python, CUDA, Torch, Transformers, llmcompressor, AutoRound, and compressed-tensors versions are recorded in versions.txt.

Deployment

This is the pre-LLiMa quantized Hugging Face artifact. Compile it separately for Sima.ai hardware and keep compiler output separate. No upload is authorized for this VLM batch.

Limitations

Quantization quality varies by language, visual domain, prompt format, context length, and runtime. MMStar and WikiText do not replace validation on the intended deployment workload.