Modelpublic

immanuelpeter/Qwen3.8-27B-Vision

sourceHugging Faceapache-2.0updated 24d agoView on Hugging Face

0likes134downloads

Model Card

Qwen3.8-27B Vision

This repository packages the Tower and learned merger from Qwen3.8-27B.

File	Tensors	What it holds
`model.safetensors`	333	Tower and learned merger, extracted from source shard 1
`config.json`		Vision-only `Qwen3_5VisionConfig`
`preprocessor_config.json`		Qwen image preprocessing configuration

Qwen implements the learned merger inside Qwen3_5VisionModel, so this repository does not need a separate Projector file.

Architecture

Component	Details
Tower	27 layers, 1152 hidden, 16 heads, 4304 intermediate, patch size 16
Token compression	2x2 spatial grouping
Learned merger	`LayerNorm(1152)`, `Linear(4608, 4608)`, GELU, `Linear(4608, 5120)`

last_hidden_state contains the raw Tower tokens. pooler_output contains the merged features at the language-model width.

Usage

See `examples/inference.py` for image feature extraction.

Validation

The release tests compare all 333 tensors with the pinned parent checkpoint using torch.equal. Fixed-image Tower and merged outputs also match the parent implementation bit-for-bit on CPU and in BF16 on an NVIDIA A100.

Reproduction

The export script reads model.visual.* from shard 1 of Qwen/Qwen3.8-27B, removes the prefix, and writes the original BF16 tensors. The script pins the parent revision.

Credits

Qwen released the Qwen3.8-27B weights and the native Transformers implementation.

License

Apache License 2.0, the same license as the source model.