Modelpublic

TokenAI-zer/Swift-Qwen3.8-27b-oQ8-mtp

sourceHugging Faceotherupdated 5d agoView on Hugging Face

0likes184downloads

Model Card

Swift-Qwen3.8-27b-oQ8-mtp

Unofficial 8-bit MLX quantization of ukisai/Swift-Qwen3.8-27b for Apple Silicon. The MTP head is retained. This is an MLX safetensors repository, not GGUF or PyTorch.

The source model and its license belong to UkisAI. I am not affiliated with UkisAI. See License and attribution before using or redistributing these weights.

This variant

Property	Value
Quantization	8-bit affine, group size 64
Weight size	27.94 GiB (30.00 GB) across 6 safetensors shards
Whole model directory	27.96 GiB (30.02 GB)
Architecture	Qwen3_5ForConditionalGeneration, with vision tower
Context limit in config	262,144 tokens; usable context depends on available memory
MTP	1 hidden layer and 29 tensors in the weight index

These values come from the shipped config.json, model.safetensors.index.json and local files. The quantization_config has a uniform 8-bit setting; unlike the oQ4 and oQ6 variants, it contains no module-specific bit overrides. Quantization is lossy, and this variant has no published post-quantization benchmark results.

Pick a variant

	[oQ4-mtp](https://huggingface.co/TokenAI-zer/Swift-Qwen3.8-27b-oQ4-mtp)	[oQ6-mtp](https://huggingface.co/TokenAI-zer/Swift-Qwen3.8-27b-oQ6-mtp)	[oQ8-mtp](https://huggingface.co/TokenAI-zer/Swift-Qwen3.8-27b-oQ8-mtp)
Weights on disk	15.81 GiB (16.97 GB), 4 shards	22.09 GiB (23.72 GB), 5 shards	27.94 GiB (30.00 GB), 6 shards
Weight precision	mixed 4/5-bit	mixed 6/8-bit	uniform 8-bit
Quantization	affine, group size 64	affine, group size 64	affine, group size 64
Choose when	prioritize memory headroom	balance size and weight precision	prioritize weight precision and have more memory

All three retain the MTP head. Task-level accuracy and speed after quantization have not been measured here. Runtime memory also depends on context length and KV cache.

Use with oMLX

Download the repository into the oMLX model directory:

~~~bash hf download TokenAI-zer/Swift-Qwen3.8-27b-oQ8-mtp --local-dir ~/.omlx/models/Swift-Qwen3.8-27b-oQ8-mtp omlx serve --model-dir ~/.omlx/models --port 8000 ~~~

The model ID is Swift-Qwen3.8-27b-oQ8-mtp. In oMLX, enable the MTP speculative head to use the included MTP weights. Without that setting, the model can still run but the extra head does not accelerate decoding.

The architecture is multimodal. Use an MLX runtime with Qwen3.5 vision support. The 262k context value is an architecture limit, not a memory guarantee.

Sampling

The shipped generationconfig.json comes from the upstream checkpoint. Start with its settings for thinking mode: temperature 1.0, topp 0.95, topk 20 and repetitionpenalty 1.0. Adjust for your task after checking output quality.

License and attribution

The upstream model identifies its license as Swift Open License v1.0. Its published terms apply to this derivative quantization; this repository does not grant a separate license to the upstream weights. See LICENSE-NOTICE.md for attribution and upstream links.

Citation

~~~bibtex @misc{swift-qwen3.8-27b-oq8-mtp, title = {Swift-Qwen3.8-27b-oQ8-mtp: 8-bit MLX quantization with MTP head}, author = {TokenAI-zer}, year = {2026}, url = {https://huggingface.co/TokenAI-zer/Swift-Qwen3.8-27b-oQ8-mtp}, note = {Unofficial quantization of ukisai/Swift-Qwen3.8-27b} } ~~~