Kritpawit/Qwen3.8-27B-NVFP4A16

Qwen3.8-27B-NVFP4A16 (tokenizer-fixed)

This is `huginnfork/Qwen3.8-27B-NVFP4A16` with one fix applied: the tokenizer no longer silently truncates every prompt to 2048 tokens.

All credit for the quantisation itself goes to `huginnfork` — an NVFP4A16 (W4A16) quantisation of `Qwen/Qwen3.8-27B`, 28.8 GiB vs 51.7 GiB bf16, MLPs-only (192 modules), with self_attn, linear_attn (SSM), the vision tower, lm_head and the MTP head kept in bf16. The model weights here are byte-identical to the original. See the original repo for full quantisation methodology and measurements.

What was broken

The original repo's tokenizer.json ships with truncation hardcoded into the Rust tokenizer's persistent state:

"truncation": {"direction": "Right", "max_length": 2048, "strategy": "LongestFirst", "stride": 0}

This contradicts the model_max_length: 262144 declared elsewhere in the same checkpoint (likely baked in by llm-compressor calibration, which commonly tokenizes calibration data at max_length: 2048, with the tokenizer saved after that config was applied).

Consequences for every downstream user of the original checkpoint:

• —Every tokenized request silently truncates to 2048 tokens before reaching the model.
• —Once truncation cuts into an image's <|image_pad|> placeholders (images needing > ~2048 combined tokens, e.g. anything above ~1500×1500 px), transformers' _check_special_mm_tokens raises a token-count mismatch that vLLM re-wraps as an opaque "Failed to apply Qwen3VLProcessor..." error.

What changed

Exactly one file differs from the original:

Everything else — weights, configs, chat template, preprocessor configs — is untouched.

With truncation disabled, over-length prompts are rejected explicitly by the serving stack (e.g. vLLM's --max-model-len check) instead of being silently truncated, which is the correct failure mode.

Verifying

from transformers import AutoTokenizer

tok = AutoTokenizer.from_pretrained("Kritpawit/Qwen3.8-27B-NVFP4A16")
print(tok.init_kwargs.get("max_length"))  # -> None (original: 2048)
print(tok.model_max_length)               # -> 262144
ids = tok("x " * 5000)["input_ids"]
print(len(ids))                           # -> full length, not capped at 2048