sluttybutfast/BattleKAT-Coder-V2.5-Dev-Vision-oQ4e

sluttybutfast/BattleKAT-Coder-V2.5-Dev-Vision-oQ4e

[image]

This is BattleKat, a battle hardened merge of great Qwen3.6-35B-A3B parts:

• —Language model: from ZQ-Dev/KAT-Coder-V2.5-Dev-oQ4e (finetuned, quantized)
• —Vision tower: from Jundot/Qwen3.6-35B-A3B-oQ4e-mtp (unmodified, bf16)
• —Chat template: from froggeric/Qwen-Fixed-Chat-Templates (v21.3)
• —Custom template additions to further improve behaviour and tool handling
• —For superior caching, run with oMLX
• —Important!: use recommended settings (DO NOT USE WITH LOW TEMP!!, force sampling in oMLX if necessary)

Recommended settings (forced in oMLX):

temp 0.8, topp=0.95, minp=0.02, topk=20, repetitionpenalty=1.1, presencepenalty=0.0, thinkingbudget=none, enablethinking=true, preservethinking=true, outputtokenlimit=16384

Verify image capability

Install:

pip install mlx-vlm

Command line:

python -m mlxvlm.generate \ --model YOURMODEL_NAME \ --image path/to/image.jpg \ --prompt "Describe this image. Extract all text" \ --max-tokens 5000

Python:

from mlxvlm import load, generate from mlxvlm.promptutils import applychat_template

model, processor = load("YOURMODELNAME") config = model.config

prompt = applychattemplate( processor, config, "Describe this image. Extract all text", numimages=1, ) output = generate(model, processor, prompt, ["path/to/image.jpg"], maxtokens=5000, verbose=True) print(output)

How the merge was done

The two source models share the same qwen35moe architecture and an identical namespace convention, which made a clean weight-level merge possible without any retraining, fine-tuning, or projector re-alignment.

1. Namespace split

All tensors in both models fall into two disjoint prefixes:

• —language_model.* -> Language model + LM head -> taken from the FINETUNE
• —vision_tower.* -> Vision encoder + merger -> taken from the BASE VLM

The merge is a straightforward union: every languagemodel.* tensor from the finetune, and every visiontower.* tensor from the base VLM.

2. Dimension compatibility

The vision projector (visiontower.merger) outputs visionconfig.outhiddensize = 2048, which matches the language model's textconfig.hiddensize = 2048. Vision features therefore project directly into the LM embedding space with no adapter needed. Image features are injected at the imagetokenid position (single-point injection; deepstackvisualindexes is empty).

3. Precision and quantization

• —Language model: retains the finetune's mixed quantization scheme -- 4-bit switch_mlp experts, 8-bit attention / shared experts / embeddings / LM head (group size 64, affine). The finetune's quantization map was carried over verbatim.
• —Vision tower: kept at full precision (bfloat16), exactly as shipped in the base VLM. MLX supports this mixed quantized-LM / bf16-vision setup.

4. Configuration

The merged config.json uses the finetune's config as the base (it holds the correct quantization map and text_config), with the following vision-related fields grafted in from the base VLM:

• —vision_config
• —imagetokenid, videotokenid
• —visionstarttokenid, visionendtokenid

The multi-token-prediction head was not carried over (mtpnumhiddenlayers remains 0, matching the finetune). It does not seem reasonable to reapply a mtp, as the finetune has derived from the base and the base mtp will result in poor matching without further retraining. The image preprocessor (preprocessorconfig.json) were taken from the base VLM, since the finetune's tokenizer configuration was text-only. The tokenizer vocabulary is identical between both sources (vocab_size = 248320, same special-token IDs), so the base VLM's template is fully compatible.

5. Verification

The merged weights were verified byte-for-byte via SHA-256 hashes of each tensor, comparing the merged output against both sources:

• —All 1677 language_model.* tensors are byte-identical to the finetune.
• —All 333 vision_tower.* tensors are byte-identical to the base VLM.

This model is a derivative combining:

• —Language model: ZQ-Dev/KAT-Coder-V2.5-Dev-oQ4e
• —Vision tower: Jundot/Qwen3.6-35B-A3B-oQ4e-mtp

License: apache-2.0. You must comply with the licenses of BOTH source models. Please cite and credit the original authors of both the base VLM and the finetune.

Reproduction

The merge was performed with a script (scripts/merge.py) that:

1. 1.Loads all safetensors from both source directories natively in MLX (preserving bfloat16 and packed-quant dtypes).
2. 2.Selects languagemodel.* tensors from the finetune and visiontower.* tensors from the base VLM.
3. 3.Builds the merged config (finetune config + base vision fields).
4. 4.Copies the image preprocessor config from the base VLM, rest from finetune.
5. 5.Saves the merged safetensors with a model.safetensors.index.json.

The merge was split into Hugging Face splittensors using the scripts/split.py.

The model was verified for correct tensors (checking text tensors were untouched) using the scripts/verify.py.

You should be able to do your own merges with the script as long as the two models are compatible (check for same dimensions, same tokenizer size). Verify and check vision capabilities (f.e. ocr a text).

The model uses a tuned froggeric chattemplate with default reasoningpreserve=true and custom system prompt addons to lower thinking loop probability. If you prefer the original chattemplate replace chattemplate.json with chat_templat.json.org or any you like.