ji-farthing/Mellum2-12B-A2.5B-Instruct-ik-llama-GGUF

Mellum2 12B A2.5B Instruct GGUF for ik_llama

This repository contains GGUF conversions of `JetBrains/Mellum2-12B-A2.5B-Instruct`.

The files were converted with an ik_llama.cpp branch that adds Mellum2 architecture support and emits the Mellum sliding-window and RoPE/YARN metadata needed by GGUF runtimes.

These files are intended as persistent convenience artifacts for ik_llama reviewers and users. They should also run on current llama.cpp builds that support the Mellum architecture.

No performance or model-quality claims are made here.

Files

Provenance

• —Source model: JetBrains/Mellum2-12B-A2.5B-Instruct
• —Source snapshot: 4ee5751ef73ac6ae5a65b76b092ffc7c3b9c60e3
• —Converter/runtime branch: joelfarthing/ik_llama.cpp, branch mellum2-support
• —PR companion branch URL: <https://github.com/joelfarthing/ik_llama.cpp/tree/mellum2-support>

The embedded chat template is the stock JetBrains Instruct template:

tokenizer.chat_template sha256 = e674cbec4c384ab50c18c91d8cada3b6931d7a7ee25d9db004366aa440c1ca86

The converted GGUF metadata includes:

• —mellum.attention.sliding_window = 1024
• —mellum.attention.sliding_window_pattern
• —mellum.rope.freq_base = 500000.0
• —mellum.rope.freq_base_swa = 500000.0
• —mellum.rope.scaling.type = yarn
• —mellum.rope.scaling.factor = 16.0
• —mellum.rope.scaling.original_context_length = 8192
• —mellum.rope.scaling.yarn_attn_factor = 1.2772588729858398
• —mellum.rope.scaling.yarn_beta_fast = 32.0
• —mellum.rope.scaling.yarn_beta_slow = 1.0

Local Validation

The BF16 and Q8_0 files were smoke-tested locally on an RTX 4070 with CUDA server builds.

Validation included:

• —Q80 with `ikllama.cpp CUDA server and --cpu-moe`
• —Q8_0 with current llama.cpp upstream CUDA server and --cpu-moe
• —BF16 with current llama.cpp upstream CUDA server and --cpu-moe
• —OpenAI-compatible chat completion request using the embedded chat template
• —deterministic long-code prompt
• —python3 -m py_compile on the extracted code
• —functional topological-sort test including cycle detection

The long-code smoke is a runtime sanity check only. It is not a benchmark and does not imply any quality ranking.

Example

./llama-server \
  -m Mellum2-12B-A2.5B-Instruct-ik-llama-Q8_0.gguf \
  -ngl 99 \
  --cpu-moe \
  -c 4096 \
  -b 512 \
  -ub 512 \
  --jinja

License

The source model card lists the license as Apache-2.0. See the upstream JetBrains model card for the authoritative license and model documentation.