Modelpublic

thread13/OrionLLM_GRM-3.2-Sky-GGUF-el4-MTP

sourceHugging Faceupdated 10h agoView on Hugging Face

0likes167downloads

Model Card

TL;DR

Recently I was running some perplexity calculations (extending initial benchmarks started here), and one thing that stood out was the quality of el4 ONYX models, especially its "quality" edition.

(Another thing that stood out was somewhat unfairly ignored quality of Kwaipilot/KAT-Coder-V2.5-Dev and OrionLLM/GRM-3.2-Sky.)

Now while el4 has an xspark draft model in their repository, I am somehow more used to the now-classic MTP one -- or may be I just somehow had more luck with mtp draft models on my specific hardware.

Either way, I used a slightly corrected version of tooltd's 'grm_mtp_graft.py to do the same trick and graft a [Qwen3.6 MTP draft model][a4lg] by a4lg to the ONYX set made by el4, and am uploading these here.

I shall probably also upload here the modified Python code and respective perplexity measurements, which could be of general interest.

[orion-sky]: https://huggingface.co/OrionLLM/GRM-3.2-Sky [el4-onyx]: https://huggingface.co/el4/GRM-3.2-Sky-ONYX-GGUF [tooltd-mtp]: https://huggingface.co/tooltd/GRM-3.2-Sky-MTP-GGUF [apex]: https://github.com/localai-org/apex-quant [unsloth]: https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/blob/main/Qwen3.6-35B-A3B-UD-Q8KXL.gguf [buzz]: https://gist.github.com/buzz/1c439684d5e3f36492ae9f64ef7e3f67 [tooltd-conv-py]: https://huggingface.co/tooltd/GRM-3.2-Sky-MTP-GGUF/blob/main/grmmtpgraft.py [a4lg]: https://huggingface.co/a4lg/Qwen3.6-35B-A3B-MTP-ONLY-GGUF [orion-sky]: https://huggingface.co/OrionLLM/GRM-3.2-Sky [sky-perf-img]: https://huggingface.co/OrionLLM/GRM-3.2-Sky/resolve/main/assets/agentic-performance-evaluation.png [iq4-xs]: https://github.com/ggml-org/llama.cpp/pull/5747 [ik1-iqk]: https://github.com/ikawrakow/ikllama.cpp/discussions/8 [ik2-imatrix]: https://github.com/ggml-org/llama.cpp/pull/4861 [code-contests-orig]: https://huggingface.co/datasets/deepmind/codecontests [boutell-kv-q8-0-directory-naming-issues]: https://www.reddit.com/r/LocalLLaMA/comments/1sq94qx/isanyonegettingrealcodingworkdonewith/ [garage-ppl-tests]: https://huggingface.co/thread13/endless-frontierBigBang-v1-GGUF-stub/blob/main/perplexity_comparison.md