petr567/Ornith-1.0-35B-Strix-Halo-Hybrid-LMStudio-GGUF

Ornith-1.0-35B Strix Halo Hybrid — LM Studio GGUF

LM Studio compatible, no integrated MTP.

Choose the correct build

This repository is the compatibility build. It preserves all 40 optimized base-model layers and removes only the appended MTP prediction layer and its qwen35moe.nextn_predict_layers metadata. No remaining tensor was converted, retrained, or requantized during the compatibility rewrite.

File

Compared with the full MTP artifact, this file is 521.38 MiB smaller. The removed data consists of exactly 20 blk.40.* prediction tensors. The output contains 733 tensors, reports qwen35moe.block_count = 40, and contains no blk.40.* or nextn_predict_layers entry.

Why a separate LM Studio build exists

The full artifact loads in newer llama.cpp builds that support integrated Qwen3.6/Ornith MTP. LM Studio runtime 2.14.0 used for the Windows validation is based on llama.cpp b8861 and rejected that appended block before allocating the model. Removing only the unsupported MTP layer made the same optimized base load successfully.

Validation on Windows 11:

• —LM Studio runtime: llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.14.0;
• —llama.cpp revision reported by LM Studio: b8861 (cf8b0db);
• —model load completed in 15.16 seconds in the compatibility smoke profile;
• —a local generation request completed successfully;
• —the incompatible full-MTP file failed in both GPU and CPU-only load tests, proving that the failure was format/runtime support rather than VRAM.

Optimization retained in this build

• —the compatible Ornith/Qwen3.6 35B A3B MoE architecture and chat template;
• —selective Q40 replacement of 20 frequently streamed MoE down-projection tensors, with the surrounding Q4K_M quantization plan unchanged;
• —all attention, dense, embedding, normalization and output tensors from the optimized 40-layer base;
• —the same inference quality as the full artifact when speculation is off.

No weights were trained or fine-tuned. Selective requantization changes the numerical representation, so the original project checked quality end-to-end on the same Orion coding slice.

Performance scope

The table below was measured on the same Windows 11 laptop with the official llama.cpp b10066 CUDA container, 16,384 context, Q8_0 K/V cache, batch 2048, ubatch 512, parallelism 1, dense/attention tensors on an RTX 4060 Laptop 8 GiB, and MoE experts in host RAM. The Hybrid, speculation off row executes the same 40-layer tensor set published here, but it is not presented as a direct LM Studio-runtime benchmark.

The stable non-speculative improvement was +23.8% at 8K and +21.8% on repeated code. The 60–67 tok/s result belongs to the linked MTP build under a compatible runtime and must not be expected from this no-MTP file.

Direct LM Studio native API benchmark

The compatibility file was then measured through LM Studio's own /api/v0/completions endpoint on the same Windows laptop. Each row generated 128 tokens at temperature 0. The 1K and 8K prompts were run twice; the repeated 8K code prompt was run three times. LM Studio reported decode throughput and TTFT directly in the response. The first TTFT is cold-prompt processing; the second shows LM Studio's prompt-cache path.

Against the b10066 no-speculation run on the same tensor set, LM Studio decode was +12.7% at 1K, +17.5% at 8K and +24.3% on repeated code. These are measured runtime differences, not a promise for other LM Studio versions or hardware.

The complete native response statistics are published in lmstudio-native-benchmark.json.

Orion four-scenario coding slice through LM Studio

The same seeded workspaces, Orion binary, prompts and deterministic checks used for the earlier laptop A/B were executed through http://127.0.0.1:1234/v1. Settings remained approval=auto, 24 agent steps, 16,384 context and a 900-second outer timeout.

Quality is identical to both the baseline and full-hybrid laptop runs. Raw decode was faster, but end-to-end agent time was longer because Orion repeated verification-guard steps; the Java coverage scenario reached the 900-second outer timeout after its file changes and tests were already complete.

The Rabbit post-run verifier initially failed to spawn the generic python command with EPERM. The saved workspace compiled successfully and the exact three-test suite passed under the bundled Python interpreter. The raw 5/8 result and original error are retained in the local research artifact; the published summary records the recovered 6/8 score and that the model itself was not rerun. orion-lmstudio-summary.json contains the check-level result.

Suggested LM Studio load profile

• —context length: 16,384 for the first validated load;
• —K cache: Q8_0;
• —V cache: Q8_0;
• —parallel sessions: 1;
• —evaluation batch size: 2048;
• —keep MoE expert layers in host RAM on an 8 GiB GPU;
• —offload dense/attention layers to the GPU as memory permits.

Start at 4K or 8K context if LM Studio's resource guardrails reject the first load, then increase context after measuring available RAM and VRAM. Context length affects KV memory but does not restore MTP; use the linked full build with a current llama.cpp runtime when speculative MTP acceleration is required.

Sources and licensing

The base model is deepreinforce-ai/Ornith-1.0-35B. The full linked build also contains compatible MTP tensors from a4lg/Qwen3.6-35B-A3B-MTP-ONLY-GGUF. This compatibility artifact contains no MTP tensors, but provenance is retained because it was derived from that combined artifact. Review the source repositories and the accompanying license files before redistribution or commercial use.