Frosty40/Ornith-1.0-35B-B70-Turbo

Ornith-1.0-35B B70 Turbo GGUF

This repository publishes a Q5_K_M GGUF for `deepreinforce-ai/Ornith-1.0-35B` plus the tested Intel Arc Pro B70 serving recipe.

The "Turbo" part is the serving stack, not a behavior-changing fine-tune:

• —same base model semantics as Ornith-1.0-35B
• —Q5KM GGUF quantization
• —llama.cpp SYCL runtime tuned for Intel Arc Pro B70
• —no safeguard edits, refusal edits, or new training
• —no speculative-head artifact in this release

For the benchmark package, raw tables, charts, and example games, see: `newjordan/Ornith-1.0-35B-B70-Turbo`.

Files

Recommended B70 Serve Config

Agent-fleet default:

GGML_SYCL_DISABLE_DNN=1 ONEAPI_DEVICE_SELECTOR=level_zero:gpu \
llama-server \
  -m ornith-1.0-35b-Q5_K_M.gguf \
  --alias ornith-1.0-35b-turbo \
  -ngl 99 -fa on -ctk f16 -ctv f16 \
  -c 131072 -np 32 -b 8192 -ub 4096 \
  --host 0.0.0.0 --port 8092 --jinja

Single deep-agent profile:

GGML_SYCL_DISABLE_DNN=1 ONEAPI_DEVICE_SELECTOR=level_zero:gpu \
llama-server \
  -m ornith-1.0-35b-Q5_K_M.gguf \
  --alias ornith-1.0-35b-turbo \
  -ngl 99 -fa on -ctk f16 -ctv f16 \
  -c 262144 -np 1 -b 8192 -ub 4096 \
  --host 0.0.0.0 --port 8092 --jinja

Avoid -np >= 56 on a single B70 in the measured fleet harness; it thrashed or timed out under the tested workload.

Measured Performance

Hardware and stack:

• —GPU: Intel Arc Pro B70, 30.3 GiB, 230 W
• —Runtime: llama.cpp SYCL
• —Quant: Q5KM
• —KV cache in the final ship profile: f16

The benchmark split compares:

• —upstream: mainline llama.cpp, default flags
• —up+flags: mainline llama.cpp plus B70 runtime flags
• —Turbo: B70 fusion build plus the same runtime flags

Prefill

The tuned runtime configuration provides most of the prefill win.

Single-Stream Decode

The B70 fusion build provides the single-stream decode gain.

Fleet Decode

Aggregate decode with a synthetic 2048+256 workload:

Accuracy Snapshot

These are reference lm-eval results carried over from the local benchmark set; serving changes are lossless for the same GGUF.

Notes

• —This release is for llama.cpp-compatible GGUF runtimes.
• —The best measured B70 profile uses f16 KV. q8_0 KV was slower at long depth on this SYCL backend in local testing.
• —The route-aware DeepSpec/Eagle3 speculative-head work is tracked separately and is not included in this artifact.