Modelpublic

XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256

sourceHugging Facemitupdated 1mo agoView on Hugging Face

5likes435downloads

Model Card

[image]

Ornith-1.0-35B GPTQ-Pro FOEM 4-bit g128 ns256

<p style="margin: 10px 0 6px 0; color: #475569; font-size: 14px; line-height: 1.5;"> These models are built and maintained on rented GPU compute. If you want to show some appreciation, a follow on X or a coffee helps keep the releases coming. </p>

<div style="display: flex; flex-wrap: wrap; align-items: center; gap: 10px; margin: 8px 0 18px 0;"> <a href="https://x.com/xreyrobert" target="blank" rel="noopener noreferrer" class="follow-link" style="display: inline-flex; align-items: center; justify-content: center; gap: 8px; min-height: 36px; box-sizing: border-box; padding: 9px 14px; border: 1px solid #111827; border-radius: 999px; background: #111827; color: #ffffff; text-decoration: none; font-weight: 700; font-size: 14px; line-height: 1;"> <svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true" style="width: 16px; height: 16px; flex: 0 0 auto;"><path d="M18.244 2.25h3.308l-7.227 8.26 8.502 11.24H16.17l-5.214-6.817L4.99 21.75H1.68l7.73-8.835L1.254 2.25H8.08l4.713 6.231zm-1.161 17.52h1.833L7.084 4.126H5.117z"></path></svg> Follow @xreyrobert </a> <a href="https://donate.stripe.com/bJe5kDbkwaYn3Jq9uL18c00" target="blank" rel="noopener noreferrer" class="support-link" style="display: inline-flex; align-items: center; justify-content: center; gap: 8px; min-height: 36px; box-sizing: border-box; padding: 9px 14px; border: 1px solid #f59e0b; border-radius: 999px; background: #fbbf24; color: #111827; text-decoration: none; font-weight: 700; font-size: 14px; line-height: 1;"> Support this work </a> </div>

This is a GPTQ-Pro 4-bit quantization of `deepreinforce-ai/Ornith-1.0-35B`.

It is a deployment artifact, not a new fine-tune. The goal is to make the Ornith 35B MoE checkpoint practical for GPTQ-compatible local and homelab serving while preserving the coding-agent oriented behavior of the source model.

The headline validation target for this release is text-generation and agentic coding workload serving with vLLM + GPTQ-Marlin. Vision paths and speculative decoding are not positioned as validated features for this release.

Source And Credits

Source model:

—`deepreinforce-ai/Ornith-1.0-35B`

Quantization tooling and reference recipe:

Thanks to deepreinforce-ai for the Ornith release, to modelcloud for GPTQModel, and to groxaxo for GPTQ-Pro and the Qwen3.6 GPTQ-Pro recipe family this run was aligned with.

Artifact Summary

Field	Value
Source model	`deepreinforce-ai/Ornith-1.0-35B`
Architecture	`Qwen3_5MoeForConditionalGeneration`
Model type	`qwen3_5_moe`
Hidden layers	`40`
Hidden size	`2048`
Experts	`256`
Experts per token	`8`
Max position embeddings	`262144`
Tensor files	`5`
Indexed tensors	`124576`
Indexed total size	`20,652,683,992` bytes
`mtp.*` tensors in index	`0`
Tokenizer class	`Qwen2TokenizerFast`

The source config contains mtp_num_hidden_layers=1, but the uploaded weight index contains no mtp.* tensors. Treat this checkpoint as non-MTP for speculative decoding unless a follow-up artifact restores and validates real MTP tensors.

Vision configuration is present from the source architecture, but multimodal serving has not been validated for this quantized release. The intended use case is text generation and coding-agent evaluation.

Quantization Recipe

Setting	Value
Method	GPTQ-Pro / GPTQModel
Quantizer	`gptqmodel:6.1.0-dev`
Bits	`4`
Group size	`128`
Symmetric quantization	`true`
Desc act	`false`
True sequential	`true`
Checkpoint format	`gptq`
Pack dtype	`int32`
Pack implementation	`cpu`
MSE	`2.0`
Damp percent	`0.05`
Damp auto increment	`0.01`
FOEM alpha	`0.25`
FOEM beta	`0.2`
Calibration samples	`256`
Calibration sequence length	`2048`
Calibration mix	code-oriented local JSONL mix
MoE routing during quantization	`ExpertsRoutingBypass`
MoE routing batch size	`64`

Excluded from quantization by dynamic rules:

—embeddings
—lm_head
—MTP modules
—norms
—vision / visual modules

The calibration data itself is not included in this model repository.

Intended Serving Shape

This checkpoint is intended for advanced users testing text-only vLLM or GPTQ-compatible serving for Qwen/Ornith MoE checkpoints.

A starting vLLM shape for long-context text serving:

bash

vllm serve XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256 \
  --served-model-name ornith-1.0-35b-gptq-pro-foem-4bit-g128-ns256-ctx262k \
  --language-model-only \
  --dtype float16 \
  --quantization gptq_marlin \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --max-num-seqs 1 \
  --kv-cache-dtype fp8_e5m2 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --enable-prefix-caching \
  --gpu-memory-utilization 0.95 \
  --trust-remote-code

Serving context for the published Smoke24/vLLM measurements:

The Smoke24/vLLM numbers were collected on an internal llm-residency vLLM deployment. The custom image recipe is not published yet, so this card does not present that image as a public reproduction target. The stable serving knobs captured from the run are listed for context.

Field	Value
Nomad job profile	`vllm-ornith-35b-gptq-pro-262k`
Served model name	`ornith-1.0-35b-gptq-pro-foem-4bit-g128-ns256-ctx262k`
Critical flags	`--dtype float16`, `--quantization gptq_marlin`, `--kv-cache-dtype fp8_e5m2`, `--reasoning-parser qwen3`, `--tool-call-parser qwen3_xml`, `--max-model-len 262144`, `--max-num-batched-tokens 2096`

Treat this as a serving starting point, not a compatibility guarantee for every vLLM release. GPTQ-Marlin, Qwen3.5 MoE handling, FP8 KV cache, and tool parser behavior are loader-version sensitive.

Public vLLM Reproducibility

This artifact has a public reproducibility path on the unmodified upstream vLLM OpenAI image:

—image: docker.io/vllm/vllm-openai:nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa8
—vLLM version observed in validation: 0.20.1rc1.dev16+g7a1eb8ac2
—GPU class: single RTX 3090 24 GB / Ampere
—--enforce-eager was not used
—no local sleep/wake patch or localhost/*sleepwake* image is required for the validation below

Validated serving shape:

—context: --max-model-len 262144 with --gpu-memory-utilization 0.96
—--language-model-only, --dtype float16, --quantization gptq_marlin
—--kv-cache-dtype fp8_e5m2, --enable-prefix-caching, --max-num-seqs 1
—--max-num-batched-tokens 2096, --max-cudagraph-capture-size 32
—--reasoning-parser qwen3, --tool-call-parser qwen3_xml

The 262k profile is tight on 24 GB GPUs; gpu_memory_utilization=0.95 was short on KV cache in validation, while 0.96 passed.

vLLM RTX 3090 / Ampere Workload Snapshot

The image above reports observed vLLM / Prometheus metrics for the served model label ornith-1.0-35b-gptq-pro-foem-4bit-g128-ns256-ctx262k over the recent workload window that included validation traffic:

Metric	Value
Context budget	`262k`
Decode-only throughput from TPOT	`~151 tok/s`
Prefill throughput	`~4,441 tok/s`
Prefix cache hit ratio	`89.5%`
Average TTFT	`0.63s`
Average E2E latency	`10.20s`

These are serving metrics, not a standalone quality benchmark. They describe the observed vLLM GPTQ-Marlin runtime behavior for this deployment shape.

MMLU-Pro 350 Selected Subset

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; border: 1px solid #cbd5e1; border-radius: 16px; box-shadow: 0 10px 15px -3px rgba(0, 0, 0, 0.05), 0 4px 6px -2px rgba(0, 0, 0, 0.05); overflow: hidden; background: #ffffff; margin-bottom: 30px;"> <div style="background: linear-gradient(135deg, #7c3aed 0%, #4f46e5 100%); padding: 20px; color: white;"> <h3 style="margin: 0; font-size: 20px; font-weight: 700; display: flex; align-items: center; gap: 8px; color: white; border: none;">📊 Evaluation & Performance Metrics</h3> <p style="margin: 4px 0 0 0; font-size: 13px; color: #ddd6fe;">2 July update: single-pass MMLU-Pro selected subset quality check for the XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256 deployment, using the same 350-question subset and evaluation harness used for the XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 selected-subset comparison.</p> </div>

<div style="padding: 24px; display: flex; flex-direction: column; gap: 24px;"> <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); gap: 16px;"> <div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);"> <span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">📚 MMLU-Pro Subset</span> <span style="font-size: 24px; font-weight: 800; color: #1e293b; display: block;">90.86%</span> <span style="font-size: 11px; color: #64748b; font-weight: 500;">318 / 350 single-pass run</span> </div> <div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);"> <span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">🧠 vs XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1</span> <span style="font-size: 24px; font-weight: 800; color: #10b981; display: block;">+0.86 pp</span> <span style="font-size: 11px; color: #64748b; font-weight: 500;">315 / 350 selected-subset run</span> </div> <div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);"> <span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">⚖️ vs Qwopus CSV</span> <span style="font-size: 24px; font-weight: 800; color: #10b981; display: block;">+3.43 pp</span> <span style="font-size: 11px; color: #64748b; font-weight: 500;">306 / 350 reference CSV</span> </div> <div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);"> <span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">⚡ vLLM Decode</span> <span style="font-size: 24px; font-weight: 800; color: #1e293b; display: block;">150.68</span> <span style="font-size: 11px; color: #64748b; font-weight: 500;">completion tok/s, request wall-time</span> </div> </div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #f8fafc 0%, #f1f5f9 100%); padding: 12px 16px; border-bottom: 1px solid #cbd5e1; font-weight: 700; font-size: 14px; color: #1e293b; display: flex; align-items: center; gap: 8px;"> <span>📚</span> MMLU-Pro Selected Subset - 2 July update </div> <div style="padding: 16px; display: flex; flex-direction: column; gap: 16px;"> <div style="background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 8px; padding: 12px 14px; font-size: 13px; color: #334155; line-height: 1.6;"> <b>Evaluation format:</b> This uses the same 350-question MMLU-Pro subset published in the <code>test_data</code> directory of <code>Jackrong/Qwopus3.6-27B-v2</code>: 7 categories, 50 questions per category. This is not a full MMLU-Pro leaderboard run. </div>

<div style="background: #eff6ff; border: 1px solid #bfdbfe; border-radius: 8px; padding: 12px 14px; font-size: 13px; color: #1e3a8a; line-height: 1.6;"> <b>Protocol note:</b> The headline <b>318 / 350 = 90.86%</b> score is a single-pass OpenAI-compatible run through the published vLLM/LiteLLM serving path. The run used <code>temperature=1.0</code>, <code>topp=0.95</code>, <code>maxtokens=0</code> (no explicit output cap sent by the runner), <code>requesttimeout=3600</code>, and <code>retries=1</code>. The prompt follows the MMLU-Pro 5-shot category-prefix style using validation split <code>cotcontent</code> and the system instruction asking for <code>The answer is ...</code>. All 350 requests completed with <code>finish_reason=stop</code>; one response had no parsed answer and is counted as wrong. There were no API errors. </div>

<div style="overflow-x: auto; width: 100%;"> <table style="display: table; width: 100%; min-width: 100%; table-layout: fixed; border-collapse: collapse; font-size: 13px;"> <thead> <tr style="background: rgba(124, 58, 237, 0.05);"> <th style="width: 42%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: left; color: #7c3aed; font-weight: bold;">Model / run</th> <th style="width: 18%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Correct / Total</th> <th style="width: 16%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Accuracy</th> <th style="width: 24%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Notes</th> </tr> </thead> <tbody> <tr style="background: rgba(16, 185, 129, 0.06);"> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 700; color: #047857;">XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #047857;">318 / 350</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 800; color: #10b981; font-size: 14px;">90.86%</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #047857;">single-pass unrestricted</td> </tr> <tr> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 selected-subset run</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">315 / 350</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: bold;">90.00%</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #10b981; font-weight: 700;">Ornith +0.86 pp</td> </tr> <tr> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Qwopus3.6-27B-v2 reference CSV</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">306 / 350</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: bold;">87.43%</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #10b981; font-weight: 700;">Ornith +3.43 pp</td> </tr> <tr> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Qwen3.6-27B-v2 reference CSV</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">297 / 350</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: bold;">84.86%</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #10b981; font-weight: 700;">Ornith +6.00 pp</td> </tr> </tbody> </table> </div>

<div style="overflow-x: auto; width: 100%;"> <table style="display: table; width: 100%; min-width: 100%; table-layout: fixed; border-collapse: collapse; font-size: 12px;"> <thead> <tr style="background: rgba(124, 58, 237, 0.03);"> <th style="width: 34%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: left;">Category</th> <th style="width: 22%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #047857;">XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256</th> <th style="width: 22%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #7c3aed;">XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1</th> <th style="width: 22%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">Delta</th> </tr> </thead> <tbody> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Biology</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #047857;">48 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">48 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">0</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Business</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">47 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">45 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #10b981; font-weight: 700;">+2</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Chemistry</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #dc2626;">44 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">45 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #dc2626; font-weight: 700;">-1</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Computer Science</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #047857;">40 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">40 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">0</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Health</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #047857;">42 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">42 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">0</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Mathematics</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">49 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">48 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #10b981; font-weight: 700;">+1</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Physics</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">48 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">47 / 50</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #10b981; font-weight: 700;">+1</td></tr> </tbody> </table> </div>

<div style="overflow-x: auto; width: 100%;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px; min-width: 500px;"> <thead> <tr style="background: rgba(124, 58, 237, 0.05);"> <th style="padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: left; color: #7c3aed; font-weight: bold;">Runtime metric</th> <th style="padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Observed value</th> </tr> </thead> <tbody> <tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Prompt tokens</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">509,541</td></tr> <tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Completion tokens</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">860,087</td></tr> <tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Request elapsed sum</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">5,708.0s</td></tr> <tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Completion throughput</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #7c3aed;">150.68 tok/s</td></tr> </tbody> </table> </div>

<p style="margin: 0; font-size: 13px; color: #475569; line-height: 1.6;"> <b>Summary:</b> On the selected 350-question MMLU-Pro evaluation set, XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256 reached <b>90.86%</b> accuracy in a single unrestricted pass. On the same 350-question selected subset, the prior XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1 unrestricted reference run reached <b>90.00%</b>. Treat the difference as a small-sample local validation signal rather than a leaderboard claim. </p> <div style="display: inline-flex; align-items: center; width: fit-content; background: #ffffff; border: 1px solid #e2e8f0; border-radius: 999px; padding: 6px 10px; font-size: 11px; color: #64748b; font-weight: 600; box-shadow: 0 1px 2px rgba(15,23,42,0.04);"> Scope note: this is not a full MMLU-Pro evaluation or official leaderboard submission. It is a selected-subset regression/quality check over the public 350-question selected subset used on the Qwopus card. </div> </div> </div> </div> </div>

Terminal-Bench 2.0 Smoke24

Terminal-Bench 2.0 Smoke24 is a fixed 24-task coding-agent comparison corpus. It is useful for fast regression and local serving comparison, but it is not a full Terminal-Bench leaderboard submission.

XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256 used the long-context card-validation shape: max_model_len=262144, max_input_tokens=220000, 30 minute task timeout, 32 CPU / 48 GiB sandbox, thinking_token_budget=32768, max_output_tokens=40000, temperature 1.0, top-p 0.95, top-k 20, and preserve_thinking=true.

Run	Score	Success rate	Wall-time	Output tokens	Observed decode	LLM API time
`ornith-1.0-35b-gptq-pro-foem-4bit-g128-ns256-ctx262k`	`17/24`	`70.8%`	`186.4m`	`498.2k`	`137.6 tok/s`	`60.3m`

Smoke24 task list and harness summary:

—`benchmarks/terminal-bench-2.0/smoke24_task_list_20260616.md`

Validation Status

Completed:

—Source model loaded for quantization.
—Code-oriented calibration mix generated.
—GPTQ-Pro FOEM quantization completed.
—Final checkpoint saved as five safetensors shards.
—Hugging Face repo file list verified after upload.
—vLLM serving metrics collected through Prometheus.
—Terminal-Bench 2.0 Smoke24 validation run completed.

Not yet validated:

—Full Terminal-Bench leaderboard submission.
—Vision or multimodal serving.
—MTP/speculative decoding.
—Broad multi-GPU serving matrix.

Limitations

—Experimental quantization.
—Smoke24 is a small 24-task slice; read one-task differences with caution.
—The calibration mix is code-heavy and was chosen for coding-agent use, not as a general-purpose calibration corpus.
—Treat this release as non-MTP for speculative decoding.
—Vision support is not validated.
—Loader behavior may vary across vLLM, Transformers, GPTQModel, and GPTQ-Marlin versions.

Files

Key files:

—model.safetensors.index.json
—model-00001-of-00005.safetensors through model-00005-of-00005.safetensors
—config.json
—generation_config.json
—processor_config.json
—quantize_config.json
—tokenizer.json
—tokenizer_config.json
—chat_template.jinja

References

—Source model: `deepreinforce-ai/Ornith-1.0-35B`
—GPTQModel: `modelcloud/gptqmodel`
—GPTQ-Pro tooling: `groxaxo/GPTQ-Pro`
—Reference recipe: `groxaxo/Qwen3.6-27B-GPTQ-Pro-4bit`
—Terminal-Bench: `laude-institute/terminal-bench`

Individual Project Notice

This repository is an individual research project. It is not affiliated with, sponsored by, or endorsed by any employer or organization.