Modelpublic

XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-v1

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

8likes491downloads

Model Card

[image]

Qwopus3.6-27B-v2 GPTQ-Pro FOEM 4-bit g128 ns256 v2

<p style="margin: 10px 0 6px 0; color: #475569; font-size: 14px; line-height: 1.5;"> These models are built and maintained on rented GPU compute. If you want to show some appreciation, a follow on X or a coffee helps keep the releases coming. </p>

<div style="display: flex; flex-wrap: wrap; align-items: center; gap: 10px; margin: 8px 0 18px 0;"> <a href="https://x.com/xreyrobert" target="blank" rel="noopener noreferrer" class="follow-link" style="display: inline-flex; align-items: center; justify-content: center; gap: 8px; min-height: 36px; box-sizing: border-box; padding: 9px 14px; border: 1px solid #111827; border-radius: 999px; background: #111827; color: #ffffff; text-decoration: none; font-weight: 700; font-size: 14px; line-height: 1;"> <svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true" style="width: 16px; height: 16px; flex: 0 0 auto;"><path d="M18.244 2.25h3.308l-7.227 8.26 8.502 11.24H16.17l-5.214-6.817L4.99 21.75H1.68l7.73-8.835L1.254 2.25H8.08l4.713 6.231zm-1.161 17.52h1.833L7.084 4.126H5.117z"></path></svg> Follow @xreyrobert </a> <a href="https://donate.stripe.com/bJe5kDbkwaYn3Jq9uL18c00" target="blank" rel="noopener noreferrer" class="support-link" style="display: inline-flex; align-items: center; justify-content: center; gap: 8px; min-height: 36px; box-sizing: border-box; padding: 9px 14px; border: 1px solid #f59e0b; border-radius: 999px; background: #fbbf24; color: #111827; text-decoration: none; font-weight: 700; font-size: 14px; line-height: 1;"> Support this work </a> </div>

This is a GPTQ-Pro 4-bit quantization of Jackrong/Qwopus3.6-27B-v2, built to make this excellent Qwopus/Qwen3.6 model practical to run in vLLM with GPTQ-Marlin kernels and long-context inference.

The goal is simple: preserve as much of the original model's character and capability as possible while making it efficient enough for single-GPU RTX 3090-class vLLM deployments.

This is not a new fine-tune. It is a quantized derivative of the original Qwopus3.6-27B-v2 model.

Source and credits

Source model:

—Jackrong/Qwopus3.6-27B-v2

Quantization methodology and reference recipe:

Thanks to Jackrong for the original Qwopus3.6 model, and to groxaxo for GPTQ-Pro and the Qwen3.6 GPTQ-Pro recipe this quantization was aligned with.

Quantization recipe

Setting	Value
Method	GPTQ-Pro / GPTQModel
Bits	`4`
Group size	`128`
Symmetric quantization	`true`
Desc act	`false`
True sequential	`true`
Calibration dataset	WikiText-2 raw train
Calibration samples	`256`
Sequence length	`2048`
MSE	`2.0`
Damp percent	`0.05`
Damp auto increment	`0.01`
FOEM alpha	`0.25`
FOEM beta	`0.2`
Batch size	`1`

Preserved modules include vision, lm_head, embeddings, and norms.

Validation showed that this artifact preserves MTP-related configuration metadata, but does not include actual mtp.* tensors in model.safetensors.index.json, so this release should be treated as non-MTP for vLLM speculative decoding.

Post-save compatibility patch:

—pad_token_id=248055
—tokenizer class patched to Qwen2TokenizerFast when needed for vLLM compatibility

Intended serving setup

This checkpoint is intended for text-only vLLM serving on RTX 3090-class hardware.

Recommended vLLM options:

bash

vllm serve XReyRobert/Qwopus3.6-27B-v2-GPTQ-Pro-FOEM-4bit-g128-ns256-v2 \
  --served-model-name qwopus3.6-27b-v2-gptq-pro-foem-4bit-g128-ns256-v2 \
  --language-model-only \
  --dtype float16 \
  --quantization gptq_marlin \
  --disable-custom-all-reduce \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --max-num-seqs 1 \
  --kv-cache-dtype fp8_e5m2 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --enable-prefix-caching \
  --max-cudagraph-capture-size 32 \
  --gpu-memory-utilization 0.95 \
  --trust-remote-code

Serving context for the published vLLM measurements:

The vLLM numbers were collected on an internal llm-residency deployment. The custom image recipe is not published yet, so this card does not present that image as a public reproduction target. The stable serving knobs captured from the run are listed for context.

Field	Value
Nomad job profile	`vllm-qwopus36-base`
Served model name	`qwopus3.6-27b-v2-gptq-pro-foem-4bit-g128-ns256-v2`
Critical flags	`--dtype float16`, `--quantization gptq_marlin`, `--kv-cache-dtype fp8_e5m2`, `--reasoning-parser qwen3`, `--tool-call-parser qwen3_coder`, `--max-model-len 131072`; `--max-num-batched-tokens` was not set explicitly.

Public vLLM Reproducibility

This artifact has a public reproducibility path on the unmodified upstream vLLM OpenAI image:

—image: docker.io/vllm/vllm-openai:nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa8
—vLLM version observed in validation: 0.20.1rc1.dev16+g7a1eb8ac2
—GPU class: single RTX 3090 24 GB / Ampere
—--enforce-eager was not used
—no local sleep/wake patch or localhost/*sleepwake* image is required for the validation below

Validated serving shape:

—context: --max-model-len 131072
—--language-model-only, --dtype float16, --quantization gptq_marlin
—--kv-cache-dtype fp8_e5m2, --enable-prefix-caching, --max-num-seqs 1
—--max-cudagraph-capture-size 32, --gpu-memory-utilization 0.95
—--reasoning-parser qwen3, --tool-call-parser qwen3_coder
—--enable-sleep-mode was included in the validation command

Startup note: this dense 27B profile can fail the first cold start after torch compile/profiling with a pessimistic KV-cache check. The public validation passed on the second start when reusing persistent vLLM/Nomad-style cache directories such as TORCHINDUCTOR_CACHE_DIR=/data/vllm-qwopus36-base/torch_compile_cache and VLLM_CACHE_ROOT=/data/vllm-qwopus36-base. Treat startup retry plus persistent compile cache as part of the serving recipe.

Reasoning / thinking mode

This model preserves Qwen3-style reasoning behavior. The validation workload below was run with thinking enabled.

MTP / speculative decoding status

This ns256-v2 artifact should be considered text-only and non-MTP for vLLM speculative decoding as published. config.json advertises mtp_num_hidden_layers=1, but the weight index does not contain source mtp.* tensors. Enabling vLLM MTP against this unpatched artifact produced essentially zero accepted draft tokens and poor throughput.

A separate experimental follow-up artifact restores real MTP tensors and quantizes the large MTP linears:

text

XReyRobert/Qwopus3.6-27B-v2-MTP-GPTQ-Pro-v1

That MTP-GPTQ artifact works and reaches good draft acceptance, but it was still slower than this non-MTP baseline on a single RTX 3090. For practical 100k-131k serving on 1x RTX 3090, this ns256-v2 non-MTP artifact remains the preferred choice.

RTX 3090 validation status

This checkpoint was validated on an RTX 3090 24GB with vLLM, max_model_len=131072, kv_cache_dtype=fp8_e5m2, prefix caching enabled, and thinking enabled.

Observed vLLM multi-turn agent workload metrics:

Metric	Observed value	Notes
Requests observed	`15`	Multi-turn agent session calls
vLLM request success count	`15/15`	No vLLM errors observed during the sample
Average prompt size	`33,172` tokens	Real multi-turn workload
Average output size	`322` tokens	Real generated responses
Average time to first token	`5.70s`	Prometheus TTFT summary
Average end-to-end request latency	`13.07s`	Includes prefill, decode, and serving overhead
Average time per output token	`0.0230s/token`	vLLM TPOT summary
Decode throughput from TPOT	about `43.5 tok/s`	Decode-only estimate
Prefix cache hit ratio	`83.2%` cumulative	vLLM prefix-cache counters
Live 60s prompt throughput	about `1,917 prompt tok/s`	Aggregate observed window
Live 60s generation throughput	about `19.1 generated tok/s`	Aggregate over full window, including prefill and idle mix
Live 60s prefix-cache hit ratio	`78.9%`	Delta over the observed window

These are practical multi-turn serving metrics, not a synthetic benchmark. They are useful for RTX 3090-class long-context serving expectations, especially multi-turn usage with prefix caching.

📊 4. Evaluation & Benchmarks

<div style="font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; border: 1px solid #cbd5e1; border-radius: 16px; box-shadow: 0 10px 15px -3px rgba(0, 0, 0, 0.05), 0 4px 6px -2px rgba(0, 0, 0, 0.05); overflow: hidden; background: #ffffff; margin-bottom: 30px;"> <div style="background: linear-gradient(135deg, #7c3aed 0%, #4f46e5 100%); padding: 20px; color: white;"> <h3 style="margin: 0; font-size: 20px; font-weight: 700; display: flex; align-items: center; gap: 8px; color: white; border: none;">📊 Evaluation & Performance Metrics</h3> <p style="margin: 4px 0 0 0; font-size: 13px; color: #ddd6fe;">24 May update: MMLU-Pro selected subset quality check for the GPTQ-Pro ns256 quantization, plus RTX 3090 vLLM serving metrics.</p> </div>

<div style="padding: 24px; display: flex; flex-direction: column; gap: 24px;"> <div style="display: grid; grid-template-columns: repeat(auto-fit, minmax(180px, 1fr)); gap: 16px;"> <div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);"> <span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">📚 MMLU-Pro Subset</span> <span style="font-size: 24px; font-weight: 800; color: #1e293b; display: block;">90.57%</span> <span style="font-size: 11px; color: #64748b; font-weight: 500;">317 / 350 resolved local run</span> </div> <div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);"> <span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">🧠 vs Qwopus BF16</span> <span style="font-size: 24px; font-weight: 800; color: #10b981; display: block;">+3.14 pp</span> <span style="font-size: 11px; color: #64748b; font-weight: 500;">306 / 350 reference CSV</span> </div> <div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);"> <span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">⚖️ vs Qwen3.6 BF16</span> <span style="font-size: 24px; font-weight: 800; color: #10b981; display: block;">+5.71 pp</span> <span style="font-size: 11px; color: #64748b; font-weight: 500;">297 / 350 reference CSV</span> </div> <div style="border: 1px solid #e2e8f0; padding: 16px; border-radius: 10px; background: #f8fafc; text-align: center; box-shadow: inset 0 2px 4px rgba(0,0,0,0.02);"> <span style="font-size: 11px; font-weight: 700; color: #7c3aed; text-transform: uppercase; display: block; margin-bottom: 6px; letter-spacing: 0.5px;">⚡ RTX 3090 vLLM</span> <span style="font-size: 24px; font-weight: 800; color: #1e293b; display: block;">43.48</span> <span style="font-size: 11px; color: #64748b; font-weight: 500;">completion tok/s, request wall-time</span> </div> </div>

<div style="background: #eff6ff; border: 1px solid #bfdbfe; border-radius: 8px; padding: 12px 14px; font-size: 13px; color: #1e3a8a; line-height: 1.6;"> <b>Protocol note:</b> The primary <b>317 / 350 = 90.57%</b> score is rescored with the official MMLU-Pro <code>computeaccuracy.py</code> L2 answer extractor, without random fallback. The generation prompt follows the MMLU-Pro <code>rungpt4o.py</code> OpenAI-compatible style (<code>Options are:</code>, <code>(A):</code>, and the system instruction asking for <code>The answer is ...</code>); it is not borrowed from the Qwopus model card or test set. The current MMLU-Pro <code>evaluatefromapiX.py</code> extractor, which selects the last matching answer, scores the same outputs at <b>320 / 350 = 91.43%</b>. This card reports the conservative L2/no-random-fallback score as the headline. </div>

<div style="overflow-x: auto; width: 100%;"> <table style="display: table; width: 100%; min-width: 100%; table-layout: fixed; border-collapse: collapse; font-size: 13px;"> <thead> <tr style="background: rgba(124, 58, 237, 0.05);"> <th style="width: 34%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: left; color: #7c3aed; font-weight: bold;">Model / run</th> <th style="width: 18%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Correct / Total</th> <th style="width: 16%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Accuracy</th> <th style="width: 16%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Δ vs Qwen</th> <th style="width: 16%; padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Δ vs Qwopus</th> </tr> </thead> <tbody> <tr style="background: rgba(16, 185, 129, 0.06);"> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 700; color: #047857;">This GPTQ-Pro ns256 artifact</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #047857;">317 / 350</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 800; color: #10b981; font-size: 14px;">90.57%</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #10b981;">+5.71 pp</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #10b981;">+3.14 pp</td> </tr> <tr> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Qwopus3.6-27B-v2 reference CSV</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">306 / 350</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: bold;">87.43%</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #10b981; font-weight: 700;">+2.57 pp</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #64748b;">baseline</td> </tr> <tr> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Qwen3.6-27B-v2 reference CSV</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">297 / 350</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: bold;">84.86%</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #64748b;">baseline</td> <td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; color: #dc2626;">-2.57 pp</td> </tr> </tbody> </table> </div>

<div style="overflow-x: auto; width: 100%;"> <table style="display: table; width: 100%; min-width: 100%; table-layout: fixed; border-collapse: collapse; font-size: 12px;"> <thead> <tr style="background: rgba(124, 58, 237, 0.03);"> <th style="width: 28%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: left;">Category</th> <th style="width: 22%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">Qwen3.6-27B</th> <th style="width: 25%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #7c3aed;">Qwopus3.6-27B-v2</th> <th style="width: 25%; padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; color: #047857;">This GPTQ-Pro ns256</th> </tr> </thead> <tbody> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Biology</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">96%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #7c3aed;">96%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #047857;">94%</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Business</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">88%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #7c3aed;">94%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">96%</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Computer Science</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">82%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #7c3aed;">84%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #047857;">82%</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Mathematics</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">90%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #7c3aed;">88%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">98%</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Physics</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">76%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #7c3aed;">86%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">92%</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Chemistry</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">74%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #7c3aed;">80%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">86%</td></tr> <tr><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; font-weight: 600;">Health</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right;">88%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #7c3aed;">84%</td><td style="padding: 7px 8px; border-bottom: 1px solid #e2e8f0; text-align: right; font-weight: 700; color: #10b981;">86%</td></tr> </tbody> </table> </div>

<p style="margin: 0; font-size: 13px; color: #475569; line-height: 1.6;"> <b>Summary:</b> On the selected 350-question MMLU-Pro evaluation set, this GPTQ-Pro ns256 artifact reached <b>90.57%</b> accuracy using the official MMLU-Pro L2 answer extractor without random fallback. This is above the published Qwopus3.6-27B-v2 reference CSV at <b>87.43%</b> and Qwen3.6-27B-v2 at <b>84.86%</b>, but the comparison should be treated cautiously because the local result is a resolved mixed run rather than a single-pass uniform evaluation, and it uses the MMLU-Pro <code>rungpt4o.py</code> prompt style rather than the current local/vLLM prompt template. </p> <div style="display: inline-flex; align-items: center; width: fit-content; background: #ffffff; border: 1px solid #e2e8f0; border-radius: 999px; padding: 6px 10px; font-size: 11px; color: #64748b; font-weight: 600; box-shadow: 0 1px 2px rgba(15,23,42,0.04);"> Scope note: non-truncated answers were kept, and only length-truncated cases were regenerated with larger output caps, up to maxtokens=16384. Prompts and instructions were not changed. Three pathological cases still ended with finish_reason=length and no parsed answer. For leaderboard-style comparability, the preferred next run is a single-pass full MMLU-Pro evaluation generated from the official GitHub scripts and submitted in the JSON/CSV format expected by the MMLU-Pro HF Space. </div> </div> </div>

<div style="border: 1px solid #cbd5e1; border-radius: 12px; overflow: hidden; box-shadow: 0 2px 4px rgba(0,0,0,0.02);"> <div style="background: linear-gradient(135deg, #f8fafc 0%, #f1f5f9 100%); padding: 12px 16px; border-bottom: 1px solid #cbd5e1; font-weight: 700; font-size: 14px; color: #1e293b; display: flex; align-items: center; gap: 8px;"> <span>⚡</span> 4.2 RTX 3090 vLLM Runtime Notes </div> <div style="padding: 16px;"> <div style="overflow-x: auto;"> <table style="width: 100%; border-collapse: collapse; font-size: 13px; min-width: 500px;"> <thead> <tr style="background: rgba(124, 58, 237, 0.05);"> <th style="padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: left; color: #7c3aed; font-weight: bold;">Metric</th> <th style="padding: 8px 10px; border-bottom: 2px solid #7c3aed; text-align: right; color: #7c3aed; font-weight: bold;">Observed value</th> </tr> </thead> <tbody> <tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Prompt tokens</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">509,541</td></tr> <tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Completion tokens</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">515,679</td></tr> <tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Request elapsed sum</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right;">11,859.8s</td></tr> <tr><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); font-weight: 600;">Completion throughput</td><td style="padding: 8px 10px; border-bottom: 1px solid rgba(128,128,128,0.15); text-align: right; font-weight: 700; color: #7c3aed;">43.48 tok/s</td></tr> </tbody> </table> </div> </div> </div> </div> </div>

Paired comparison on the same 350 question IDs is positive but should be treated cautiously because the subset is small and the local result is not a single-pass uniform run. Against Qwopus3.6-27B-v2, this run has 20 local-only correct answers and 9 Qwopus-only correct answers (+3.14 pp, McNemar p≈0.061). Against Qwen3.6-27B-v2, it has 32 local-only correct answers and 12 Qwen-only correct answers (+5.71 pp, McNemar p≈0.0037).

Compatibility notes

This artifact was built and validated for text-only vLLM serving without speculative decoding. Do not enable MTP on this artifact as published; the mtp.* tensors are absent from the weight index. Vision-related modules were not validated for vision use in this release.

Limitations

—Experimental quantization.
—MTP/speculative decoding is not supported by this published artifact because mtp.* tensors are missing.
—Quality has been checked on Jackrong's 350-question MMLU-Pro subset only; this is not a full MMLU-Pro evaluation or an official leaderboard submission.
—The subset result uses the official MMLU-Pro answer extractor, but the prompt style is the MMLU-Pro run_gpt4o.py OpenAI-compatible template, not the current local/vLLM template.
—RTX 3090 metrics above are observed workload numbers, not a controlled benchmark suite.
—Long-context and tool-calling workflows were validated on the described local vLLM/Hermes setup; behavior may vary on other serving stacks, hardware, or generation settings.

References

—Source model: Jackrong/Qwopus3.6-27B-v2
—GPTQ-Pro tooling: groxaxo/GPTQ-Pro
—Reference GPTQ-Pro recipe: groxaxo/Qwen3.6-27B-GPTQ-Pro-4bit
—MMLU-Pro benchmark repository: TIGER-AI-Lab/MMLU-Pro
—MMLU-Pro HF Space / leaderboard: TIGER-Lab/MMLU-Pro

Individual project notice

This repository is an individual research project. It is not affiliated with, sponsored by, or endorsed by any employer or organization.