XReyRobert/Qwopus3.6-35B-A3B-v1-GPTQ-Pro

[image]

Qwopus3.6-35B-A3B-v1 GPTQ-Pro

<p style="margin: 10px 0 6px 0; color: #475569; font-size: 14px; line-height: 1.5;"> These models are built and maintained on rented GPU compute. If you want to show some appreciation, a follow on X or a coffee helps keep the releases coming. </p>

<div style="display: flex; flex-wrap: wrap; align-items: center; gap: 10px; margin: 8px 0 18px 0;"> <a href="https://x.com/xreyrobert" target="blank" rel="noopener noreferrer" class="follow-link" style="display: inline-flex; align-items: center; justify-content: center; gap: 8px; min-height: 36px; box-sizing: border-box; padding: 9px 14px; border: 1px solid #111827; border-radius: 999px; background: #111827; color: #ffffff; text-decoration: none; font-weight: 700; font-size: 14px; line-height: 1;"> <svg viewBox="0 0 24 24" fill="currentColor" aria-hidden="true" style="width: 16px; height: 16px; flex: 0 0 auto;"><path d="M18.244 2.25h3.308l-7.227 8.26 8.502 11.24H16.17l-5.214-6.817L4.99 21.75H1.68l7.73-8.835L1.254 2.25H8.08l4.713 6.231zm-1.161 17.52h1.833L7.084 4.126H5.117z"></path></svg> Follow @xreyrobert </a> <a href="https://buymeacoffee.com/xrrxrr" target="blank" rel="noopener noreferrer" class="support-link" style="display: inline-flex; align-items: center; justify-content: center; gap: 8px; min-height: 36px; box-sizing: border-box; padding: 9px 14px; border: 1px solid #f59e0b; border-radius: 999px; background: #fbbf24; color: #111827; text-decoration: none; font-weight: 700; font-size: 14px; line-height: 1;"> Buy me a coffee </a> </div>

This is a GPTQ-Pro 4-bit quantization of `Jackrong/Qwopus3.6-35B-A3B-v1`.

It is a deployment artifact, not a new fine-tune. The goal is to keep the Qwopus/Qwen3.6 35B-A3B MoE checkpoint practical for GPTQ-compatible local serving while documenting exactly what has and has not been validated.

The recommended public positioning for this release is text-first: the source checkpoint is multimodal, and this artifact preserves vision/visual tensors, but vision serving has not yet been validated for this quantized checkpoint.

Source And Credits

Source model:

• —`Jackrong/Qwopus3.6-35B-A3B-v1`

Quantization tooling and reference recipe:

• —`groxaxo/GPTQ-Pro`
• —`groxaxo/Qwen3.6-27B-GPTQ-Pro-4bit`

Thanks to Jackrong for the Qwopus3.6 model and to groxaxo for the GPTQ-Pro tooling and Qwen3.6 GPTQ-Pro recipe this run was aligned with.

Artifact Summary

The artifact contains source MTP and vision/visual tensors in its weight index. That does not mean MTP speculative decoding or multimodal serving is already recommended. The validated use so far is text-oriented GPTQ serving and Terminal-Bench agent evaluation.

Quantization Recipe

Dynamic skip rules preserved these module families instead of quantizing them:

• —embed_tokens
• —lm_head
• —mtp
• —norm
• —vision
• —visual

In practical terms, the language tower linear layers are the intended GPTQ-Pro payload, while embeddings, norms, MTP, and vision-related tensors remain preserved as non-quantized tensors.

Intended Serving Shape

This checkpoint is intended for advanced users testing text-only vLLM or GPTQ-compatible serving for Qwen/Qwopus MoE checkpoints.

A starting vLLM shape for text-only testing:

vllm serve XReyRobert/Qwopus3.6-35B-A3B-v1-GPTQ-Pro \
  --served-model-name qwopus3.6-35b-a3b-v1-gptq-pro \
  --language-model-only \
  --dtype float16 \
  --quantization gptq_marlin \
  --tensor-parallel-size 1 \
  --max-model-len 262144 \
  --max-num-seqs 1 \
  --kv-cache-dtype fp8_e5m2 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --enable-prefix-caching \
  --gpu-memory-utilization 0.95 \
  --trust-remote-code

Serving context for the published Smoke24/vLLM measurements:

The Smoke24/vLLM numbers were collected on an internal llm-residency vLLM deployment. The custom image recipe is not published yet, so this card does not present that image as a public reproduction target. The stable serving knobs captured from the run are listed for context.

Treat the command as a serving starting point, not a compatibility guarantee for every vLLM release. GPTQ-Marlin, Qwen3.6 MoE handling, and multimodal processor behavior are all loader-version sensitive.

The RTX 3090 image above reflects separate 262k-context serving validation.

Public vLLM Reproducibility

This artifact has a public reproducibility path on the unmodified upstream vLLM OpenAI image:

• —image: docker.io/vllm/vllm-openai:nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa8
• —vLLM version observed in validation: 0.20.1rc1.dev16+g7a1eb8ac2
• —GPU class: single RTX 3090 24 GB / Ampere
• —--enforce-eager was not used
• —no local sleep/wake patch or localhost/*sleepwake* image is required for the validation below

Validated serving shapes:

• —--max-model-len 131072 validated with --gpu-memory-utilization 0.95
• —--max-model-len 262144 validated with --gpu-memory-utilization 0.96
• —--language-model-only, --dtype float16, --quantization gptq_marlin
• —--kv-cache-dtype fp8_e5m2, --enable-prefix-caching, --max-num-seqs 1
• —--max-num-batched-tokens 2096, --max-cudagraph-capture-size 32
• —--reasoning-parser qwen3, --tool-call-parser qwen3_coder

The 262k profile is tight on 24 GB GPUs; gpu_memory_utilization=0.95 was short on KV cache in validation, while 0.96 passed.

Validation And Benchmarks

Completed artifact checks:

• —Local shard index inspection completed before upload.
• —Remote file list verified after upload.
• —Remote model.safetensors.index.json verified after upload.
• —Index metadata total size matches the local safetensor shards.
• —The remote artifact contains the expected six safetensor shards.

Terminal-Bench 2.0 Smoke24 result and associated vLLM serving measurements. This Smoke24 run used max_model_len=131072 for apples-to-apples comparison with the other local models in this publication batch:

Smoke24 is a fixed 24-task Terminal-Bench 2.0 comparison corpus, not a full Terminal-Bench leaderboard run. The score above is useful for fast regression and local serving comparison, not for broad model ranking.

Task list and harness shape:

• —`benchmarks/terminal-bench-2.0/smoke24_task_list_20260616.md`

MTP And Vision Status

• —config.json advertises MTP support, and the index contains mtp.* tensors.
• —MTP tensors were preserved, not the primary quantization target for this release.
• —MTP speculative decoding has not yet been validated as a recommended path for this artifact.
• —Vision/visual tensors are present, but multimodal serving has not yet been validated for this quantized artifact.

For now, publish and use this as a text-first GPTQ-Pro MoE artifact.

Limitations

• —Experimental quantization.
• —Terminal-Bench Smoke24 is a small local comparison corpus, not a full benchmark submission.
• —MTP speculative decoding is not yet a supported recommendation for this artifact.
• —Vision tensors are preserved, but vision behavior has not been validated.
• —Loader behavior may vary across vLLM, Transformers, GPTQModel, and GPTQ-Marlin versions.

Files

Key files:

• —model.safetensors.index.json
• —model-00001-of-00006.safetensors through model-00006-of-00006.safetensors
• —config.json
• —quantize_config.json
• —processor_config.json
• —tokenizer.json
• —UPLOAD_MANIFEST.json

UPLOAD_MANIFEST.json records the upload guardrail checks and artifact inspection summary.

References

• —Source model: `Jackrong/Qwopus3.6-35B-A3B-v1`
• —GPTQ-Pro tooling: `groxaxo/GPTQ-Pro`
• —Reference recipe: `groxaxo/Qwen3.6-27B-GPTQ-Pro-4bit`
• —Terminal-Bench: `laude-institute/terminal-bench`

Individual Project Notice

This repository is an individual research project. It is not affiliated with, sponsored by, or endorsed by any employer or organization.