XReyRobert/gptq-pro-smoke24-agentic-3090

GPTQ-Pro Smoke24 Agentic 3090 Dashboard

Static dashboard for quickly visualizing GPTQ-Pro model positioning on Terminal-Bench 2.0 Smoke24 in a local RTX 3090-class agentic workload context.

Smoke24 is a fixed 24-task Terminal-Bench 2.0 slice selected as 12 shortest prior successes and 12 shortest prior failures from the recovery-corrected Qwopus3.6-27B-v2-GPTQ-Pro-v1 aggregate. It is intended as a fast local-serving regression and positioning lens, not as a replacement for a full Terminal-Bench leaderboard submission.

The default dashboard chart keeps the maximum served context row for each local model family. Qwen3.8 low, medium, and xhigh are retained as separate rows because they are a controlled reasoning-effort comparison on the same 210K deployment. Context-specific rows remain available through the standard, long-context, and all-row filters.

The separate Qwen3.8 syv 245K row is a runtime-profile comparison using the syv AutoRound W4A16 checkpoint, DFlash2, and KVarN. Its patched vLLM endpoint did not accept thinking_token_budget, so the 78,750-token combined output cap was the only hard generation limit. It must not be read as a controlled quantization-only comparison with the three GPTQ-Pro effort rows.

Terminology used in the dashboard:

• —Model serving time: cumulative time spent waiting on LLM/vLLM calls, reported per solved task.
• —End-to-end task time: full benchmark elapsed time, including agent orchestration, shell/tool actions, setup, waits, and verifier flow.
• —vLLM decode-only: server-side generation speed, excluding orchestration and tool latency.