groxaxo/Qwen-AgentWorld-35B-A3B-GPTQ-Pro-Int4

Qwen-AgentWorld-35B-A3B-GPTQ-Pro-Int4

<!-- polished-overview:start -->

Overview

Qwen-AgentWorld-35B-A3B-GPTQ-Pro-Int4 is a GPTQ-quantized checkpoint intended for efficient GPU inference, published by `groxaxo`. It is intended for open-source evaluation, reproducible experimentation, and compatible local or hosted inference workflows. The wording below is deliberately limited to what can be verified from this repository's metadata and artifacts.

At a glance

What is included

• —*.safetensors (5 files)
• —config.json
• —generation_config.json
• —tokenizer.json
• —tokenizer_config.json
• —chat_template.jinja
• —quantize_config.json
• —Additional configuration, tokenizer, processor, or shard files (13 visible artifacts total)

Quick start

vLLM (documented configuration)

vllm serve groxaxo/Qwen-AgentWorld-35B-A3B-GPTQ-Pro-Int4 \
  --quantization gptq_marlin \
  --dtype float16 \
  --trust-remote-code

This command is taken from the repository documentation. Adjust tensor parallelism, context length, and cache settings to match your hardware and vLLM version.

Compatibility and responsible use

• —Use a runtime that explicitly supports this format, architecture, and modality.
• —Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
• —Review the source model card and license before redistribution or deployment.
• —Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
• —Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws. <!-- polished-overview:end -->

A compact 4-bit GPTQ-Pro quantization of Qwen-AgentWorld-35B-A3B — built for local agent simulation, tool-use reasoning, terminal/world-state prediction, and long-context agent workflows.

This repository contains an INT4 GPTQ-Pro quantized version of `Qwen/Qwen-AgentWorld-35B-A3B`, preserving the core architecture and behavior of the original language world model while dramatically reducing the weight footprint for practical local serving.

The upstream model is not just another chat model with a tool-use sticker slapped on it. Qwen-AgentWorld is trained as a language world model: given an agent’s action and interaction history, it predicts the next environment observation/state. That makes it especially interesting for agentic workflows where the model needs to reason about what should happen next inside tools, terminals, browsers, operating systems, software projects, and simulated environments.

This quantized build is intended for people who want to actually run the thing, not frame the BF16 checkpoint and admire it from across the room.

What this model is good at

This model is best suited for:

• —Agentic environment simulation
• —Tool-call and MCP-style interaction modeling
• —Terminal state prediction
• —Software-engineering workflow simulation
• —Browser / web / OS interaction reasoning
• —Synthetic trajectory generation for agents
• —Long-context planning and replay
• —Local agent research
• —OpenAI-compatible backend serving

The upstream AgentWorld model covers seven major interaction domains:

Model details

Why this quant exists

The original BF16 checkpoint is powerful, but heavy. This GPTQ-Pro INT4 build is aimed at practical deployment where you want the AgentWorld behavior without needing a datacenter-class inference box.

Use this when you want:

• —Smaller disk and VRAM footprint
• —Faster local experimentation
• —OpenAI-compatible serving
• —Multi-GPU consumer inference
• —Agent simulation without calling external APIs
• —A world-model backend for agent training, evaluation, or synthetic rollout generation

Quantization is not magic. INT4 can affect fidelity, especially for very long-context workloads. For serious benchmark or production use, compare against the BF16 upstream model on your own tasks.

Recommended serving: vLLM

Use a recent vLLM build. Qwen3.5 MoE support is new enough that old wheels may fail or silently behave like garbage. Delightful hobby, this one.

pip install -U vllm