drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16

Nemotron-3-Nano-Omni-30B-A3B — INT4

INT4 weight-only quantization of `nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16`. Sized to fit a single ≥ 24 GB consumer / workstation GPU.

Validation

Loaded and verified with vLLM ≥ 0.20.0 (native nemotron_v3 / NanoNemotronVL path). Round-trip correctness: bit-exact within INT4 quantization step (per-layer dequantize MAE ≈ 1e-5).

Score on needle-1M-bench-mvp 50K

Single miss is the deepest needle (depth 49,496 / 50,566). All 9 other depths score 100 %.

Leaderboard: `drawais/needle-1M-bench-mvp`. Per-row YAML: `.eval_results/nemotron-omni-30b-a3b-w4a16-50k.yaml`.

Load (vLLM, text)

from vllm import LLM, SamplingParams

llm = LLM(
    model="drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16",
    trust_remote_code=True,
    max_model_len=65536,
)
params = SamplingParams(temperature=0.6, top_p=0.95, max_tokens=4096)
print(llm.generate(["Hello, world!"], params)[0].outputs[0].text)

Serve (vLLM, OpenAI-compatible)

vllm serve drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16 \
  --trust-remote-code \
  --max-model-len 65536 \
  --gpu-memory-utilization 0.94

Then point any OpenAI client at the local endpoint:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="dummy")
print(client.chat.completions.create(
    model="drawais/Nemotron-3-Nano-Omni-30B-A3B-W4A16",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=64,
).choices[0].message.content)

For multimodal usage (image / audio / video), reasoning controls, recommended --reasoning-parser nemotron_v3, tool-calling flags, and per-modality serving recommendations, follow the upstream Nemotron-3-Nano-Omni model card. If audio inputs are used: pip install vllm[audio].

Footprint

~22 GB on disk for the weights. Total VRAM should leave headroom for KV cache and multimodal-encoder activations; recommended:

• —≥ 32 GB consumer for full context + multimodal
• —≥ 24 GB with reduced --max-model-len and text-only usage

trust_remote_code=True is required.

Acknowledgements

Source model © NVIDIA Corporation, released under the NVIDIA Open Model Agreement. This artifact is a Derivative Work as defined in that agreement. See `LICENSE` and `NOTICE` for full text and required attribution.

License

NVIDIA Open Model Agreement (Release Date: April 2, 2026).

Commercially usable. You are free to create and distribute Derivative Works. NVIDIA does not claim ownership of outputs.

The full agreement text is included in `LICENSE`. The attribution notice required by Section 3(c) is in `NOTICE`:

Licensed by NVIDIA Corporation under the NVIDIA Open Model Agreement.