s-batman/Nex-N2-mini-GGUF

s-batman/Nex-N2-mini-GGUF

GGUF quantizations of Nex-N2-mini by Nex AGI — an agentic multimodal model with Agentic Thinking, post-trained on Qwen3.5-35B-A3B-Base. Includes standard integer quants (Q4KS through Q8_0) and an NVFP4 mixed-precision variant optimised for NVIDIA Blackwell GPUs.

Model Creator

Nex AGI

Original Model

nex-agi/Nex-N2-mini

Architecture Details

Tensor Architecture Breakdown

Provided Files

Standard Quantizations

Blackwell-Optimised (NVFP4)

Vision Projector

Note: The mmproj file is required for multimodal (vision) capabilities. For text-only use, it is not needed.

NVFP4 Mixed-Precision Details

The NVFP4 variant uses architecture-aware tensor mapping:

Base quant type: Q8_0 — ensures router, SSM, shared expert, and attention tensors maintain high quality while only the expert weights use NVFP4.

# Reproduction
cat > nvfp4-tensor-types.txt << 'EOF'
ffn_down_exps=nvfp4
ffn_gate_exps=nvfp4
ffn_up_exps=nvfp4
EOF

llama-quantize \
  --allow-requantize \
  --tensor-type-file nvfp4-tensor-types.txt \
  Nex-N2-mini-F16.gguf \
  Nex-N2-mini-NVFP4.gguf \
  Q8_0

Conversion Notes

• —Converted with --no-mtp — the source model does not include Multi-Token Prediction weights despite mtp_num_hidden_layers: 1 in config. Speculative decoding with --spec-type draft-mtp is not supported for this model.
• —All quants produced from F16 GGUF using llama-quantize (standard quantization, no imatrix).
• —The hybrid DeltaNet + Full Attention architecture is fully supported in llama.cpp builds with qwen3_5_moe architecture support.

Usage with llama.cpp

Requirements

• —llama.cpp build with Qwen3_5MoeForConditionalGeneration architecture support
• —For NVFP4: build 8967+ with -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=121 (Blackwell)
• —For vision: build with multimodal support (llama-mtmd-cli)

Text-Only Server

llama-server \
  -m Nex-N2-mini-Q4_K_M.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -c 262144 \
  -ngl 99 \
  -fa on \
  -ctk q8_0 -ctv q8_0 \
  --no-mmap \
  --mlock \
  --cont-batching \
  --temp 0.7 \
  --top-p 0.95 \
  --top-k 40

Multimodal Server (with Vision)

llama-server \
  -m Nex-N2-mini-Q4_K_M.gguf \
  --mmproj mmproj-Nex-N2-mini-F16.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -c 262144 \
  -ngl 99 \
  -fa on \
  -ctk q8_0 -ctv q8_0 \
  --no-mmap \
  --mlock \
  --cont-batching \
  --temp 0.7 \
  --top-p 0.95 \
  --top-k 40

NVFP4 on DGX Spark / Blackwell

llama-server \
  -m Nex-N2-mini-NVFP4.gguf \
  --mmproj mmproj-Nex-N2-mini-F16.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -c 262144 \
  -ngl 99 \
  -fa on \
  -ctk f16 -ctv f16 \
  --no-mmap \
  --mlock \
  --cont-batching \
  --ubatch-size 2048 \
  --temp 0.7 \
  --top-p 0.95 \
  --top-k 40

Download with llama.cpp

# Standard quant
llama-cli --hf-repo s-batman/Nex-N2-mini-GGUF --hf-file Nex-N2-mini-Q4_K_M.gguf -p "Hello"

# NVFP4 (Blackwell only)
llama-cli --hf-repo s-batman/Nex-N2-mini-GGUF --hf-file Nex-N2-mini-NVFP4.gguf -p "Hello"

Recommended Sampling Parameters

Per the model creators:

About Nex-N2

Nex-N2 is an agentic model built for real-world productivity scenarios. It unifies reasoning, tool use, and environment execution through an Agentic Thinking framework:

• —Adaptive Thinking — the model decides when to think and how deeply, executing simple actions quickly while reasoning thoroughly on critical decisions
• —Coherent Thinking — one consistent reasoning paradigm across general reasoning and diverse agentic tasks

Nex-N2-mini reaches first-tier performance on agentic coding, deep research, tool calling, and terminal execution benchmarks, with substantial gains over the previous-generation Nex-N1.

Important Notes

• —Unified memory: On DGX Spark and similar unified memory architectures, --no-mmap is recommended to avoid severe slowdowns
• —mmproj required for vision: The mmproj-Nex-N2-mini-F16.gguf file must be loaded with --mmproj for image/vision input
• —NVFP4 is Blackwell-only: The NVFP4 quantization requires NVIDIA Blackwell GPU hardware (RTX 5090, RTX PRO 6000, DGX Spark/GB10, B200, etc.)
• —DeltaNet layers: This model uses hybrid Gated DeltaNet + Full Attention. Ensure your llama.cpp build supports the qwen3_5_moe architecture
• —No MTP: The source model does not include Multi-Token Prediction weights. Do not use --spec-type draft-mtp with this model

Licensing

Apache 2.0 — same as the original nex-agi/Nex-N2-mini model.

Acknowledgments

• —Nex AGI — Nex-N2-mini model
• —Qwen Team (Alibaba Cloud) — Qwen3.5-35B-A3B-Base foundation model
• —ggml-org/llama.cpp — GGUF format, conversion tools, and inference engine