Modelpublic

FreedomAISVR/Nex-N2-mini-MXFP4-MOE-GGUF

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes86downloads

Model Card

Nex-N2-mini-MXFP4 MOE-GGUF

GGUF quantization of nex-agi/Nex-N2-mini — a 35B MoE agentic model (3B active) built on Qwen3.5-35B-A3B-Base with 256 experts, Gated DeltaNet hybrid attention, 262K context, and 27-layer vision encoder.

Quantized to MXFP4 MOE format for efficient inference with minimal quality loss.

About MXFP4 MOE

MXFP4MOE (Microscaling FP4 for Mixture-of-Experts) is an OCP standard 4-bit format. Expert weights are stored in MXFP4 while non-expert tensors (attention, embeddings, norms) remain at Q80, balancing quality and compression for MoE models. Works on any GPU or CPU.

Files

Filename	Type	Size	Description
`nex-n2-mini-mxfp4_moe.gguf`	GGUF (MXFP4 MOE)	18.43 GB	Quantized text model weights
`mmproj-nex-n2-mini-f16.gguf`	F16 mmproj	0.84 GB	Vision encoder projector (27-layer ViT, 1152 hidden)
`README.md`	Markdown	-	Model card

Quantization Details

Property	Value
Format	MXFP4 MOE
Bits Per Weight	4.57 BPW
File Size	18.43 GB (text) + 0.84 GB (mmproj)
Tensor Count	733 (text) + 334 (mmproj)
Architecture	qwen35moe

Model Description

—Developer: Nex AGI
—Base Model: Qwen3.5-35B-A3B-Base
—Architecture: Mixture-of-Experts (MoE) with Gated DeltaNet + full attention
—Parameters: 35B total, 3B activated per token
—Experts: 256 routed experts (8 per token) + 1 shared
—Context Length: 262,144 tokens
—Vision: 27-layer ViT encoder (1152 hidden), image-text-to-text
—Languages: English, Chinese, multilingual
—License: Apache 2.0

Usage

llama.cpp (CLI)

bash

# Text + Image
llama-cli -m nex-n2-mini-mxfp4_moe.gguf \
  --mmproj mmproj-nex-n2-mini-f16.gguf \
  --image photo.jpg \
  -p "Describe this image in detail" \
  -n 512

# Text only
llama-cli -m nex-n2-mini-mxfp4_moe.gguf \
  -p "Explain quantum computing in simple terms" \
  -n 512

# OpenAI-compatible server
llama-server -m nex-n2-mini-mxfp4_moe.gguf \
  --mmproj mmproj-nex-n2-mini-f16.gguf \
  --port 8080

llama-cpp-python

python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="FreedomAISVR/Nex-N2-mini-MXFP4-MOE-GGUF",
    filename="nex-n2-mini-mxfp4_moe.gguf",
    n_gpu_layers=-1,
)

response = llm.create_chat_completion([
    {"role": "user", "content": "What is the capital of France?"}
])
print(response["choices"][0]["message"]["content"])

Direct download

python

from huggingface_hub import hf_hub_download

for filename in ["nex-n2-mini-mxfp4_moe.gguf", "mmproj-nex-n2-mini-f16.gguf"]:
    hf_hub_download(
        repo_id="FreedomAISVR/Nex-N2-mini-MXFP4-MOE-GGUF",
        filename=filename,
        local_dir="./models"
    )

Quantization Pipeline

1. Download source weights
   huggingface_hub.snapshot_download("nex-agi/Nex-N2-mini")

2. Convert text model to F16 GGUF
   convert_hf_to_gguf.py --outtype f16

3. Extract vision encoder
   convert_hf_to_gguf.py --mmproj --outtype f16

4. Quantize to MXFP4 MOE
   llama-quantize --allow-requantize nex-n2-mini-f16.gguf nex-n2-mini-mxfp4_moe.gguf MXFP4_MOE

Hardware

Component	Specification
GPU	NVIDIA RTX 5060 Ti (Blackwell)
System RAM	64 GB
Storage	NVMe

License

Apache 2.0 — same as the original nex-agi/Nex-N2-mini.