vikrant1123/Nex-N2.5-mini-Uncensored-APEX-GGUF

Nex-N2.5-mini-Uncensored APEX GGUF

APEX mixed-precision GGUF quantizations of orcarouter/Nex-N2.5-mini-Uncensored, created from the original BF16 weights using llama.cpp and localai-org/apex-quant.

These files are intended for use with recent versions of llama.cpp and compatible GGUF frontends.

About the model

The source model is an abliterated / refusal-reduced variant of Nex-AGI's Nex-N2.5-mini.

Key architecture details:

• —~34B total parameters
• —Mixture-of-Experts architecture
• —40 model layers
• —256 routed experts
• —Top-8 expert routing
• —One shared expert per layer
• —Hybrid gated-delta-net + full attention
• —Native vision support
• —262,144-token maximum context
• —English and Chinese support
• —Apache-2.0 license

Source model:

orcarouter/Nex-N2.5-mini-Uncensored

Original GGUF / imatrix source:

orcarouter/Nex-N2.5-mini-Uncensored-GGUF

Quantization method

These models use APEX (Adaptive Precision for EXpert Models) by localai-org/apex-quant.

APEX is a MoE-aware mixed-precision quantization method. Instead of assigning one quantization type to the entire model, different precision levels are assigned to different layers and tensor groups.

This repository is therefore not equivalent to a normal Q2, Q3, Q4, or Q5 GGUF.

The quantization pipeline used here was:

Original BF16 Hugging Face checkpoint
        ↓
llama.cpp convert_hf_to_gguf.py
        ↓
F16 GGUF
        ↓
APEX mixed-precision quantization
        +
OrcaRouter imatrix.dat
        ↓
Final APEX GGUF

All APEX quantizations were produced directly from the F16 GGUF.

No Q8, Q6, Q4, or other quantized GGUF was used as an intermediate source.

Importance matrix

These quantizations use:

orcarouter/Nex-N2.5-mini-Uncensored-GGUF/imatrix.dat

The imatrix was produced by OrcaRouter from a 1200-chunk calibration corpus containing approximately equal portions of:

• —English prose
• —Chinese text
• —Real source code

OrcaRouter reports a median expert-tensor coverage of approximately 99.2% for this calibration run.

This is important for aggressive importance-matrix-dependent quantization types such as IQ2_S.

Note: this is the OrcaRouter imatrix, not the calibration matrix used for the official APEX benchmark results. Therefore, upstream APEX benchmark numbers should not be interpreted as measurements of these files.

Available quantizations

Not every file listed above may be uploaded yet. Check the Files and versions tab for currently available builds.

About the Mini filename

APEX officially calls the smallest profile APEX Mini rather than "I-Mini".

Mini already uses importance-matrix-dependent quantization for its aggressive middle-layer expert tensors, so it is built here using the same OrcaRouter imatrix.dat.

Conversion details

The original Hugging Face BF16 weights were converted to F16 GGUF with a recent version of llama.cpp.

The conversion requires --no-mtp for this model:

python convert_hf_to_gguf.py \
  /path/to/Nex-N2.5-mini-Uncensored \
  --no-mtp \
  --outtype f16 \
  --use-temp-file \
  --outfile Nex-N2.5-mini-Uncensored-F16.gguf

--no-mtp is required because the model configuration declares an MTP layer while the released checkpoint does not contain the corresponding mtp.* tensors.

Example APEX quantization

Example for APEX I-Compact:

./scripts/quantize.sh \
  --profile i-compact \
  --layers 40 \
  --imatrix /path/to/imatrix.dat \
  Nex-N2.5-mini-Uncensored-F16.gguf \
  Nex-N2.5-mini-Uncensored-APEX-I-Compact.gguf

Example for APEX Mini:

./scripts/quantize.sh \
  --profile mini \
  --layers 40 \
  --imatrix /path/to/imatrix.dat \
  Nex-N2.5-mini-Uncensored-F16.gguf \
  Nex-N2.5-mini-Uncensored-APEX-Mini.gguf

Vision support

The main APEX GGUF files contain the language-model portion.

For image input, also download:

mmproj-Nex-N2.5-mini-Uncensored-F16.gguf

The F16 vision projector can be used with all APEX quantization tiers in this repository.

Vision example

Using llama-mtmd-cli:

./llama-mtmd-cli \
  -m Nex-N2.5-mini-Uncensored-APEX-I-Compact.gguf \
  --mmproj mmproj-Nex-N2.5-mini-Uncensored-F16.gguf \
  -ngl 20 \
  -c 8192 \
  --image image.png \
  -p "Describe this image."

Adjust -ngl according to your available VRAM.

Text inference

Example with llama-cli:

./llama-cli \
  -m Nex-N2.5-mini-Uncensored-APEX-I-Compact.gguf \
  -ngl 20 \
  -c 8192 \
  --jinja \
  -p "Explain mixture-of-experts routing in simple terms."

For GPUs that cannot hold the complete model, llama.cpp can partially offload layers to the GPU while keeping the remaining weights in system RAM.

Server

Example OpenAI-compatible server:

./llama-server \
  -m Nex-N2.5-mini-Uncensored-APEX-I-Compact.gguf \
  --mmproj mmproj-Nex-N2.5-mini-Uncensored-F16.gguf \
  -ngl 20 \
  -c 8192 \
  --jinja \
  --host 0.0.0.0 \
  --port 8080

Remove --mmproj if vision input is not needed.

Reasoning

The model's chat template uses reasoning_effort for controlling reasoning behavior.

For direct answers without extended reasoning, compatible clients can pass:

{
  "chat_template_kwargs": {
    "reasoning_effort": "none"
  }
}

When reasoning is enabled, make sure the generation limit is large enough for the model to complete its reasoning block and produce the final answer.

Requirements

Use a recent llama.cpp build with qwen35moe support.

Older llama.cpp builds may fail with an error similar to:

unknown architecture 'qwen35moe'

CUDA, ROCm, Metal, and CPU inference depend on your llama.cpp build and hardware.

Hardware notes

These are mixed-precision MoE models.

Only a subset of routed experts is active for each token, but the complete model weights still need to be available through VRAM, system RAM, or memory mapping.

Memory usage is approximately:

model file size
+ KV / recurrent-state cache
+ compute buffers
+ ~0.9 GB vision projector when vision is enabled

If the model does not fit completely in VRAM, reduce -ngl.

For example:

-ngl 15
-ngl 20
-ngl 25

and monitor VRAM usage to find an appropriate setting for your hardware.

Benchmarks

No benchmark results are claimed for these quantizations yet.

The benchmark results published by the APEX project were obtained on other model weights and with their own calibration setup. They should not be treated as measurements of the files in this repository.

If benchmark results for these exact GGUF files are added later, they will be clearly identified as such.

Safety and intended use

This repository contains quantized versions of an abliterated / refusal-reduced model.

The source model has substantially reduced safety alignment and may generate harmful, unethical, offensive, or otherwise unsafe content that the original aligned model would refuse.

Users are responsible for:

• —evaluating the model before deployment;
• —adding appropriate safety and moderation controls where needed;
• —complying with applicable laws and regulations;
• —reviewing the original model card, intended-use guidance, risks, and limitations.

This repository only changes the numerical representation of the source weights through quantization. It does not add safety alignment or moderation.

License

The source model is distributed under the Apache License 2.0.

These quantized weights preserve the source model's Apache-2.0 licensing.

Please also review the original model repository for attribution, notices, usage guidance, and any applicable access conditions.

Credits

Model

nex-agi/Nex-N2.5-mini

Uncensored / abliterated checkpoint

orcarouter/Nex-N2.5-mini-Uncensored

Importance matrix and original GGUF work

orcarouter/Nex-N2.5-mini-Uncensored-GGUF

APEX quantization

localai-org/apex-quant

GGUF runtime and tooling

ggml-org/llama.cpp

Thanks to all upstream authors and contributors.

Disclaimer

This repository is an independent quantization / redistribution of upstream weights.

It is not an official release from Nex-AGI, OrcaRouter, the APEX project, or llama.cpp.

All trademarks, model names, and project names belong to their respective owners.