CoolFace
Modelpublic

RemySkye/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-GGUF

sourceHugging Faceotherupdated 3mo agoView on Hugging Face
10likes4kdownloads
Model Card

NVIDIA Nemotron Labs 3 Puzzle 75B-A9B GGUF

Community GGUF conversion of `nvidia/NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16`.

Important llama.cpp compatibility notice

This model was converted with the still-unmerged `ggml-org/llama.cpp` PR #25444, pinned to commit af49ef5cd990d039dbf360dd3a9f3b5dafdd1726, plus a narrowly scoped converter compatibility fix for the official BF16 checkpoint's model.layers.* tensor prefix and bounded writeback for large lazy tensors on the high-RAM Colab runtime. Until equivalent support is merged into mainline llama.cpp, use a build containing PR #25444 to load these files.

PR #25444 adds NemotronHPuzzleForCausalLM / nemotron_h_puzzle support, heterogeneous per-layer MoE settings, and the model's two-block MTP draft head. This repository is not an official NVIDIA or llama.cpp release.

Files

  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-BF16.gguf — BF16 master GGUF
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q2_K.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_S.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_M.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q3_K_L.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-IQ4_XS.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_K_S.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_K_M.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_K_S.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_K_M.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q6_K.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q8_0.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_0.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q4_1.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_0.gguf
  • —NVIDIA-Nemotron-Labs-3-Puzzle-75B-A9B-Q5_1.gguf

No importance matrix was used for any quantization in this repository.

Shape-fallback warning

Puzzle uses heterogeneous expert widths from 1280 to 2688. Some expert down-projection rows, especially width 2688, are not divisible by the 256-element block size used by K-quants. The pinned llama.cpp quantizer therefore applies its compatible fallback automatically on those tensors: Q2K/Q3K to Q40, IQ4XS to IQ4NL, Q4K to Q50, Q5K to Q51, and Q6K to Q8_0. Consequently, named K-quant files are mixed recipes rather than a single tensor type everywhere. Required norms, scales, and other small tensors may remain F32.

Artifact integrity and quantizer compatibility are checked automatically, but successful conversion does not prove task-quality parity with BF16. Evaluate very small formats before practical use.

Provenance

  • —Source revision: 8fe5546888e9bd03fdbf52d808adebdfca901b52
  • —Source safetensors: 156,596,801,168 bytes across 31 model shards plus mtp.safetensors
  • —llama.cpp PR: #25444
  • —PR commit: af49ef5cd990d039dbf360dd3a9f3b5dafdd1726
  • —Patched converter SHA-256: 1aaa36ac789fc6eceebefe19d4d80c3c9dc56185a4a3e956411bc0478ee46afc
  • —Patched large-write GGUF writer SHA-256: 350db0132703b3b4025ee61e344b7d7400b9c3a2692b87dd6ca186d82423fa30
  • —Conversion: pinned safetensors in hybrid RAM/disk storage, then local convert_hf_to_gguf.py --outtype bf16
  • —Quantization: pinned llama-quantize, without --imatrix
  • —License: the source model is governed by the OpenMDW License Agreement 1.1

Follow the prompting, safety, license, and usage guidance in the original NVIDIA model card.