rodrigoramosrs/veriloop-coder-e2-nvfp4

<div align="center"> <h1>VeriLoop Coder E2 · NVFP4</h1> <p><strong>Coding-Optimized Quantized Model (NVIDIA NVFP4)</strong></p> <p> <a href="https://huggingface.co/tsinghua-sigs-robot-lab/VeriLoop-E2">Original Model ↗</a> · <a href="https://github.com/rodrigoramosrs">GitHub</a> · Apache-2.0 </p> </div>

Overview

This repository contains an NVFP4 quantization of VeriLoop E2, an open 27B post-trained model built on Qwen3.8-27B for code, mathematics, and physics. Its core reasoning discipline is VeriLoop-Governed Recurrence (VGR): candidate states are recursively proposed, externally checked, and retained only when the protected evidence state improves without regression.

Quantized by Rodrigo Ramos.

Quantization Approach

Produced with NVIDIA Model Optimizer using the canonical `NVFP4_W4A4_WEIGHT_LOCAL_HESSIAN_CFG` recipe, verbatim: static per-block 4-bit weights (group size 16) + dynamic 4-bit activations, FP8 attention, local-Hessian calibration with MSE and fp8 scale sweep.

Calibration ran on 512 samples × 512 tokens (262144 tokens) of competitive-programming code from nvidia/Nemotron-Competitive-Programming-v1, keeping quantization faithful on the distributions that matter most for coding tasks.

The result retains the original model's strong software-engineering capabilities at roughly one third of the BF16 footprint, ready for NVFP4-capable inference stacks on Blackwell GPUs.

Files

Usage

This checkpoint targets NVFP4-capable runtimes (Blackwell GPUs). Plain transformers cannot load modelopt NVFP4 checkpoints directly.

Compatibility note: per the NVIDIA recipe, a small set of projections (linear_attn.in_proj_a/b, conv1d), plus the embeddings and lm_head, intentionally ship in BF16. The exclude_modules list in hf_quant_config.json (mirrored in config.json) declares exactly which modules those are — inference loaders must honor it and serve those modules unquantized. If a loader errors with "weight is torch.bfloat16 but the quant config declares NVFP4", it is ignoring the exclusion list, not a checkpoint defect.

vLLM

pip install 'vllm>=0.17.0'

vllm serve rodrigoramosrs/veriloop-coder-e2-nvfp4 \
    --tensor-parallel-size 1 \
    --max-model-len 131072 \
    --kv-cache-dtype fp8_e4m3 \
    --gpu-memory-utilization 0.92

SGLang

pip install 'sglang[all]'

python -m sglang.launch_server \
    --model-path rodrigoramosrs/veriloop-coder-e2-nvfp4 \
    --trust-remote-code \
    --mem-fraction-static 0.88

TensorRT-LLM

Build the engine from this checkpoint with trtllm-build using an NVFP4 + FP8-KV configuration, then serve with trtllm-serve. See the TensorRT-LLM documentation for the exact flags for your GPU.

Reproducing

pip install "nvidia-modelopt[all]" torch transformers accelerate datasets safetensors

# needs ~55 GB VRAM across CUDA GPUs for the 27B BF16 source model
python scripts/quantize_veriloop.py \
    --model ./model-bf16 \
    --output ./model-nvfp4 \
    --calib-size 512 \
    --calib-seq-len 512

python scripts/validate_nvfp4.py ./model-nvfp4

On Windows the Triton JIT needs a C compiler: run from a Visual Studio Native Tools prompt (vcvars64) with CC pointing at cl.exe. See the header of scripts/quantize_veriloop.py for multi-GPU layout tuning (--layers-split, --gpu-order).

Acknowledgements

• —Libo Wang and the Intelligent Robotics Laboratory, Tsinghua SIGS for developing the original VeriLoop E2 model.
• —NVIDIA for Model Optimizer and the NVFP4 quantization tooling.
• —The original model repository: tsinghua-sigs-robot-lab/VeriLoop-E2

License

Apache-2.0. The weights are quantized from the original Apache-2.0 licensed model. See the original repository for full licensing details and third-party notices.