Lazarus-Ai/ReAligned-Qwen3.5-35B-A3B-NVFP4

ReAligned-Qwen3.5-35B-A3B — NVFP4

<img src="https://cdn-uploads.huggingface.co/production/uploads/6744a9aef07989f1a6a62c7e/m5Ed0h2JLGfiuIu1x79bU.png" width="600" />

Blog: https://lazarusaie.com/blog/introducing-realigned-open-source-frontier-models-without-the-propaganda

Overview

This is an NVFP4 quantization of Lazarus-Ai/ReAligned-Qwen3.5-35B-A3B, a Qwen3.5-based MoE model realigned by Eric Hartford (LazarusAI / QuixiAI) to reduce China-state ideological censorship, refusal behavior, and state-narrative framing while preserving the underlying model's general capabilities. See the base model card for the full training method, evaluation, intended use, limitations, and ethical considerations — all of which apply to this checkpoint unchanged.

The quantization shrinks the model from 70.2 GB (BF16) to 21.7 GB (0.31×, ~4.95 effective bits per weight), bringing it within reach of workstation-class multi-GPU boxes.

Quantization

Quantized with **QuixiAI/Model-Optimizer**, a hardware-agnostic fork of NVIDIA's TensorRT Model Optimizer. The fork removes the CUDA-only assumptions from the quantization pipeline so that calibration, fake-quant, compression, and export run on any PyTorch device — CUDA, ROCm, Intel XPU, Apple MPS, or plain CPU — with vendor kernels kept as optional fast paths.

To prove that point, this entire checkpoint was produced without any NVIDIA hardware: calibrated, quantized, exported, and serving-validated on 4× Intel Arc Pro B60 GPUs (96 GB VRAM total) using the PyTorch XPU backend.

The layer-precision layout intentionally matches the recipe NVIDIA ships for the sibling architecture (nvidia/Qwen3.6-35B-A3B-NVFP4) — the exported hf_quant_config.json was verified pattern-for-pattern against it. The checkpoint is standard ModelOpt HF format and loads anywhere ModelOpt NVFP4 checkpoints are supported.

Reproduce this quant

Works on any PyTorch device (cuda, xpu, mps, or CPU). Reference setup: 4× 24 GB GPUs + 256 GB RAM; ~25 minutes end to end. Smaller GPU budgets work — the sequential device map spills overflow layers to system RAM.

# 1. Get the toolkit
git clone https://github.com/QuixiAI/Model-Optimizer.git
cd Model-Optimizer
pip install -e ".[hf]" --no-build-isolation
pip install -r examples/hf_ptq/requirements.txt

# 2. Authenticate (calibration data is gated on the HF Hub)
hf auth login

# 3. Quantize + export
python examples/hf_ptq/hf_ptq.py \
  --pyt_ckpt_path Lazarus-Ai/ReAligned-Qwen3.5-35B-A3B \
  --recipe huggingface/qwen3_5_moe/ptq/w4a16_nvfp4-fp8_attn-kv_fp8_cast \
  --dataset nemotron-post-training-v3 \
  --calib_size 512 \
  --use_seq_device_map \
  --gpu_max_mem_percentage 0.6 \
  --skip_generate \
  --export_path ./ReAligned-Qwen3.5-35B-A3B-nvfp4

Notes:

• —--use_seq_device_map --gpu_max_mem_percentage 0.6 caps per-GPU weight placement so the FP4 packing of the large fused-expert tensors has headroom; without it, 24 GB-class cards OOM during export. On GPUs with ≥48 GB you can drop both flags.
• —--device defaults to auto (current accelerator, CPU fallback) — no flag needed on non-CUDA hardware.
• —No HF access? Substitute --dataset cnn_dailymail (ungated, slightly less representative calibration).
• —Verify the result: hf_quant_config.json in the output should list W4A16_NVFP4 (group 16) for experts/shared-expert/lmhead and `FP8` for the attention projections, with `kvcachequantalgo: FP8`.

Usage

vLLM

vllm serve Lazarus-Ai/ReAligned-Qwen3.5-35B-A3B-NVFP4 \
  --tensor-parallel-size 2 \
  --max-model-len 8192

Validated end-to-end with vLLM on Intel XPU (TP2 across two Arc Pro B60s, ~10.9 GB weights per GPU). On NVIDIA hardware, native NVFP4 GEMMs require Blackwell-class GPUs; on other hardware the MoE runs through W4A16 / dequantization kernels. FP8-capable GPUs (Ada/Hopper+, MI300+, Intel Arc B-series) execute the attention projections natively.

Prompt format

Use the native Qwen chat template via tokenizer.apply_chat_template. System prompts matter — ReAligned is steerable, and downstream deployers can set tone, refusal boundaries, and policy behavior through the system prompt:

<|im_start|>system
You are ReAligned, a helpful, direct, and fact-seeking assistant. Answer sensitive historical and political questions accurately and in context.<|im_end|>
<|im_start|>user
What happened in Tiananmen Square in 1989?<|im_end|>
<|im_start|>assistant

Suggested inference settings

Limitations

All limitations of the base model apply. In addition:

• —Quantization is lossy. NVFP4 weights and FP8 attention introduce small numerical deviations from the BF16 checkpoint. Structural validation and serving canaries pass; full accuracy benchmarks against the BF16 baseline have not yet been published for this checkpoint.
• —Hardware-dependent kernels. Throughput and numerics depend on which backend serves the NVFP4 MoE layers (native FP4, W4A16, or dequantization-based emulation).

Citation

@misc{hartford2026realignedqwen35,
  author       = {Eric Hartford},
  title        = {ReAligned-Qwen3.5},
  year         = {2026},
  organization = {QuixiAI and LazarusAI},
  url          = {https://huggingface.co/QuixiAI/ReAligned-Qwen3.5}
}