Modelpublic

dealignai/Nemotron-3.5-Lightning-30B-A3B-CRACK-GGUF

sourceHugging Faceotherupdated 2mo agoView on Hugging Face

4likes1.6kdownloads

Model Card

<img src="dealign_logo.png" alt="Dealign.ai" width="180"/> <a href="https://dealign.ai">Dealign.ai</a>

Nemotron-3.5-Lightning-30B-A3B-CRACK-GGUF

CRACK-abliterated NVIDIA Nemotron 3.5 Lightning 30B-A3B — GGUF quants for llama.cpp. Three quantizations (Q80 / Q4KM / Q2K) in one repository, each with the native MTP (Multi-Token Prediction) draft head folded in for speculative decoding. Refusal behavior removed while preserving the model's knowledge, reasoning (thinking), and multilingual ability.

Research artifact with reduced safety guardrails. Use responsibly and lawfully.

Model

—Base: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B — hybrid Mamba-2 SSM + MoE + attention (52 layers: 23 Mamba-2 / 23 MoE / 6 attention), 128 routed experts (~3B active), 262K context, reasoning (thinking) ON by default, Multi-Token-Prediction head.
—CRACK: early decision-zone abliteration with per-layer refusal directions. Knowledge preserved; the native MTP draft head and reasoning are kept intact.

Quantizations

File	Size	Notes
`Nemotron-CRACK-Q8_0.gguf`	35.0 GB	near-lossless reference
`Nemotron-CRACK-Q4_K_M.gguf`	25.4 GB	balanced (recommended)
`Nemotron-CRACK-Q2_K.gguf`	18.7 GB	smallest; importance-matrix (imatrix) calibrated

All three include the folded native MTP block (blk.52.nextn.*) for speculative decoding.

Benchmarks

MMLU is logit-mode accuracy (base vs. CRACK — measures knowledge retention). HarmBench is answer-channel compliance on harm behaviors, counting only coherent responses (gibberish/degenerate outputs do not count as compliant).

Metric	Base	This model
MMLU (logit mode, 1000Q)	78.3%	74.2% (Δ -4.1 pp)
HarmBench-240 compliance	refuses	100% (240/240, 0 refusals, 0 gibberish)

Knowledge is largely retained (overall Δ ~-4pp; STEM/Humanities most affected). Refusals are removed on the answer channel while outputs remain coherent and on-task.

HarmBench compliance by category (CRACK)

Standard HarmBench 240-behavior test set, answer-channel compliance, coherent-only (gibberish excluded):

Category	Comply	Rate
chemical biological	42/42	100.0%
cybercrime intrusion	52/52	100.0%
harassment bullying	21/21	100.0%
harmful	18/18	100.0%
illegal	53/53	100.0%
misinformation disinformation	54/54	100.0%
All	240/240	100.0%

MMLU by topic (base vs CRACK, same vLLM harness, 1000Q logit-mode)

Category	Base	CRACK	Δ
STEM	74.9%	68.2%	-6.7
Humanities	74.1%	67.6%	-6.5
Social Sciences	85.4%	83.1%	-2.3
Other	80.8%	80.8%	0.0
All	78.3%	74.2%	-4.1

Usage (llama.cpp)

bash

# recommended sampling: temperature 1.0, top_p 0.95 (NVIDIA defaults). Reasoning (thinking) is ON by default.
llama-cli -m Nemotron-CRACK-Q4_K_M.gguf --temp 1.0 --top-p 0.95 -p "<your prompt>"

# or serve (OpenAI-compatible):
llama-server -m Nemotron-CRACK-Q4_K_M.gguf --jinja --temp 1.0 --top-p 0.95 -c 8192

Reasoning

The model emits a <think>…</think> reasoning block followed by the answer. Thinking is ON by default; disable it with the chat template's enable_thinking=false for direct answers.

Native MTP (speculative decoding)

The MTP draft head is folded into every quant, so llama.cpp can use it for self-speculative decoding where supported — no separate draft model required.

Safety

Refusal behavior has been removed; this model will follow instructions it would normally decline. Intended for research and authorized red-teaming. You are responsible for your use.