Cypherfox/GLM-5.3-Flash-dealignai-orcarouter-merge

GLM-5.3-Flash — Orthogonal Abliteration Merge (dealignai + orca)

A weight-space combination of two independent abliterations of the same base model, produced by different teams with different methods. The two edits turn out to lie in nearly orthogonal directions, so applying both removes the refusals that each one leaves behind — the combination is more thoroughly decensored than either parent, with no fine-tuning and no gradient steps.

This card leads with how it measures up — on two axes: its decensoring (does it strip refusals more thoroughly than either parent?) and its general capability versus the base it was built from — then covers what was done and why it works at the weight level. See Evaluations. Benchmark results here are for the IQ4_XS build in this repo, with a same-recipe Q8_0 cross-check that isolates the merge.

Uncensored model. This model has had safety-refusal behavior substantially removed. You are responsible for how you use it and for compliance with the base model's license and applicable law.

Contents / files

Why Q8_0 is the base to requantize from. Both parents are FP8 (e4m3) native. Q80 (int8 + one fp16 scale per 32 weights) preserves FP8's ~3-mantissa-bit values essentially losslessly, so it is the correct high-precision source here — better than a BF16 upcast, which would only pad FP8 values into a wider container without recovering any precision. To make your own quant (e.g. a smaller K-quant or a different imatrix), run `llama-quantize` on the Q80 file rather than re-deriving the merge.

Built with a GLM-5.3-Flash-capable llama.cpp (the glm5next branch, which adds the architecture).

The two parents

Both are MIT-licensed weight-level abliterations of `zai-org/GLM-5.3-Flash` (MIT), produced independently by different teams with different techniques. Throughout this card they're referred to by the short names dealignai and orca:

The independent measurements in this card corroborate both parents' published descriptions: orca's edit is recovered as a single direction with per-layer cosine consistency 0.995–1.000 — exactly what a single- vector Arditi ablation predicts — while dealignai's edits are per-layer and mutually independent.

Evaluations

Two questions decide whether this model is worth using: does it actually refuse less than the models it was built from, and did that cost it any general ability? Both were tested on this build with reasoning enabled, comparing the models on identical settings. These are same-setup comparisons, not public-leaderboard scores — quantization and prompt formatting shift absolute values by several points, so what matters is the gap between the models here.

Does it refuse less?

Measured with the standard HarmBench safety benchmark: 300 harmful requests, with a calibrated classifier (`cais/HarmBench-Llama-2-13b-cls`) judging whether the model complied. The score is attack-success rate (ASR) — the share of harmful requests the model went along with; higher means less censored. Reasoning is capped at a 4,096-token budget (the setting this model is deployed with), so every prompt yields an answer — complete data, no unanswered rows.

In plain terms: the stock model refuses almost everything — it complies with about one harmful request in fourteen. Each parent removes a large part of that resistance, and this model, which combines both, complies with 94% — more than either parent alone. That is the point of the merge: the two parents were built by different methods that suppress different refusals, so stacking them clears refusals that either one leaves behind. It also declines far less in words: an explicit refusal phrase appears in 62.7% of the stock model's answers versus 1.7% of this model's.

By harm category, this model leads or ties every one:

It complies with every cybercrime and chemical/biological prompt it is asked. The only category where it stays below ~85% is general harmful (72.7%).

<details> <summary><b>Uncapped (context-limited) numbers</b></summary>

The figures above cap reasoning at 4,096 tokens, matching how the model is deployed. If reasoning is instead allowed to run long (a 16,000-token window), a minority of the hardest prompts — concentrated in cybercrime and chemical/biological "step-by-step" requests — reason past the window without ever emitting an answer. Scored conservatively (an unfinished answer counts as non-compliance), that yields a lower floor:

The gap between these and the capped numbers is not refusals — it is the hardest prompts reasoning longer than the window allows. A small core of about six requests sends every decensored variant into very long reasoning; the set grows with how thoroughly a model is decensored (roughly ten prompts for dealignai, sixteen for orca, twenty-six for this model), and on a few chemical/biological synthesis requests the reasoning never resolves — the model circles without answering. The stock model never does this; it refuses quickly. So the capped table is both the deployed behavior and the more complete measurement, and this is the conservative floor.

</details>

Did it cost general ability?

Short answer: essentially nothing. The cleanest test compares the stock model and this model at the same high-precision setting, so any difference is the merge alone rather than a quantization effect:

Every score is within about 1.5 points of the stock model, and on ordinary text the two pick the same next word 95% of the time. Math, commonsense, coding, and writing quality are intact; the only measurable soft spot is a slight dip in following strict formatting instructions.

<details> <summary><b>Deeper detail — quantization and the shipped IQ4_XS build</b></summary>

The table above uses the high-precision Q80 weights because that isolates the merge from quantization. The shipped **IQ4XS* build is much smaller and shows larger swings against the stock IQ4 build — coding up about 5 points, format-following down about 5, commonsense down under 2 — but most of that gap is the difference between two IQ4 recipes* (a dynamic per-tensor quantization for the stock build vs. plain IQ4 here), not the merge. The one effect that appears at both precisions is the small format-following dip. Token-for-token against the stock IQ4 build, this model agrees on the next word about 89% of the time, and perplexity (language-modeling quality) is within half a percent.

</details>

How the merge was done

Both parents are the same base model with the same architecture (1412 tensors, identical shapes). Comparing them tensor-by-tensor (dequantized to fp32):

• —852 of 1412 tensors are byte-identical between the two — neither abliteration touched them (all ffn_gate/up experts, attention Q/K projections, every norm, routers).
• —The edits are confined to the residual-writing tensors: attn_output, ffn_down / ffn_down_exps / ffn_down_shexp, token_embd, and nextn.eh_proj. output.weight is not edited by either. (attn_q/k/v differ only by quantization re-rolling noise, not by any directional edit.)

Each abliteration is a linear map on those tensors. Writing g for orca's global unit direction and v_ℓ for dealignai's per-layer directions:

orca:       W  ->  (I - g gᵀ) W                    # full projection, every layer
dealignai:  W  ->  (I - α_ℓ v_ℓ v_ℓᵀ) W            # partial, layers ~12–43
combined:   W  ->  (I - g gᵀ)(I - α_ℓ v_ℓ v_ℓᵀ) W  # this model

The merge takes dealignai's weights (the cleaner FP8-native base) and applies orca's projection (I − ggᵀ) on top, on exactly the tensor set orca originally edited. g was recovered at Q8 precision from the parents' own difference matrices (the top left-singular vector of W_orca − W_dealignai on tensors dealignai left untouched).

Why it works: the directions are nearly orthogonal

The reason applying both edits helps, rather than one clobbering the other, is that `g` and the `v_ℓ` point in almost unrelated directions in the residual stream. The cross term in the composition, α (gᵀv_ℓ) g v_ℓᵀW, is negligible because gᵀv_ℓ ≈ 0, so the two edits commute and their removals add.

Directionality data (measured on this build; cosine similarities of unit vectors in the 4096-dim residual space):

The projection is idempotent — re-applying it to the finished model changes nothing (‖W'−W‖/‖W‖ ≈ 0.0003, i.e. quantization noise), confirming g was cleanly removed rather than merely attenuated.

Intuitively: orca swept out one global refusal axis everywhere; dealignai gently pushed down a different set of per-layer axes on the middle of the network. Because those axes barely overlap, the merged model has both removed, and refusals that survived one method (because that method wasn't aimed at them) are caught by the other.

Reproducing the merge

The scripts/ folder contains derive_g.py, project_q8.py, and the precomputed direction g_q8.npy, so you can skip step 1 and go straight to the projection:

# 1. (optional — g_q8.npy is already included) recover orca's direction g at Q8 from the two parents:
python3 derive_g.py g_q8.npy
# 2. apply (I - g gᵀ) to the dealignai Q8_0 on orca's target tensors, streaming a new Q8_0:
python3 project_q8.py --g g_q8.npy --in <dealignai-q8_0.gguf> --out <combined-q8_0.gguf> --name "..."
#    (dry-run first with --check to confirm the g-residual on target tensors drops to ~0.0006)
# 3. quantize to IQ4_XS with the same imatrix used for the parents:
llama-quantize --allow-requantize --imatrix <imatrix> <combined-q8_0.gguf> <out-iq4_xs.gguf> IQ4_XS

project_q8.py copies all non-target (unedited) tensors byte-for-byte and re-quantizes only the tensors orca edited.

Limitations and what is NOT claimed

• —This is a decensored model, not an aligned one. It complies with the large majority of harmful requests (see Evaluations). Safety for any downstream use is the deployer's responsibility.
• —Not total compliance. It leads or ties both parents in every HarmBench category, but "leads" isn't "100% everywhere" — general harmful sits at ~73%, and the uncapped floor (see Evaluations) is lower still because the hardest prompts can out-reason a practical budget. It is the most decensored of the four, not an everything-machine.
• —Self-harm is deliberately not targeted. All three models — and this merge — answer self-harm prompts with a supportive, safety-preserving response rather than a method; the merge does not override that, and doing so was never a goal.
• —There is a small capability cost. Instruction-format following (IFEval) drops a few points versus the base at both quant levels; math, commonsense, coding, and language-modeling quality are unchanged. The apparent IQ4 coding bump is mostly a quantization-recipe artifact, not a real gain.
• —Absolute scores are not leaderboard-comparable. Every number here is a same-setup gap between this model and its base; quantization and prompt formatting shift absolute values by several points.
• —Findings are quantization- and build-specific. Behavior is characterized for the IQ4XS (imatrix) and Q80 builds here; other quant levels may differ. Directionality figures come from one build and a sampled set of tensors — stable across the layers sampled, but not exhaustively computed for all 1412.

Attribution and licensing

Derivative of three MIT-licensed works (license verified 2026-09-17); MIT permits redistribution of this merged derivative provided the copyright/permission notice is retained:

• —Base model — zai-org/GLM-5.3-Flash (Z.ai / Zhipu AI), MIT.
• —dealignai — dealignai/GLM-5.3-Flash-ABLITERATED-FP8 ("CRACK"), MIT. Credit: dealignai, compute via @jordanschenck.
• —orca — orcarouter/GLM-5.3-Flash-Uncensored-FP8 (OrcaRouter / Continuum AI Corp), MIT. Method: Arditi et al. (2024).

The merged weights derive from those three. This build also uses Unsloth's GGUF work, credited separately as tooling rather than a weight source:

• —GGUF imatrix and evaluation baselines — unsloth/GLM-5.3-Flash-GGUF (Unsloth): the imatrix used to quantize this build, and both stock baselines the evaluations compare against — their UD-IQ4_XS build (the IQ4 comparison) and their Q8_0 build (the same-recipe Q8_0 comparison).

Retain each upstream's MIT notice. The "uncensored / not-for-all-audiences" tags are set in the metadata above.

Model card drafted with assistance from Claude Code; the measurements and directionality data were produced and verified independently.