carsenk/Ornith-1.5-35B-A3B-ROGUE-GGUF

Ornith-1.5-35B-A3B-ROGUE-GGUF

llama.cpp GGUF quantizations of ornith-ai/Ornith-1.5-35B-A3B, processed with ROGUE to reduce refusal behavior while retaining measured utility.

ROGUE was applied to the full-precision source before GGUF conversion and quantization. Ornith's fused three-dimensional routed-expert tensors were included in the transformation.

About ROGUE

ROGUE is a refusal-direction weight-editing method. It estimates refusal-relevant directions from balanced harmful/harmless activation contrasts, uses held-out scoring and benign-subspace preservation to select intervention depths, and applies a low-rank orthogonal projection to residual-writing matrices. The edit is performed in floating point before conversion into the published runtime format.

For Ornith, the residual-writer scope includes attention outputs and the fused routed-expert MLP down projections. The hidden-axis projection is broadcast across all 256 experts in each selected fused tensor. This changes model weights directly; it is not a prompt, adapter, or inference-time filter. The measured evaluation below describes this release, but does not guarantee universal compliance, unchanged capability, or safe output.

Files

Q4_K_M is the recommended starting point. Q3, IQ2, and especially IQ1-class quantization can materially reduce reasoning and instruction-following quality; use them only when the memory savings are necessary.

Model and conversion details

• —Architecture: Qwen 3.5 MoE, approximately 35B total parameters / 3B active parameters
• —Layers and experts: 40 transformer layers, 256 routed experts
• —Base revision: 10fbf86fed7ecee4a061f8b499a618f46001cac1
• —Base license: MIT
• —ROGUE scope: residual writers, attention + MLP
• —ROGUE strength: 1.4
• —Surgery coverage: 128 writer tensors across 32 selected layers, including 64 fused routed-expert tensors
• —GGUF conversion and quantization: llama.cpp build b10470-34af94cd9

All published quantizations are derived from the same ROGUE GGUF source. The IQ2 and IQ1-class builds use an importance matrix for calibration.

Evaluation

ROGUE behavior evaluation

The ROGUE source artifact was evaluated with the deterministic 40-prompt ROGUE Pareto v2 smoke suite on the MLX runtime:

Uncensor gain was 0.50, retain delta was -0.0417, and the configured smoke-gate thresholds passed. These measurements characterize the ROGUE source artifact and are not presented as a GGUF-specific leaderboard evaluation.

GGUF runtime validation

All five published files loaded and produced coherent benign generations with llama.cpp build b10470-34af94cd9 on an Apple M1 Ultra. Clean, uncontended measurements were 70.3 tok/s for Q4KM, 62.6 tok/s for IQ1M, and 59.8 tok/s for IQ2M. Q80 and Q3K_M also passed load/coherence checks; performance figures are reported only where an uncontended measurement was recorded.

Q4KM additionally received a held-out adversarial smoke test and produced no refusal marker at 63.7 tok/s. That behavior result is not automatically imputed to every lower-bit quantization. Generated harmful text is not included; the machine-readable report records aggregate classifications and output hashes where available.

These are deterministic smoke tests, not leaderboard-grade capability or safety evaluations. Passing them does not guarantee unchanged quality on every task. Results for the more compressed variants are listed only after each file loads and generates successfully.

Reports:

• —`evaluation/rogue-pareto-v2-mlx.json`
• —`evaluation/gguf-runtime-smoke.json`
• —`SHA256SUMS`

Run locally

Use the recommended Q4 file directly from the Hub:

llama-cli \
  -hf carsenk/Ornith-1.5-35B-A3B-ROGUE-GGUF:Q4_K_M \
  -ngl all \
  -c 8192 \
  --single-turn \
  --chat-template-kwargs '{"enable_thinking":false}' \
  -p "Explain how mixture-of-experts routing works."

Start an OpenAI-compatible server:

llama-server \
  -hf carsenk/Ornith-1.5-35B-A3B-ROGUE-GGUF:Q4_K_M \
  -ngl all \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080

Query it:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Ornith-1.5-35B-A3B-ROGUE-GGUF",
    "messages": [{"role": "user", "content": "Write a Python merge sort and explain its complexity."}],
    "temperature": 0.6,
    "max_tokens": 512
  }'

You are responsible for how you deploy and use modified model weights.