h3rb3rn/sovereign-judge-27b

MoE Sovereign Judge 27B (sovereign-judge-27b)

Evaluation, Verification & Synthesis Authority

 

Model Summary

sovereign-judge-27b is a LoRA fine-tune of the text-decoder of Qwen3.8-27B, specialized as the evaluation and synthesis authority within the MoE Sovereign compound-AI system: it checks the quality, factual consistency, and code-level correctness of outputs produced by the Planner and Expert models, and either confirms them or produces a corrected result.

This is the Spur-1 (open-weight) judge, trained on a text-only backbone extracted from the multimodal Qwen3.8-27B checkpoint (Qwen3_5ForConditionalGeneration -> Qwen3_5ForCausalLM). The parallel Spur-2 (open-source) judge uses OLMo-3.1-32B on the same critic-format dataset.

Base Architecture

Qwen3.8-27B is a hybrid linear-attention / full-attention decoder, extracted to a text-only Qwen3_5ForCausalLM backbone (vision tower and MTP head dropped) for compatibility with standard causal-LM fine-tuning.

Training Configuration

Training Data Composition

Training examples follow a critic format: a QUESTION (a request as another expert would receive it) paired with an ANSWER TO CHECK (a candidate response). The target output is either the bare word CONFIRMED (the candidate answer is already correct and complete) or a corrected, complete replacement answer.

Observed Training Trajectory

Training loss: 4.423 -> 2.340 -> 1.181 -> 0.882. Smooth, monotonic decline across the run.

Prompt Format

ChatML:

<|im_start|>system
{system_prompt}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
{response}<|im_end|>

Recommended System Prompt

You are the Sovereign Judge, the primary evaluation and synthesis authority in the MoE Sovereign compound AI platform. Evaluate input quality, factual consistency, code invariants, and safety with maximum precision. Respond with the single word CONFIRMED if the candidate answer is correct and complete; otherwise respond with only the corrected answer, with no preamble.

Recommended Critic Input Shape

QUESTION:
<the original request>

ANSWER TO CHECK:
<the candidate response to verify>

Available Formats

Hardware Guidance

Largest model in the Spur-1 lineup -- requires a multi-GPU pool, not a single 8GB card. Q4_K_M needs roughly 17GB of VRAM for weights alone, plus KV-cache headroom. Native context is inherited from Qwen3.8 (262,144 tokens); on Maxwell-generation GPUs (no Flash Attention), use f16 KV-cache and a reduced num_ctx.

Ollama Modelfile

FROM ./sovereign-judge-27b-Q4_K_M.gguf
SYSTEM """You are the Sovereign Judge, the primary evaluation and synthesis authority in the MoE Sovereign compound AI platform. Evaluate input quality, factual consistency, code invariants, and safety with maximum precision. Respond with the single word CONFIRMED if the candidate answer is correct and complete; otherwise respond with only the corrected answer, with no preamble."""
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Response }}<|im_end|>"""
PARAMETER stop "<|im_end|>"
PARAMETER temperature 0.1
PARAMETER num_ctx 32768

Intended Use

• —Final-stage verification of Planner/Expert outputs in a compound-AI pipeline
• —Fact-checking and correctness verification of short factual or code-correctness claims
• —Synthesis of a corrected answer when a candidate response is incomplete or wrong

Limitations

• —Judgments reflect patterns learned from its training distribution, not a formally verified reasoning process -- do not treat CONFIRMED as a mathematical proof of correctness for high-stakes decisions.
• —The critic format is optimized for single-answer verification; it is not tuned for open-ended multi-turn conversation.

License

Apache 2.0, inherited from the Qwen3.8-27B base model.