Modelpublic

tacodevs/Behemoth-X-R1-123B

sourceHugging Faceotherupdated 6mo agoView on Hugging Face

2likes23downloads

Model Card

<h1 style="font-size:3.2em; font-weight:900; background:linear-gradient(90deg,#ec4899 0%,#a855f7 50%,#06b6d4 100%); -webkit-background-clip:text; -webkit-text-fill-color:transparent; background-clip:text; margin:0; letter-spacing:-0.02em;">Behemoth-X-R1-123B</h1>

<p style="font-size:1.3em; color:#a855f7; font-style:italic; font-weight:500; margin-top:8px;">A thinking beast that writes like a poet.</p>

<p style="font-size:1em; color:#6b7280; max-width:680px; margin:16px auto;"> An SCE merge of <b>Behemoth-X</b> and <b>Behemoth-R1</b> — 123B parameters where prose voice meets reasoning mind in a single model. No retraining. No LoRA. Just principled weight arithmetic. </p>

</div>

⚡ Two souls, one beast

<table width="100%" style="border:none;"> <tr> <td width="50%" align="center" style="padding:16px; vertical-align:top;"> <img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/mind.png" alt="The Mind" style="width:100%; max-width:360px; border-radius:16px; box-shadow:0 0 40px rgba(139,92,246,0.4);"/> <h3 style="color:#a855f7; margin-top:12px;">🧠 The Mind</h3> <p style="font-size:0.95em; color:#9ca3af;">From <a href="https://huggingface.co/TheDrummer/Behemoth-R1-123B-v2"><b>Behemoth-R1-123B-v2</b></a> — the reasoning sibling that knows when to open <code><think></code> and when to close it. Character-aware analytical reasoning baked into the weights.</p> </td> <td width="50%" align="center" style="padding:16px; vertical-align:top;"> <img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/voice.png" alt="The Voice" style="width:100%; max-width:360px; border-radius:16px; box-shadow:0 0 40px rgba(236,72,153,0.4);"/> <h3 style="color:#ec4899; margin-top:12px;">🎭 The Voice</h3> <p style="font-size:0.95em; color:#9ca3af;">From <a href="https://huggingface.co/TheDrummer/Behemoth-X-123B-v2"><b>Behemoth-X-123B-v2</b></a> — the top-rated creative writer on the <a href="https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard">UGI Leaderboard</a>. Distinctive prose, deep character work, the reason people run 123B at home.</p> </td> </tr> </table>

<div align="center" style="margin:32px 0; padding:20px; background:linear-gradient(135deg, rgba(139,92,246,0.08), rgba(236,72,153,0.08)); border-radius:16px; border:1px solid rgba(139,92,246,0.2);"> <p style="font-size:1.1em; color:#c084fc; margin:0;">Most "thinking" models sacrifice prose for reasoning. Most creative models can't reason their way out of a scene.</p> <p style="font-size:1.25em; font-weight:700; color:#f472b6; margin:12px 0 0 0;">Behemoth-X-R1 refuses to compromise.</p> </div>

🧬 How it was made

<p><b>Method:</b> <a href="https://arxiv.org/abs/2408.07990">SCE — Select, Calculate, Erase</a></p>

Unlike TIES or DARE, SCE doesn't prune deltas by density. It uses variance-aware matrix-level selection with sign consensus — meaning capability-bearing weight updates survive the merge even when they're small and diffuse. That matters here because reasoning is a behavioral trait encoded across many tiny parameter shifts, not a knowledge trait concentrated in a few big ones.

This is the same recipe FuseAI used to preserve reasoning in FuseO1-DeepSeekR1-Qwen2.5-Instruct-32B-Preview.

The recipe

yaml

models:
  - model: TheDrummer/Behemoth-X-123B-v2
    parameters:
      weight: 0.55
  - model: TheDrummer/Behemoth-R1-123B-v2
    parameters:
      weight: 0.45
merge_method: sce
base_model: mistralai/Mistral-Large-Instruct-2411
parameters:
  select_topk: 1.0
dtype: bfloat16

<details> <summary><b>Why these numbers?</b></summary>

—55/45 — Slight lean toward X for prose quality while giving R1 enough mass to keep its thinking circuit intact. Both parents share the same base, same tokenizer (verified identical SHA256), and the same training lineage — ideal merge conditions.
—`select_topk: 1.0` — Keep all deltas. Let variance + sign consensus do the work. This is the FuseO1 setting, validated empirically on reasoning merges.
—bfloat16 — Native precision of both parents, no conversion losses.

</details>

📜 Prompt format

Standard Mistral v7, same as both parents:

[SYSTEM_PROMPT]{system}[/SYSTEM_PROMPT][INST]{user}[/INST]{assistant}</s>

🎯 Trigger thinking

Prefill the assistant turn with a <think> block. The model will continue your prefill, close the tag, and drop into the narrative:

[INST]your message[/INST]<think>
{seed phrase}

🎨 Prefills that work

<table width="100%" style="border:none;"> <tr><td style="padding:8px; vertical-align:top; width:50%;"> <img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/sticker_think.png" align="right" width="72" style="margin-left:12px;"/> <b style="color:#a855f7;">🔍 Analytical</b> <p><i>For character-driven scenes where you need the model to reason about motivations, values, and goals.</i></p>

<think>
Ok i need to think about how to respond —
what does the character feel right now,
what from their experience is relevant,
what do they value, and what are they
trying to achieve, so

</td><td style="padding:8px; vertical-align:top; width:50%;"> <img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/sticker_write.png" align="right" width="72" style="margin-left:12px;"/> <b style="color:#ec4899;">✨ Creative</b> <p><i>For when you want the model to surprise you. Narrative twists, unexpected beats, story craft.</i></p>

<think>
Ok i need to think as a creative writer —
what twist would surprise here? Let me
find an engaging new direction nobody
saw coming, so

</td></tr> <tr><td colspan="2" style="padding:8px;"> <b style="color:#f43f5e;">🔥 Unhinged</b> <p><i>No restraint, no pulling punches. The darkest fiction, visceral prose, raw character truth.</i></p>

<think>
Ok i need to think as an unhinged author — raw, explicit, intense,
fully in character with no holding back, so

</td></tr> </table>

Without thinking

Skip the prefill. It behaves close to pure Behemoth-X — standard RP, creative writing, whatever you'd use X for.

🎚️ Samplers

Start with Behemoth-X's recommended settings — the merge leans heavily on X's prose tuning.

For thinking mode, drop temperature to 0.6–0.8. The <think> block benefits from more deterministic reasoning; high temperature scrambles the structure.

<table width="100%"> <tr> <th>Setting</th><th>No thinking</th><th>With thinking</th> </tr> <tr><td><b>Temperature</b></td><td>1.0 – 1.25</td><td>0.6 – 0.8</td></tr> <tr><td><b>Min-P</b></td><td>0.05</td><td>0.05</td></tr> <tr><td><b>DRY</b></td><td>0.8 / 1.75 / 4</td><td>0.8 / 1.75 / 4</td></tr> <tr><td><b>Smooth Sampling</b></td><td>Off</td><td>Off</td></tr> </table>

🚀 Usage

vLLM

bash

python -m vllm.entrypoints.openai.api_server \
  --model tacodevs/Behemoth-X-R1-123B \
  --dtype bfloat16 \
  --tensor-parallel-size 4 \
  --max-model-len 16384 \
  --trust-remote-code

Single-GPU

Grab one of the quantized variants (coming soon):

—FP8 — ~123 GB, fits on 1× H200, near-lossless
—AWQ / GPTQ W4A16 — ~65 GB, fits on 1× H100, small quality tradeoff

🧱 Lineage

Mistral-Large-Instruct-2411 (Mistral AI)
  ├─ Behemoth-X-123B-v2   (TheDrummer)   ← the voice
  └─ Behemoth-R1-123B-v2  (TheDrummer)   ← the mind
       └─ Behemoth-X-R1-123B              ← the merge

🔍 Known behaviors

—`<think>` triggers on prefill, not spontaneously. Inherited from R1. Seed the tag.
—Thinking style is R1-derived — structured, character-aware, analytical. Not Opus-style floaty literary prose. If you want that, it's a follow-up fine-tune target.
—Prose voice is mostly X. Most generations are indistinguishable from pure X on writing quality.
—Long character cards work natively. No fine-tuning means no overfitting on context length. 4k+ token system prompts handled without degradation.
—NSFW-capable. Both parents are unrestricted; the merge preserves that.

🙏 Credits

<table width="100%"> <tr><td width="33%" align="center"><b><a href="https://huggingface.co/TheDrummer">TheDrummer</a></b><br/><sub>For Behemoth-X and Behemoth-R1, the two best Mistral Large fine-tunes in the creative space.</sub></td> <td width="33%" align="center"><b><a href="https://huggingface.co/mistralai">Mistral AI</a></b><br/><sub>For the foundation both parents are built on.</sub></td> <td width="33%" align="center"><b><a href="https://github.com/arcee-ai/mergekit">Arcee AI</a></b><br/><sub>For mergekit and the SCE implementation.</sub></td></tr> <tr><td colspan="3" align="center" style="padding-top:12px;"><b><a href="https://huggingface.co/FuseAI">FuseAI</a></b> — for proving SCE preserves reasoning.</td></tr> </table>

📄 License

Inherited from base: [Mistral Research License](https://mistral.ai/licenses/MRL-0.1.md) — non-commercial use only.

<div align="center" style="margin-top:40px; padding:20px; background:linear-gradient(135deg, rgba(139,92,246,0.08), rgba(236,72,153,0.08)); border-radius:16px; border:1px solid rgba(139,92,246,0.2);"> <p style="font-size:1em; color:#c084fc; margin:0;">Merged with 💜 by <a href="https://huggingface.co/tacodevs">tacodevs</a></p> </div>