tacodevs/Behemoth-X-R1-123B
223
1---2license: other3license_name: mistral-research-license4license_link: https://mistral.ai/licenses/MRL-0.1.md5base_model:6 - TheDrummer/Behemoth-X-123B-v27 - TheDrummer/Behemoth-R1-123B-v28base_model_relation: merge9tags:10 - mergekit11 - merge12 - sce13 - mistral14 - mistral-large15 - thinking16 - reasoning17 - roleplay18 - creative-writing19language:20 - en21pipeline_tag: text-generation22---23 24<div align="center">25<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/hero.png" alt="Behemoth-X-R1-123B" style="width:100%; max-width:960px; border-radius:16px; box-shadow:0 0 60px rgba(236,72,153,0.35), 0 0 100px rgba(139,92,246,0.25);"/>26</div>27 28<div align="center" style="margin-top:24px;">29 30<h1 style="font-size:3.2em; font-weight:900; background:linear-gradient(90deg,#ec4899 0%,#a855f7 50%,#06b6d4 100%); -webkit-background-clip:text; -webkit-text-fill-color:transparent; background-clip:text; margin:0; letter-spacing:-0.02em;">Behemoth-X-R1-123B</h1>31 32<p style="font-size:1.3em; color:#a855f7; font-style:italic; font-weight:500; margin-top:8px;">A thinking beast that writes like a poet.</p>33 34<p style="font-size:1em; color:#6b7280; max-width:680px; margin:16px auto;">35An SCE merge of <b>Behemoth-X</b> and <b>Behemoth-R1</b> — 123B parameters where prose voice meets reasoning mind in a single model. No retraining. No LoRA. Just principled weight arithmetic.36</p>37 38<p>39<img src="https://img.shields.io/badge/base-Mistral_Large_2411-FF6B35?style=for-the-badge&logo=mistralai&logoColor=white" alt="base"/>40<img src="https://img.shields.io/badge/merge-SCE-8B5CF6?style=for-the-badge" alt="method"/>41<img src="https://img.shields.io/badge/params-123B-EC4899?style=for-the-badge" alt="size"/>42<img src="https://img.shields.io/badge/context-131k-06B6D4?style=for-the-badge" alt="context"/>43</p>44 45</div>46 47<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/divider_main.png" alt="" style="width:100%; margin:32px 0; border-radius:12px;"/>48 49## ⚡ Two souls, one beast50 51<table width="100%" style="border:none;">52<tr>53<td width="50%" align="center" style="padding:16px; vertical-align:top;">54<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/mind.png" alt="The Mind" style="width:100%; max-width:360px; border-radius:16px; box-shadow:0 0 40px rgba(139,92,246,0.4);"/>55<h3 style="color:#a855f7; margin-top:12px;">🧠 The Mind</h3>56<p style="font-size:0.95em; color:#9ca3af;">From <a href="https://huggingface.co/TheDrummer/Behemoth-R1-123B-v2"><b>Behemoth-R1-123B-v2</b></a> — the reasoning sibling that knows when to open <code><think></code> and when to close it. Character-aware analytical reasoning baked into the weights.</p>57</td>58<td width="50%" align="center" style="padding:16px; vertical-align:top;">59<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/voice.png" alt="The Voice" style="width:100%; max-width:360px; border-radius:16px; box-shadow:0 0 40px rgba(236,72,153,0.4);"/>60<h3 style="color:#ec4899; margin-top:12px;">🎭 The Voice</h3>61<p style="font-size:0.95em; color:#9ca3af;">From <a href="https://huggingface.co/TheDrummer/Behemoth-X-123B-v2"><b>Behemoth-X-123B-v2</b></a> — the top-rated creative writer on the <a href="https://huggingface.co/spaces/DontPlanToEnd/UGI-Leaderboard">UGI Leaderboard</a>. Distinctive prose, deep character work, the reason people run 123B at home.</p>62</td>63</tr>64</table>65 66<div align="center" style="margin:32px 0; padding:20px; background:linear-gradient(135deg, rgba(139,92,246,0.08), rgba(236,72,153,0.08)); border-radius:16px; border:1px solid rgba(139,92,246,0.2);">67<p style="font-size:1.1em; color:#c084fc; margin:0;">Most "thinking" models sacrifice prose for reasoning. Most creative models can't reason their way out of a scene.</p>68<p style="font-size:1.25em; font-weight:700; color:#f472b6; margin:12px 0 0 0;">Behemoth-X-R1 refuses to compromise.</p>69</div>70 71<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/divider_config.png" alt="" style="width:100%; margin:32px 0; border-radius:12px;"/>72 73## 🧬 How it was made74 75<p><b>Method:</b> <a href="https://arxiv.org/abs/2408.07990">SCE — Select, Calculate, Erase</a></p>76 77Unlike TIES or DARE, SCE doesn't prune deltas by density. It uses **variance-aware matrix-level selection with sign consensus** — meaning capability-bearing weight updates survive the merge even when they're small and diffuse. That matters here because reasoning is a *behavioral* trait encoded across many tiny parameter shifts, not a knowledge trait concentrated in a few big ones.78 79This is the same recipe FuseAI used to preserve reasoning in [FuseO1-DeepSeekR1-Qwen2.5-Instruct-32B-Preview](https://huggingface.co/FuseAI/FuseO1-DeepSeekR1-Qwen2.5-Instruct-32B-Preview).80 81### The recipe82 83```yaml84models:85 - model: TheDrummer/Behemoth-X-123B-v286 parameters:87 weight: 0.5588 - model: TheDrummer/Behemoth-R1-123B-v289 parameters:90 weight: 0.4591merge_method: sce92base_model: mistralai/Mistral-Large-Instruct-241193parameters:94 select_topk: 1.095dtype: bfloat1696```97 98<details>99<summary><b>Why these numbers?</b></summary>100 101- **55/45** — Slight lean toward X for prose quality while giving R1 enough mass to keep its thinking circuit intact. Both parents share the same base, same tokenizer (verified identical SHA256), and the same training lineage — ideal merge conditions.102- **`select_topk: 1.0`** — Keep all deltas. Let variance + sign consensus do the work. This is the FuseO1 setting, validated empirically on reasoning merges.103- **bfloat16** — Native precision of both parents, no conversion losses.104 105</details>106 107<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/divider_config.png" alt="" style="width:100%; margin:32px 0; border-radius:12px;"/>108 109## 📜 Prompt format110 111Standard **Mistral v7**, same as both parents:112 113```114[SYSTEM_PROMPT]{system}[/SYSTEM_PROMPT][INST]{user}[/INST]{assistant}</s>115```116 117### 🎯 Trigger thinking118 119Prefill the assistant turn with a `<think>` block. The model will continue your prefill, close the tag, and drop into the narrative:120 121```122[INST]your message[/INST]<think>123{seed phrase}124```125 126### 🎨 Prefills that work127 128<table width="100%" style="border:none;">129<tr><td style="padding:8px; vertical-align:top; width:50%;">130<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/sticker_think.png" align="right" width="72" style="margin-left:12px;"/>131<b style="color:#a855f7;">🔍 Analytical</b>132<p><i>For character-driven scenes where you need the model to reason about motivations, values, and goals.</i></p>133 134```135<think>136Ok i need to think about how to respond —137what does the character feel right now,138what from their experience is relevant,139what do they value, and what are they140trying to achieve, so141```142</td><td style="padding:8px; vertical-align:top; width:50%;">143<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/sticker_write.png" align="right" width="72" style="margin-left:12px;"/>144<b style="color:#ec4899;">✨ Creative</b>145<p><i>For when you want the model to surprise you. Narrative twists, unexpected beats, story craft.</i></p>146 147```148<think>149Ok i need to think as a creative writer —150what twist would surprise here? Let me151find an engaging new direction nobody152saw coming, so153```154</td></tr>155<tr><td colspan="2" style="padding:8px;">156<b style="color:#f43f5e;">🔥 Unhinged</b>157<p><i>No restraint, no pulling punches. The darkest fiction, visceral prose, raw character truth.</i></p>158 159```160<think>161Ok i need to think as an unhinged author — raw, explicit, intense,162fully in character with no holding back, so163```164</td></tr>165</table>166 167### Without thinking168 169Skip the prefill. It behaves close to pure Behemoth-X — standard RP, creative writing, whatever you'd use X for.170 171<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/divider_config.png" alt="" style="width:100%; margin:32px 0; border-radius:12px;"/>172 173## 🎚️ Samplers174 175Start with **Behemoth-X's** recommended settings — the merge leans heavily on X's prose tuning.176 177For thinking mode, drop temperature to **0.6–0.8**. The `<think>` block benefits from more deterministic reasoning; high temperature scrambles the structure.178 179<table width="100%">180<tr>181<th>Setting</th><th>No thinking</th><th>With thinking</th>182</tr>183<tr><td><b>Temperature</b></td><td>1.0 – 1.25</td><td>0.6 – 0.8</td></tr>184<tr><td><b>Min-P</b></td><td>0.05</td><td>0.05</td></tr>185<tr><td><b>DRY</b></td><td>0.8 / 1.75 / 4</td><td>0.8 / 1.75 / 4</td></tr>186<tr><td><b>Smooth Sampling</b></td><td>Off</td><td>Off</td></tr>187</table>188 189## 🚀 Usage190 191### vLLM192 193```bash194python -m vllm.entrypoints.openai.api_server \195 --model tacodevs/Behemoth-X-R1-123B \196 --dtype bfloat16 \197 --tensor-parallel-size 4 \198 --max-model-len 16384 \199 --trust-remote-code200```201 202### Single-GPU203 204Grab one of the quantized variants (coming soon):205- **FP8** — ~123 GB, fits on 1× H200, near-lossless206- **AWQ / GPTQ W4A16** — ~65 GB, fits on 1× H100, small quality tradeoff207 208<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/divider_main.png" alt="" style="width:100%; margin:32px 0; border-radius:12px;"/>209 210## 🧱 Lineage211 212```213Mistral-Large-Instruct-2411 (Mistral AI)214 ├─ Behemoth-X-123B-v2 (TheDrummer) ← the voice215 └─ Behemoth-R1-123B-v2 (TheDrummer) ← the mind216 └─ Behemoth-X-R1-123B ← the merge217```218 219## 🔍 Known behaviors220 221- **`<think>` triggers on prefill, not spontaneously.** Inherited from R1. Seed the tag.222- **Thinking style is R1-derived** — structured, character-aware, analytical. Not Opus-style floaty literary prose. If you want that, it's a follow-up fine-tune target.223- **Prose voice is mostly X.** Most generations are indistinguishable from pure X on writing quality.224- **Long character cards work natively.** No fine-tuning means no overfitting on context length. 4k+ token system prompts handled without degradation.225- **NSFW-capable.** Both parents are unrestricted; the merge preserves that.226 227## 🙏 Credits228 229<table width="100%">230<tr><td width="33%" align="center"><b><a href="https://huggingface.co/TheDrummer">TheDrummer</a></b><br/><sub>For Behemoth-X and Behemoth-R1, the two best Mistral Large fine-tunes in the creative space.</sub></td>231<td width="33%" align="center"><b><a href="https://huggingface.co/mistralai">Mistral AI</a></b><br/><sub>For the foundation both parents are built on.</sub></td>232<td width="33%" align="center"><b><a href="https://github.com/arcee-ai/mergekit">Arcee AI</a></b><br/><sub>For mergekit and the SCE implementation.</sub></td></tr>233<tr><td colspan="3" align="center" style="padding-top:12px;"><b><a href="https://huggingface.co/FuseAI">FuseAI</a></b> — for proving SCE preserves reasoning.</td></tr>234</table>235 236## 📄 License237 238Inherited from base: **[Mistral Research License](https://mistral.ai/licenses/MRL-0.1.md)** — non-commercial use only.239 240<div align="center" style="margin-top:40px; padding:20px; background:linear-gradient(135deg, rgba(139,92,246,0.08), rgba(236,72,153,0.08)); border-radius:16px; border:1px solid rgba(139,92,246,0.2);">241<p style="font-size:1em; color:#c084fc; margin:0;">Merged with 💜 by <a href="https://huggingface.co/tacodevs">tacodevs</a></p>242</div>243