tacodevs/Behemoth-X-R1-123B-FP8

1---2license: other3license_name: mistral-research-license4license_link: https://mistral.ai/licenses/MRL-0.1.md5base_model: tacodevs/Behemoth-X-R1-123B6base_model_relation: quantized7tags:8  - mistral9  - fp810  - w8a811  - compressed-tensors12  - quantized13  - thinking14  - roleplay15  - creative-writing16language:17  - en18pipeline_tag: text-generation19---20 21<div align="center">22<img src="https://huggingface.co/tacodevs/Behemoth-X-R1-123B/resolve/main/assets/hero.png" alt="Behemoth-X-R1-123B" style="width:100%; max-width:960px; border-radius:16px; box-shadow:0 0 60px rgba(236,72,153,0.35), 0 0 100px rgba(139,92,246,0.25);"/>23</div>24 25<div align="center" style="margin-top:24px;">26 27<h1 style="font-size:3em; font-weight:900; background:linear-gradient(90deg,#ec4899 0%,#a855f7 50%,#06b6d4 100%); -webkit-background-clip:text; -webkit-text-fill-color:transparent; background-clip:text; margin:0;">Behemoth-X-R1-123B · FP8</h1>28 29<p style="font-size:1.2em; color:#a855f7; font-style:italic;">Single-GPU beast mode.</p>30 31<p>32<img src="https://img.shields.io/badge/quant-FP8_Dynamic-8B5CF6?style=for-the-badge" alt="quant"/>33<img src="https://img.shields.io/badge/VRAM-~130GB-EC4899?style=for-the-badge" alt="vram"/>34<img src="https://img.shields.io/badge/runs_on-1x_H200-06B6D4?style=for-the-badge" alt="gpu"/>35</p>36 37</div>38 39## About40 41FP8 dynamic quantization of [`tacodevs/Behemoth-X-R1-123B`](https://huggingface.co/tacodevs/Behemoth-X-R1-123B). Near-lossless quality, half the weight bytes, fits on a single H200.42 43- **Method:** W8A8 dynamic quantization via [llm-compressor](https://github.com/vllm-project/llm-compressor)44- **Format:** `compressed-tensors`45- **Size:** ~115 GB46- **Calibration:** None needed (dynamic scheme)47 48## Usage with vLLM49 50```bash51python -m vllm.entrypoints.openai.api_server \52  --model tacodevs/Behemoth-X-R1-123B-FP8 \53  --max-model-len 16384 \54  --gpu-memory-utilization 0.95 \55  --trust-remote-code56```57 58Fits on **1× H200 141GB** with ~30k context window.59 60## See the main model card61 62Full documentation, prompt format, prefill examples, credits, and everything else is on the source repo:63 64### 👉 [tacodevs/Behemoth-X-R1-123B](https://huggingface.co/tacodevs/Behemoth-X-R1-123B)65 66## License67 68Inherited from base: **[Mistral Research License](https://mistral.ai/licenses/MRL-0.1.md)** — non-commercial use only.69