TheHouseOfTheDude/Behemoth-R1-V2_ModelOpt-NVFP4

Behemoth-R1-V2 ModelOpt NVFP4

NVFP4 quantized version of TheDrummer/Behemoth-R1-123B-v2 using NVIDIA Model Optimizer.

Quantization Details

Hardware Requirements

• —Optimal: NVIDIA Blackwell GPUs (B100, B200, RTX PRO 6000 Blackwell)
• —Compatible: Hopper/Ampere (will use weight-only mode)

Usage with vLLM

from vllm import LLM, SamplingParams

llm = LLM(
    model="TheHouseOfTheDude/Behemoth-R1-V2_ModelOpt-NVFP4",
    quantization="modelopt",
    trust_remote_code=True,
)

sampling_params = SamplingParams(temperature=0.8, top_p=0.95, max_tokens=512)
outputs = llm.generate(["Write a story about..."], sampling_params)
print(outputs[0].outputs[0].text)

Chat Template

Uses Mistral v7 (Non-Tekken) format. See the original model card for usage details.

Credits

• —Original Model: TheDrummer
• —Quantization: TheHouseOfTheDude
• —Quantization Framework: NVIDIA ModelOpt