inference-optimization/MiniMax-M2.5-BF16

1# MiniMax M2.5 Model SGLang Deployment Guide2 3[English Version](./sglang_deploy_guide.md) | [Chinese Version](./sglang_deploy_guide_cn.md)4 5We recommend using [SGLang](https://github.com/sgl-project/sglang) to deploy the [MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5) model. SGLang is a high-performance inference engine with excellent serving throughput, efficient and intelligent memory management, powerful batch request processing capabilities, and deeply optimized underlying performance. We recommend reviewing SGLang's official documentation to check hardware compatibility before deployment.6 7## Applicable Models8 9This document applies to the following models. You only need to change the model name during deployment.10 11- [MiniMaxAI/MiniMax-M2.5](https://huggingface.co/MiniMaxAI/MiniMax-M2.5)12- [MiniMaxAI/MiniMax-M2.1](https://huggingface.co/MiniMaxAI/MiniMax-M2.1)13- [MiniMaxAI/MiniMax-M2](https://huggingface.co/MiniMaxAI/MiniMax-M2)14 15The deployment process is illustrated below using MiniMax-M2.5 as an example.16 17## System Requirements18 19- OS: Linux20 21- Python: 3.9 - 3.1222 23- GPU:24 25  - compute capability 7.0 or higher26 27  - Memory requirements: 220 GB for weights, 240 GB per 1M context tokens28 29The following are recommended configurations; actual requirements should be adjusted based on your use case:30 31- **96G x4** GPU: Supports a total KV Cache capacity of 400K tokens.32 33- **144G x8** GPU: Supports a total KV Cache capacity of up to 3M tokens.34 35> **Note**: The values above represent the total aggregate hardware KV Cache capacity. The maximum context length per individual sequence remains **196K** tokens.36 37## Deployment with Python38 39It is recommended to use a virtual environment (such as **venv**, **conda**, or **uv**) to avoid dependency conflicts. 40 41We recommend installing SGLang in a fresh Python environment:42 43```bash44uv venv45source .venv/bin/activate46uv pip install sglang47```48 49Run the following command to start the SGLang server. SGLang will automatically download and cache the MiniMax-M2.5 model from Hugging Face.50 514-GPU deployment command:52 53```bash54python -m sglang.launch_server \55    --model-path MiniMaxAI/MiniMax-M2.5 \56    --tp-size 4 \57    --tool-call-parser minimax-m2 \58    --reasoning-parser minimax-append-think \59    --host 0.0.0.0 \60    --trust-remote-code \61    --port 8000 \62    --mem-fraction-static 0.8563```64 658-GPU deployment command:66 67```bash68python -m sglang.launch_server \69    --model-path MiniMaxAI/MiniMax-M2.5 \70    --tp-size 8 \71    --ep-size 8 \72    --tool-call-parser minimax-m2 \73    --trust-remote-code \74    --host 0.0.0.0 \75    --reasoning-parser minimax-append-think \76    --port 8000 \77    --mem-fraction-static 0.8578```79 80## Testing Deployment81 82After startup, you can test the SGLang OpenAI-compatible API with the following command:83 84```bash85curl http://localhost:8000/v1/chat/completions \86    -H "Content-Type: application/json" \87    -d '{88        "model": "MiniMaxAI/MiniMax-M2.5",89        "messages": [90            {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant."}]},91            {"role": "user", "content": [{"type": "text", "text": "Who won the world series in 2020?"}]}92        ]93    }'94```95 96## Common Issues97 98### MiniMax-M2 model is not currently supported99 100Please upgrade to the latest stable version, >= v0.5.4.post1.101 102## Getting Support103 104If you encounter any issues while deploying the MiniMax model:105 106- Contact our technical support team through official channels such as email at [model@minimax.io](mailto:model@minimax.io)107 108- Submit an issue on our [GitHub](https://github.com/MiniMax-AI) repository109 110We continuously optimize the deployment experience for our models. Feedback is welcome!111 112