Modelpublic

amd/MiniMax-M2.1-MXFP4

sourceHugging Faceotherupdated 4mo agoView on Hugging Face

1likes1.7kdownloads

Model Card

Model Overview

—Model Architecture: MiniMaxM2ForCausalLM
—Input: Text
—Output: Text
—Supported Hardware Microarchitecture: AMD MI300 MI350/MI355
—ROCm: 7.0
—PyTorch: 2.8.0
—Transformers: 4.57.1
—Operating System(s): Linux
—Inference Engine: SGLang/vLLM
—Model Optimizer: AMD-Quark (v0.11)
—Weight quantization: OCP MXFP4, Static
—Activation quantization: OCP MXFP4, Dynamic

Model Quantization

The model was quantized from QuixiAI/MiniMax-M2.1-bf16 using AMD-Quark. The weights are quantized to MXFP4 and activations are quantized to MXFP4.

Quantization scripts:

cd Quark/examples/torch/language_modeling/llm_ptq/
export exclude_layers="lm_head *block_sparse_moe.gate* *self_attn*"
python3 quantize_quark.py --model_dir $MODEL_DIR \
                          --quant_scheme mxfp4 \
                          --num_calib_data 128 \
                          --exclude_layers $exclude_layers \
                          --skip_evaluation \
                          --multi_gpu  \
                          --trust_remote_code \
                          --model_export hf_format \
                          --output_dir $output_dir

For further details or issues, please refer to the AMD-Quark documentation or contact the respective developers.

Evaluation

The model was evaluated on gsm8k benchmarks using the vllm framework.

Accuracy

<table> <tr> <td><strong>Benchmark</strong> </td> <td><strong>QuixiAI/MiniMax-M2.1-bf16 </strong> </td> <td><strong>amd/MiniMax-M2.1-MXFP4(this model)</strong> </td> <td><strong>Recovery</strong> </td> </tr> <tr> <td>gsm8k (flexible-extract) </td> <td>0.9356 </td> <td>0.9348 </td> <td>99.91% </td> </tr> </table>

Reproduction

The GSM8K results were obtained using the vLLM framework, based on the Docker image rocm/vllm-dev:nightly_main_20260211, and vLLM is installed inside the container.

Preparation in container

To download the evaluation script, reinstallation is not required.

# Install vLLM code repo
git clone https://github.com/vllm-project/vllm.git
cd vllm
git checkout v0.13.0
cd ..

Launching server

VLLM_ROCM_USE_AITER=1 \
VLLM_DISABLE_COMPILE_CACHE=1 \
vllm serve "$MODEL" \
    --tensor-parallel-size 4 \
    --trust-remote-code \
    --max-model-len 32768 \
    --port 8899

Evaluating model in a new terminal

python vllm/tests/evals/gsm8k/gsm8k_eval.py --host http://127.0.0.1 --port 8899 --num-questions 1000 --save-results logs