inference-optimization/MiniMax-M2.5-BF16

1# MiniMax-M2.5 工具调用指南2 3[英文版](./tool_calling_guide.md) | [中文版](./tool_calling_guide_cn.md)4 5MiniMax-M2.5 支持与 MiniMax-M2 相同的工具调用语法。6 7## 简介8 9MiniMax-M2.5 模型支持工具调用功能，使模型能够识别何时需要调用外部工具，并以结构化格式输出工具调用参数。本文档提供了有关如何使用 MiniMax-M2.5 工具调用功能的详细说明。10 11## 基础示例12 13以下 Python 脚本基于 OpenAI SDK 实现了一个天气查询工具调用示例：14 15```python16from openai import OpenAI17import json18 19client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")20 21def get_weather(location: str, unit: str):22    return f"Getting the weather for {location} in {unit}..."23 24tool_functions = {"get_weather": get_weather}25 26tools = [{27    "type": "function",28    "function": {29        "name": "get_weather",30        "description": "Get the current weather in a given location",31        "parameters": {32            "type": "object",33            "properties": {34                "location": {"type": "string", "description": "City and state, e.g., 'San Francisco, CA'"},35                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}36            },37            "required": ["location", "unit"]38        }39    }40}]41 42response = client.chat.completions.create(43    model=client.models.list().data[0].id,44    messages=[{"role": "user", "content": "What's the weather like in San Francisco? use celsius."}],45    tools=tools,46    tool_choice="auto"47)48 49print(response)50 51tool_call = response.choices[0].message.tool_calls[0].function52print(f"Function called: {tool_call.name}")53print(f"Arguments: {tool_call.arguments}")54print(f"Result: {get_weather(**json.loads(tool_call.arguments))}")55```56 57**输出示例：**58```59Function called: get_weather60Arguments: {"location": "San Francisco, CA", "unit": "celsius"}61Result: Getting the weather for San Francisco, CA in celsius...62```63 64## 手动解析模型输出65 66**我们强烈建议使用 vLLM 或 SGLnag 来解析工具调用。** 如果您无法使用支持 MiniMax-M2.5 的推理引擎（如 vLLM 和 SGLang）的内置解析器，或需要使用其他推理框架（如 transformers、TGI 等），您可以使用以下方法手动解析模型的原始输出。这种方法需要您自己解析模型输出的 XML 标签格式。67 68### 使用 Transformers 的示例69 70这是一个使用 transformers 库的完整示例：71 72```python73from transformers import AutoTokenizer74 75def get_default_tools():76    return [77        {78          "name": "get_current_weather",79          "description": "Get the latest weather for a location",80          "parameters": {81              "type": "object", 82              "properties": {83                  "location": {84                      "type": "string", 85                      "description": "A certain city, such as Beijing, Shanghai"86                  }87              }, 88          }89          "required": ["location"],90          "type": "object"91        }92    ]93 94# Load model and tokenizer95tokenizer = AutoTokenizer.from_pretrained(model_id)96prompt = "What's the weather like in Shanghai today?"97messages = [98    {"role": "system", "content": "You are a helpful assistant."},99    {"role": "user", "content": prompt},100]101 102# Enable function calling tools103tools = get_default_tools()104 105# Apply chat template and include tool definitions106text = tokenizer.apply_chat_template(107    messages,108    tokenize=False,109    add_generation_prompt=True,110    tools=tools111)112 113# Send request (using any inference service)114import requests115payload = {116    "model": "MiniMaxAI/MiniMax-M2.5",117    "prompt": text,118    "max_tokens": 4096119}120response = requests.post(121    "http://localhost:8000/v1/completions",122    headers={"Content-Type": "application/json"},123    json=payload,124    stream=False,125)126 127# Model output needs manual parsing128raw_output = response.json()["choices"][0]["text"]129print("Raw output:", raw_output)130 131# Use the parsing function below to process the output132tool_calls = parse_tool_calls(raw_output, tools)133```134 135## 🛠️ 工具调用定义136 137### 工具结构138 139工具调用需要在请求体中定义 `tools` 字段。每个工具由以下部分组成：140 141```json142{143  "tools": [144    {145      "name": "search_web",146      "description": "Search function.",147      "parameters": {148        "properties": {149          "query_list": {150            "description": "Keywords for search, list should contain 1 element.",151            "items": { "type": "string" },152            "type": "array"153          },154          "query_tag": {155            "description": "Category of query",156            "items": { "type": "string" },157            "type": "array"158          }159        },160        "required": [ "query_list", "query_tag" ],161        "type": "object"162      }163    }164  ]165}166```167 168**字段说明：**169- `name`：函数名称170- `description`：函数描述171- `parameters`：函数参数定义172  - `properties`：参数属性定义，其中键是参数名称，值包含详细的参数描述173  - `required`：必需参数列表174  - `type`：参数类型（通常为 "object"）175 176### 内部处理格式177 178在 MiniMax-M2.5 模型内部处理时，工具定义会被转换为特殊格式并连接到输入文本中。以下是一个完整示例：179 180```181]~!b[]~b]system182You are a helpful assistant.183 184# Tools185You may call one or more tools to assist with the user query.186Here are the tools available in JSONSchema format:187 188<tools>189<tool>{"name": "search_web", "description": "Search function.", "parameters": {"type": "object", "properties": {"query_list": {"type": "array", "items": {"type": "string"}, "description": "Keywords for search, list should contain 1 element."}, "query_tag": {"type": "array", "items": {"type": "string"}, "description": "Category of query"}}, "required": ["query_list", "query_tag"]}}</tool>190</tools>191 192When making tool calls, use XML format to invoke tools and pass parameters:193 194<minimax:tool_call>195<invoke name="tool-name-1">196<parameter name="param-key-1">param-value-1</parameter>197<parameter name="param-key-2">param-value-2</parameter>198...199</invoke>200[e~[201]~b]user202When were the latest announcements from OpenAI and Gemini?[e~[203]~b]ai204<think>205```206 207**格式说明：**208 209- `]~!b[]~b]system`：系统消息开始标记210- `[e~[`：消息结束标记211- `]~b]user`：用户消息开始标记212- `]~b]ai`：助手消息开始标记213- `]~b]tool`：工具结果消息开始标记214- `<tools>...</tools>`：工具定义区域，每个工具都用 `<tool>` 标签包装，内容为 JSON Schema215- `<minimax:tool_call>...</minimax:tool_call>`：工具调用区域216- `<think>...</think>`：生成过程中的思考过程标记217 218### 模型输出格式219 220MiniMax-M2.5 使用结构化的 XML 标签格式：221 222```xml223<minimax:tool_call>224<invoke name="search_web">225<parameter name="query_tag">["technology", "events"]</parameter>226<parameter name="query_list">["\"OpenAI\" \"latest\" \"release\""]</parameter>227</invoke>228<invoke name="search_web">229<parameter name="query_tag">["technology", "events"]</parameter>230<parameter name="query_list">["\"Gemini\" \"latest\" \"release\""]</parameter>231</invoke>232</minimax:tool_call>233```234 235每个工具调用使用 `<invoke name="function_name">` 标签，参数使用 `<parameter name="parameter_name">` 标签包装。236 237## 手动解析工具调用结果238 239### 解析工具调用240 241MiniMax-M2.5 使用结构化的 XML 标签，这需要一种不同的解析方法。核心函数如下：242 243```python244import re245import json246from typing import Any, Optional, List, Dict247 248 249def extract_name(name_str: str) -> str:250    """Extract name from quoted string"""251    name_str = name_str.strip()252    if name_str.startswith('"') and name_str.endswith('"'):253        return name_str[1:-1]254    elif name_str.startswith("'") and name_str.endswith("'"):255        return name_str[1:-1]256    return name_str257 258 259def convert_param_value(value: str, param_type: str) -> Any:260    """Convert parameter value based on parameter type"""261    if value.lower() == "null":262        return None263        264    param_type = param_type.lower()265    266    if param_type in ["string", "str", "text"]:267        return value268    elif param_type in ["integer", "int"]:269        try:270            return int(value)271        except (ValueError, TypeError):272            return value273    elif param_type in ["number", "float"]:274        try:275            val = float(value)276            return val if val != int(val) else int(val)277        except (ValueError, TypeError):278            return value279    elif param_type in ["boolean", "bool"]:280        return value.lower() in ["true", "1"]281    elif param_type in ["object", "array"]:282        try:283            return json.loads(value)284        except json.JSONDecodeError:285            return value286    else:287        # Try JSON parsing, return string if failed288        try:289            return json.loads(value)290        except json.JSONDecodeError:291            return value292 293 294def parse_tool_calls(model_output: str, tools: Optional[List[Dict]] = None) -> List[Dict]:295    """296    Extract all tool calls from model output297    298    Args:299        model_output: Complete output text from the model300        tools: Tool definition list for getting parameter type information, format can be:301               - [{"name": "...", "parameters": {...}}]302               - [{"type": "function", "function": {"name": "...", "parameters": {...}}}]303    304    Returns:305        Parsed tool call list, each element contains name and arguments fields306    307    Example:308        >>> tools = [{309        ...     "name": "get_weather",310        ...     "parameters": {311        ...         "type": "object",312        ...         "properties": {313        ...             "location": {"type": "string"},314        ...             "unit": {"type": "string"}315        ...         }316        ...     }317        ... }]318        >>> output = '''<minimax:tool_call>319        ... <invoke name="get_weather">320        ... <parameter name="location">San Francisco</parameter>321        ... <parameter name="unit">celsius</parameter>322        ... </invoke>323        ... </minimax:tool_call>'''324        >>> result = parse_tool_calls(output, tools)325        >>> print(result)326        [{'name': 'get_weather', 'arguments': {'location': 'San Francisco', 'unit': 'celsius'}}]327    """328    # Quick check if tool call marker is present329    if "<minimax:tool_call>" not in model_output:330        return []331    332    tool_calls = []333    334    try:335        # Match all <minimax:tool_call> blocks336        tool_call_regex = re.compile(r"<minimax:tool_call>(.*?)</minimax:tool_call>", re.DOTALL)337        invoke_regex = re.compile(r"<invoke name=(.*?)</invoke>", re.DOTALL)338        parameter_regex = re.compile(r"<parameter name=(.*?)</parameter>", re.DOTALL)339        340        # Iterate through all tool_call blocks341        for tool_call_match in tool_call_regex.findall(model_output):342            # Iterate through all invokes in this block343            for invoke_match in invoke_regex.findall(tool_call_match):344                # Extract function name345                name_match = re.search(r'^([^>]+)', invoke_match)346                if not name_match:347                    continue348                349                function_name = extract_name(name_match.group(1))350                351                # Get parameter configuration352                param_config = {}353                if tools:354                    for tool in tools:355                        tool_name = tool.get("name") or tool.get("function", {}).get("name")356                        if tool_name == function_name:357                            params = tool.get("parameters") or tool.get("function", {}).get("parameters")358                            if isinstance(params, dict) and "properties" in params:359                                param_config = params["properties"]360                            break361                362                # Extract parameters363                param_dict = {}364                for match in parameter_regex.findall(invoke_match):365                    param_match = re.search(r'^([^>]+)>(.*)', match, re.DOTALL)366                    if param_match:367                        param_name = extract_name(param_match.group(1))368                        param_value = param_match.group(2).strip()369                        370                        # Remove leading and trailing newlines371                        if param_value.startswith('\n'):372                            param_value = param_value[1:]373                        if param_value.endswith('\n'):374                            param_value = param_value[:-1]375                        376                        # Get parameter type and convert377                        param_type = "string"378                        if param_name in param_config:379                            if isinstance(param_config[param_name], dict) and "type" in param_config[param_name]:380                                param_type = param_config[param_name]["type"]381                        382                        param_dict[param_name] = convert_param_value(param_value, param_type)383                384                tool_calls.append({385                    "name": function_name,386                    "arguments": param_dict387                })388    389    except Exception as e:390        print(f"Failed to parse tool calls: {e}")391        return []392    393    return tool_calls394```395 396**使用示例：**397 398```python399# Define tools400tools = [401    {402        "name": "get_weather",403        "parameters": {404            "type": "object",405            "properties": {406                "location": {"type": "string"},407                "unit": {"type": "string"}408            },409            "required": ["location", "unit"]410        }411    }412]413 414# Model output415model_output = """Let me help you query the weather.416<minimax:tool_call>417<invoke name="get_weather">418<parameter name="location">San Francisco</parameter>419<parameter name="unit">celsius</parameter>420</invoke>421</minimax:tool_call>"""422 423# Parse tool calls424tool_calls = parse_tool_calls(model_output, tools)425 426# Output results427for call in tool_calls:428    print(f"Function called: {call['name']}")429    print(f"Arguments: {call['arguments']}")430    # Output: Function called: get_weather431    #         Arguments: {'location': 'San Francisco', 'unit': 'celsius'}432```433 434### 执行工具调用435 436完成解析后，您可以执行相应的工具并构造返回结果：437 438```python439def execute_function_call(function_name: str, arguments: dict):440    """Execute function call and return result"""441    if function_name == "get_weather":442        location = arguments.get("location", "Unknown location")443        unit = arguments.get("unit", "celsius")444        # Build function execution result445        return {446            "role": "tool", 447            "content": [448              {449                "name": function_name,450                "type": "text",451                "text": json.dumps({452                    "location": location, 453                    "temperature": "25", 454                    "unit": unit, 455                    "weather": "Sunny"456                }, ensure_ascii=False)457              }458            ] 459          }460    elif function_name == "search_web":461        query_list = arguments.get("query_list", [])462        query_tag = arguments.get("query_tag", [])463        # Simulate search results464        return {465            "role": "tool",466            "content": [467              {468                "name": function_name,469                "type": "text",470                "text": f"Search keywords: {query_list}, Category: {query_tag}\nSearch results: Relevant information found"471              }472            ]473          }474    475    return None476```477 478### 将工具执行结果返回给模型479 480在成功解析工具调用后，您应该将工具执行结果添加到对话历史中，以便模型在后续交互中可以访问和利用这些信息。请参考 [chat_template.jinja](https://huggingface.co/MiniMaxAI/MiniMax-M2.5/blob/main/chat_template.jinja) 了解连接格式。481 482## 参考文献483 484- [MiniMax-M2.5 模型仓库](https://github.com/MiniMax-AI/MiniMax-M2.5)485- [vLLM 项目主页](https://github.com/vllm-project/vllm)486- [SGLang 项目主页](https://github.com/sgl-project/sglang)487- [OpenAI Python SDK](https://github.com/openai/openai-python)488 489## 获取支持490 491如果遇到任何问题：492 493- 通过邮箱 [model@minimax.io](mailto:model@minimax.io) 等官方渠道联系我们的技术支持团队494 495- 在我们的仓库提交 Issue496 497- 通过我们的 [官方企业微信交流群](https://github.com/MiniMax-AI/MiniMax-AI.github.io/blob/main/images/wechat-qrcode.jpeg) 反馈498 499我们会持续优化模型的使用体验，欢迎反馈！