MiniMaxAI/MiniMax-VL-01

1# MiniMax-Text-01 函数调用（Function Call）功能指南2 3## 📖 简介4 5MiniMax-Text-01 模型支持函数调用功能，使模型能够识别何时需要调用外部函数，并以结构化格式输出函数调用参数。本文档详细介绍了如何使用 MiniMax-Text-01 的函数调用功能。6 7## 🛠️ 函数调用的定义8 9### 函数结构体10 11函数调用需要在请求体中定义 `tools` 字段，每个函数由以下部分组成：12 13```json14{15  "tools": [16    {17      "type": "function",18      "function": {19        "name": "function_name",           // 函数名称，必填20        "description": "function_description", // 函数描述，应简明扼要说明函数功能21        "parameters": {                    // 函数参数定义，符合 JSON Schema 格式22          "type": "object",                // 参数整体类型，固定为object23          "properties": {                  // 参数属性对象24            "param_name": {                // 参数名称25              "description": "参数描述",    // 参数说明26              "type": "string|number|boolean|array|object" // 参数类型27            }28          },29          "required": ["param1", "param2"]  // 必填参数列表30        }31      }32    }33  ]34}35```36 37### 示例38 39以下是一个简单的天气查询函数定义示例：40 41```json42"tools": [43  {44    "type": "function",45    "function": {46      "name": "get_current_weather",47      "description": "Get the latest weather for a location",48      "parameters": {49        "type": "object", 50        "properties": {51          "location": {52            "type": "string", 53            "description": "A certain city, such as Beijing, Shanghai"54          }55        }, 56        "required": ["location"]57      }58    }59  }60]61```62 63### 完整请求示例64 65下面是一个包含函数定义的完整Python代码示例：66 67```python68payload = json.dumps({69    "model": "MiniMax-VL-01",70    "messages": [71        {72            "role": "system",73            "content": "MM Intelligent Assistant is a large-scale language model developed by MiniMax and has no interfaces to call other products. MiniMax is a China technology company that has been committed to conducting research related to large models."74        },75        {76            "role": "user",77            "content": "上海今天天气怎么样？"78        }79    ],80    "tools": [81        {82            "type": "function",83            "function": {84                "name": "get_current_weather",85                "description": "Get the latest weather for a location",86                "parameters": {87                  "type": "object", 88                  "properties": {89                    "location": {90                      "type": "string", 91                      "description": "A certain city, such as Beijing, Shanghai"92                    }93                  }, 94                  "required": ["location"]95                }96            }97        }98    ],99    "tool_choice": "auto",100    "stream": True,101    "max_tokens": 10000,102    "temperature": 0.9,103    "top_p": 1104})105```106 107## 🔄 函数调用的输入格式108 109在模型内部处理时，函数定义会被转换为特殊格式并拼接到输入文本中：110 111```112<beginning_of_sentence>system function_setting=functions113{"name": "get_current_weather", "description": "Get the latest weather for a location", "parameters": {"type": "object", "properties": {"location": {"type": "string", "description": "A certain city, such as Beijing, Shanghai"}}, "required": ["location"]}}<end_of_sentence>114```115 116注意事项：1171. 函数定义位于系统设置之后、对话数据之前1182. 使用 `function_setting=functions` 标记函数定义区域1193. 每个函数定义使用JSON字符串表示1204. 区域以 `<end_of_sentence>` 结束121 122## 📤 模型的函数调用输出123 124当模型决定调用函数时，它会在响应中使用特殊格式输出函数调用信息：125 126````127<function_call>```typescript128functions.get_current_weather({"location": "上海"})129```130````131 132"<function_call>" 是 special token, 后面的 "functions.函数名(参数 json 结构体)", 需要字符串匹配出参数, 交外部执行.133 134## 📥 函数执行结果的处理135 136当函数调用成功执行后，模型将返回以下格式的输出：137 138````typescript139```typescript140functions.get_current_weather({"location": "Shanghai"})141```142````143 144您可以使用以下正则表达式方法提取函数名称和参数，便于后续处理：145 146````python147def parse_function_calls(content: str):148    """149    解析模型返回的函数调用内容，提取函数名和参数150    151    参数:152        content: 模型返回的原始内容字符串153        154    返回:155        解析后的函数调用信息字典，包含函数名和参数156    """157    # 匹配 typescript 代码块158    pattern = r"```typescript\n(.+?)?\n```"159    matches = re.finditer(pattern, content, re.DOTALL)160    161    for match in matches:162        function_code = match.group(1)163        # 提取函数名和参数164        function_match = re.search(r'functions\.(\w+)\((.+)\)', function_code)165        166        if not function_match:167            continue168            169        function_name = function_match.group(1)170        arguments_str = function_match.group(2)171        172        try:173            # 解析参数JSON174            arguments = json.loads(arguments_str)175            print(f"调用函数: {function_name}, 参数: {arguments}")176            177            # 示例: 处理天气查询函数178            if function_name == "get_current_weather":179                location = arguments.get("location", "未知位置")180                # 构建函数执行结果181                return {182                    "role": "function", 183                    "name": function_name, 184                    "text": json.dumps({185                        "location": location, 186                        "temperature": "25", 187                        "unit": "celsius", 188                        "weather": "晴朗"189                    }, ensure_ascii=False)190                }191        except json.JSONDecodeError as e:192            print(f"参数解析失败: {arguments_str}, 错误: {e}")193    194    return {}195````196 197成功解析函数调用后，您应将函数执行结果添加到对话历史中，以便模型在后续交互中能够访问和利用这些信息。198 199## 💻 使用 Transformers 库的函数调用示例200 201MiniMax-VL-01 官方仓库提供了使用 Transformers 库进行函数调用的完整示例。您可以在 [MiniMaxAI/MiniMax-VL-01 huggingface 仓库](https://huggingface.co/MiniMaxAI/MiniMax-VL-01/blob/main/main.py) 中查看源代码。202 203以下是使用 Transformers 库实现函数调用的关键部分：204 205```python206def get_default_tools():207    return [208        {209            "type": "function",210            "function": {211                "name": "get_current_weather",212                "description": "Get the latest weather for a location",213                "parameters": {214                    "type": "object", 215                    "properties": {216                        "location": {217                            "type": "string", 218                            "description": "A certain city, such as Beijing, Shanghai"219                        }220                    }, 221                    "required": ["location"]222                }223            }224        }225    ]226 227# 加载模型和分词器228tokenizer = AutoTokenizer.from_pretrained(model_id)229prompt = "What's the weather like in Shanghai today?"230messages = [231    {"role": "system", "content": [{"type": "text", "text": "You are a helpful assistant created by Minimax based on MiniMax-Text-01 model."}]},232    {"role": "user", "content": [{"type": "text", "text": prompt}]},233]234 235# 启用函数调用工具236tools = get_default_tools()237 238# 应用聊天模板，并加入工具定义239text = tokenizer.apply_chat_template(240    messages,241    tokenize=False,242    add_generation_prompt=True,243    tools=tools244)245 246# 生成回复247model_inputs = tokenizer(text, return_tensors="pt").to("cuda")248quantized_model = AutoModelForCausalLM.from_pretrained(249    model_id,250    torch_dtype="bfloat16",251    device_map=device_map,252    quantization_config=quantization_config,253    trust_remote_code=True,254    offload_buffers=True,255)256generation_config = GenerationConfig(257    max_new_tokens=20,258    eos_token_id=200020,259    use_cache=True,260)261 262# 执行生成263generated_ids = quantized_model.generate(**model_inputs, generation_config=generation_config)264response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]265```266 267### 运行方式268 269您可以通过以下命令运行示例代码：270 271```bash272export SAFETENSORS_FAST_GPU=1273python main.py --quant_type int8 --world_size 8 --model_id <model_path> --enable_tools274```275 276参数说明：277- `--quant_type`: 量化类型，可选 "default" 或 "int8"278- `--world_size`: GPU 数量，int8 量化至少需要 8 个 GPU279- `--model_id`: 模型路径280- `--enable_tools`: 启用函数调用功能281 282### 结果处理283符合预期的情况下，你将得到以下输出284 285````base286```typescript287functions.get_current_weather({"location": "Shanghai"})288```289````290 291你可以使用正则表达式提取出需要调用的 function 和 对应的参数292 293````python294def try_parse_tool_calls(content: str):295    pattern = r"```typescript\n(.+?)?\n```"296    matches = re.finditer(pattern, content, re.DOTALL)297    298    for match in matches:299        function_code = match.group(1)300        function_match = re.search(r'functions\.(\w+)\((.+)\)', function_code)301        302        if not function_match:303            continue304            305        function_name = function_match.group(1)306        arguments_str = function_match.group(2)307        308        try:309            arguments = json.loads(arguments_str)310            print(f"tool_calls: [{{'type': 'function', 'function': {{'name': '{function_name}', 'arguments': {arguments}}}}}]")311            312            if function_name == "get_current_weather":313                location = arguments.get("location", "Unknown")314                return {"role": "function", "name": function_name, "text": f'{{"location": "{location}", "temperature": "25", "unit": "celsius", "weather": "Sun"}}'}315        except json.JSONDecodeError as e:316            print(f"Failed parse tools: {arguments_str}, Error: {e}")317    318    return {}319````320 321### 聊天模板322 323MiniMax-VL-01 使用特定的聊天模板格式处理函数调用。聊天模板定义在 `tokenizer_config.json` 中：324 325```json326"{% for message in messages %}{% if message['role'] == 'system' %}{{ '<beginning_of_sentence>system ai_setting=assistant\n' }}{% for item in message['content'] %}{% if item.type == 'image' %}<image>{% elif item.type == 'text' %}{{ item.text }}{% endif %}{% endfor %}{{ '<end_of_sentence>\n' }}{% endif %}{% if message['role'] == 'assistant' %}{{ '<beginning_of_sentence>ai name=assistant\n' }}{% for item in message['content'] %}{% if item.type == 'image' %}<image>{% elif item.type == 'text' %}{{ item.text }}{% endif %}{% endfor %}{{ '<end_of_sentence>\n' }}{% endif %}{% if message['role'] == 'user' %}{{ '<beginning_of_sentence>user name=user\n' }}{% for item in message['content'] %}{% if item.type == 'image' %}<image>{% elif item.type == 'text' %}{{ item.text }}{% endif %}{% endfor %}{{ '<end_of_sentence>\n' }}{% endif %}{% if message['role'] == 'function' %}{{ '<beginning_of_sentence>system function_response=functions\n' + '{\"name\": \"' + message['name'] + '\", \"response\": ' + message['content'][0]['text'] + '}' + '<end_of_sentence>\n'}}{% endif %}{% endfor %}{% if tools %}{% for function in tools %}{{ '<beginning_of_sentence>system function_setting=functions\n' + function | tojson + '<end_of_sentence>\n'}}{% endfor %}{% endif %}{% if add_generation_prompt %}{{ '<beginning_of_sentence>ai name=assistant\n' }}{% generation %}{% endgeneration %}{% endif %}"327 328```329 330## 📝 注意事项331 3321. 函数名称应当遵循编程语言的命名规范，避免使用特殊字符3332. 参数描述应当简洁明了，帮助模型理解参数的用途和约束3343. 模型并不保证每次都会调用函数，这取决于用户的输入和模型的判断3354. 函数调用结果应当以结构化方式返回，便于模型理解和处理