Felipe97/llama-cpp-compiled

1# Auto-Parser Architecture2 3The auto-parser automatically analyzes chat templates to determine how to parse model outputs, including content, reasoning, and tool calls.4 5## Overview6 7The unified auto-parser uses a pure differential, compositional approach (inspired by the `git diff` algorithm) to analyze chat templates:8 9**Core Philosophy**:10 11- **Minimize Hardcoded Patterns**: All markers extracted through template comparison (the only heuristic is JSON detection to distinguish `JSON_NATIVE` from tag-based formats)12- **Compositional Architecture**: Separate analyzer structs for reasoning, content, and tools — each responsible for its own analysis and parser construction13 14**Analysis + Parser Building in Two Steps**:15 161. `autoparser::autoparser tmpl_analysis(tmpl)` — runs all differential comparisons and populates the analysis structs172. `autoparser::peg_generator::generate_parser(tmpl, generation_params, tmpl_analysis)` — uses the analysis to build a PEG parser and optional GBNF grammar18 19## Data Structures20 21All structs are defined in [common/chat-auto-parser.h](common/chat-auto-parser.h).22 23### Top-Level: `autoparser` (main analyzer and generator)24 25[common/chat-auto-parser.h:367-388](common/chat-auto-parser.h#L367-L388) — top-level analysis result aggregating `jinja_caps`, `reasoning`, `content`, and `tools` sub-analyses, plus `preserved_tokens` (union of all non-empty markers).26 27### `analyze_reasoning`28 29[common/chat-auto-parser.h:254-274](common/chat-auto-parser.h#L254-L274) — reasoning analysis result: `mode` enum, `start` marker (e.g. `<think>`), and `end` marker (e.g. `</think>`).30 31### `analyze_content`32 33[common/chat-auto-parser.h:280-295](common/chat-auto-parser.h#L280-L295) — content analysis result: `mode` enum, `start`/`end` markers, and `requires_nonnull_content` flag.34 35### `analyze_tools` and its sub-structs36 37- [common/chat-auto-parser.h:176-194](common/chat-auto-parser.h#L176-L194) — `tool_format_analysis`: `mode` enum, `section_start/end`, `per_call_start/end`, JSON field names (`function_field`, `name_field`, `args_field`, `id_field`, `gen_id_field`), and format flags (`fun_name_is_key`, `tools_array_wrapped`)38- [common/chat-auto-parser.h:196-200](common/chat-auto-parser.h#L196-L200) — `tool_function_analysis`: `name_prefix`, `name_suffix`, `close` markers around function names39- [common/chat-auto-parser.h:202-210](common/chat-auto-parser.h#L202-L210) — `tool_arguments_analysis`: `start/end` container markers, `name_prefix/suffix`, `value_prefix/suffix`, `separator`40- [common/chat-auto-parser.h:212-217](common/chat-auto-parser.h#L212-L217) — `tool_id_analysis`: `pos` enum, `prefix`/`suffix` markers around call ID values41- [common/chat-auto-parser.h:301-361](common/chat-auto-parser.h#L301-L361) — `analyze_tools`: aggregates the four sub-structs above42 43### Enums44 45**`reasoning_mode`**: How the template handles reasoning/thinking blocks.46 47| Value           | Description                                                                       |48|-----------------|-----------------------------------------------------------------------------------|49| `NONE`          | No reasoning markers detected                                                     |50| `TAG_BASED`     | Tag-based: `<think>...</think>` (start can be empty for delimiter-style formats)  |51| `TOOLS_ONLY`    | Reasoning only appears in tool call responses, not plain content                  |52 53**Generation Prompt & Reasoning Prefill**: Computed in `common_chat_templates_apply_jinja` before invoking either the specialized handlers or the auto-parser, by rendering the template twice — once with `add_generation_prompt=false` and once with `add_generation_prompt=true` — and storing the diff suffix as `generation_params::generation_prompt`. This string is propagated into `common_chat_params::generation_prompt` and `common_chat_parser_params::generation_prompt`.54 55The generation prompt is prepended to model output before PEG parsing via `wrap_for_generation_prompt()`. The portion *before* the reasoning start marker (if any) is prepended as a literal to ensure any boilerplate added by the template is consumed. The full string is also fed to the grammar sampler via `llama_sampler_accept` (stored in `common_params_sampling::grammar_prefill`), advancing the grammar past tokens already in the prompt. It is used to determine the reasoning budget sampler's initial state — COUNTING if the prefill tokens begin with the reasoning start sequence (but don't also contain the end sequence), IDLE otherwise.56 57**`grammar_prefill`** (`common_params_sampling`): The generation prompt string tokenized and accepted by the grammar sampler at init time. Only applied when `grammar_external` is false (i.e., the grammar was not set explicitly by the user).58 59Three outcomes for reasoning-prefill handling (in `generate_parser()`):60 611. **Start+end in generation prompt** (e.g. `<think></think>\n`): the parser sees reasoning as opened and immediately closed; whitespace-only reasoning content is discarded.622. **Only start in generation prompt** (e.g. `<think>\n`): the parser sees reasoning as already open.633. **Start marker present but not at the end** (e.g. Apriel's `<|begin_assistant|>` followed by boilerplate): the marker is a template artifact; the start literal is cleared so reasoning uses delimiter-style (end-only). For templates that ignore `add_generation_prompt` (empty diff), the rendered `data.prompt` is used as fallback — but only for non-TOOLS_ONLY modes, since in TOOLS_ONLY the start tag is model-generated and may appear in prior conversation turns.64 65**`content_mode`**: How the template wraps assistant content.66 67| Value                    | Description                                                    |68|--------------------------|----------------------------------------------------------------|69| `PLAIN`                  | No content markers                                             |70| `ALWAYS_WRAPPED`         | Content always wrapped: `<response>...</response>`             |71| `WRAPPED_WITH_REASONING` | Content wrapped only when reasoning is present                 |72 73**`tool_format`**: Classification of tool call structure.74 75| Value            | Description                                                      |76|------------------|------------------------------------------------------------------|77| `NONE`           | No tool support detected                                         |78| `JSON_NATIVE`    | Pure JSON: `{"name": "X", "arguments": {...}}`                   |79| `TAG_WITH_JSON`  | Tag-based with JSON args: `<function=X>{...}</function>`         |80| `TAG_WITH_TAGGED`| Tag-based with tagged args: `<param=key>value</param>`           |81 82**`call_id_position`**: Where call IDs appear in tag-based formats.83 84| Value                    | Description                                  |85|--------------------------|----------------------------------------------|86| `NONE`                   | No call ID support detected                  |87| `PRE_FUNC_NAME`          | Before function name                         |88| `BETWEEN_FUNC_AND_ARGS`  | Between function name and arguments          |89| `POST_ARGS`              | After arguments                              |90 91## Tool Calling Formats92 93### JSON_NATIVE94 95**Structure**: The entire tool call (function name, arguments, values) is in JSON format. Optional enclosing tags around the section.96 97**Detection**: Function name appears inside a JSON structure (quotes preceded by `{` or `:`).98 99**Examples**:100 101Standard OpenAI-style:102 103```json104<tool_call>105{"name": "get_weather", "arguments": {"location": "Paris", "unit": "celsius"}}106</tool_call>107```108 109Mistral Nemo with array wrapper:110 111```json112[TOOL_CALLS]113[{"name": "calculate", "arguments": {"expr": "2+2"}}]114```115 116Function name as JSON key (Apertus style):117 118```json119{"get_weather": {"location": "Paris"}}120```121 122---123 124### TAG_WITH_JSON125 126**Structure**: Function name is outside JSON, in tag attributes or XML-style tags. Arguments are a JSON object.127 128**Detection**: Function name not in JSON, but argument names appear in JSON context.129 130**Examples**:131 132Functionary v3.1:133 134```xml135<function=get_weather>{"location": "Paris", "unit": "celsius"}</function>136```137 138MiniMax:139 140```xml141<minimax:tool_call>142<tool_name>calculate</tool_name>143<arguments>{"expr": "2+2"}</arguments>144</minimax:tool_call>145```146 147---148 149### TAG_WITH_TAGGED150 151**Structure**: Both function name and argument names are in XML-style tags. String values are unquoted; non-string values are JSON-formatted.152 153**Detection**: Neither function name nor argument names appear in a JSON context.154 155**Examples**:156 157Qwen/Hermes XML format:158 159```xml160<function=get_weather>161<param=location>Paris</param>162<param=unit>celsius</param>163</function>164```165 166Mixed types:167 168```xml169<function=calculate>170<param=expr>2+2</param>171<param=precision>2</param>172<param=options>{"round": true}</param>173</function>174```175 176String values (`Paris`, `celsius`, `2+2`) are unquoted; `options` (object type) is JSON-formatted.177 178---179 180## Analysis Flow181 182```text183autoparser::autoparser(tmpl)184    |185    |-- Phase 1: analyze_reasoning(tmpl, jinja_caps.supports_tool_calls)186    |     |-- R1: compare_reasoning_presence()   — with/without reasoning_content field187    |     |-- R2: compare_thinking_enabled()     — enable_thinking=false vs true188    |     '-- R3: compare_reasoning_scope()      — reasoning+content vs reasoning+tools189    |           (only if supports_tool_calls)190    |191    |-- Phase 2: analyze_content(tmpl, reasoning)192    |     '-- C1: compares content-only vs tools output and content-only vs reasoning output193    |194    |-- Phase 3: analyze_tools(tmpl, jinja_caps, reasoning)195    |     (skipped entirely if !jinja_caps.supports_tool_calls)196    |     |197    |     |-- T1: analyze_tool_calls()           — no tools vs with tools; classifies format198    |     |         |-- JSON path → analyze_tool_call_format_json_native()199    |     |         '-- tag path → analyze_tool_call_format_non_json()200    |     |201    |     (if format != NONE and format != JSON_NATIVE:)202    |     |203    |     |-- T2: check_per_call_markers()       — 1 call vs 2 calls; moves section→per-call if needed204    |     |         (only if supports_parallel_tool_calls)205    |     |206    |     |-- T3: extract_function_markers()     — func_alpha vs func_beta; extracts name prefix/suffix/close207    |     |208    |     |-- T4: analyze_arguments()            — (TAG_WITH_TAGGED only)209    |     |         |-- A1: extract_argument_name_markers()   — arg_name_A vs arg_name_B210    |     |         '-- A2: extract_argument_value_markers()  — value "XXXX" vs "YYYY"211    |     |212    |     |-- T5: extract_argument_separator()   — 1 arg vs 2 args; finds separator between args213    |     |214    |     |-- T6: extract_args_markers()         — 0 args vs 1 arg; finds args container markers215    |     |216    |     '-- T7: extract_call_id_markers()      — call_id "call00001" vs "call99999"217    |218    '-- collect_preserved_tokens()               — union of all non-empty markers219    |220    '-- apply workarounds()                      — post-hoc patches for edge-case templates221    |222    v223autoparser (analysis result)224    |225    v226autoparser::peg_generator::generate_parser(tmpl, inputs, analysis)227    |-- analysis.build_parser(inputs)            — builds PEG parser arena228    |     |-- reasoning.build_parser(ctx)        — reasoning parser (mode-dependent)229    |     |-- content.build_parser(ctx)          — content parser (mode-dependent)230    |     '-- tools.build_parser(ctx)            — tool parser (dispatches by tool_format)231    |           |-- build_tool_parser_json_native()232    |           |-- build_tool_parser_tag_json()233    |           '-- build_tool_parser_tag_tagged()234    |235    |-- Build GBNF grammar (if tools present and trigger_marker non-empty)236    '-- Set grammar_triggers from section_start or per_call_start237    |238    v239common_chat_params (prompt, parser, grammar, triggers, preserved_tokens)240```241 242## Entry Point243 244The auto-parser is invoked in [common/chat.cpp:1280-1310](common/chat.cpp#L1280-L1310) in `common_chat_templates_apply_jinja`. A few specialized templates are handled first (Ministral/Magistral Large 3, GPT-OSS with `<|channel|>`, Functionary v3.2 with `>>>all`), then the auto-parser handles everything else via `autoparser::autoparser` + `peg_generator::generate_parser`.245 246## Algorithm Details247 248### Core Mechanism: Differential Comparison249 250All analysis phases use the same factorized comparison function declared in [common/chat-auto-parser-helpers.h:68](common/chat-auto-parser-helpers.h#L68):251 252```cpp253compare_variants(tmpl, params_A, params_modifier)254```255 256This creates variant B by applying a modifier lambda to a copy of `params_A`, renders both through the template, and computes a `diff_split` ([common/chat-auto-parser.h:28-37](common/chat-auto-parser.h#L28-L37)):257 258- `prefix` — common prefix between A and B259- `suffix` — common suffix between A and B260- `left` — unique to variant A261- `right` — unique to variant B262 263The diff is computed via `calculate_diff_split()`, which finds the longest-common-prefix and longest-common-suffix, then iteratively moves incomplete `<...>` or `[...]` markers from the prefix/suffix into left/right until stable (tag boundary fixing).264 265Text is segmentized into markers and non-marker fragments using `segmentize_markers()`, which splits on `<...>` and `[...]` boundaries.266 267### Phase 1: Reasoning Analysis268 269**R1 — `compare_reasoning_presence()`**: Compares assistant message with vs without a `reasoning_content` field.270 271- Searches `diff.right` (output with reasoning) for the reasoning content needle272- Uses PEG parsers to find surrounding markers:273  - If both pre/post markers found in `diff.right` → `TAG_BASED`274  - If both found but post marker only in the full output B → `TAG_BASED` (template forces markers; handled via prefill)275  - If only post marker found → `TAG_BASED` (delimiter-style, empty start)276- Sets `reasoning.start` and `reasoning.end`277 278**R2 — `compare_thinking_enabled()`**: Compares `enable_thinking=false` vs `true` with a generation prompt.279 280- Detects template-added reasoning markers: `enable_thinking=true` appends a non-empty marker → sets `reasoning.start`, mode = `TAG_BASED`281- Handles the reverse case (`enable_thinking=false` appends the marker instead): extracts both start (from the preceding segment) and end markers; mode = `TAG_BASED`282- The reasoning prefill (markers added by the template) is later extracted in `common_chat_templates_apply_jinja` and prepended to model output before parsing283 284**R3 — `compare_reasoning_scope()`**: Compares assistant message with reasoning+text-content vs reasoning+tool-calls.285 286- Only runs if `jinja_caps.supports_tool_calls`287- Detects `TOOLS_ONLY`: reasoning content present in B (with tools) but not in A (with text content)288- Extracts reasoning markers from the tool call output using PEG parsers289 290### Phase 2: Content Analysis291 292**C1**: Two comparisons in the `analyze_content` constructor:293 294- Comparison 1: content-only output vs tool-call output → `diff_tools`295- Comparison 2: content-only output vs reasoning+empty-content output → `diff_reasoning`296 297Classification logic:298 299- `PLAIN`: `diff_tools.left` equals the response string (content is the entire diff, no wrapper)300- `ALWAYS_WRAPPED`: markers found surrounding the content text in `pure_content` → extracts `start`/`end`301 302### Phase 3: Tool Call Analysis303 304**T1 — `analyze_tool_calls()`**: Compares no-tools vs with-tools output.305 306- Extracts the tool call section as `diff.right`307- Calls `analyze_tool_call_format()` which first strips reasoning markers from the haystack, then:308  - Calls `in_json_haystack()` for both function name and argument name needles309  - `in_json_haystack()` uses a PEG parser to check whether the needle appears in a JSON context (preceded by `{` or `:` with surrounding quotes)310  - If function name is in JSON → `JSON_NATIVE` → `analyze_tool_call_format_json_native()`311  - If function name not in JSON, arg name is in JSON → `TAG_WITH_JSON`312  - If neither in JSON → `TAG_WITH_TAGGED`313  - `analyze_tool_call_format_json_native()`: parses the JSON object, matches field values to needles to populate `name_field`, `args_field`, `id_field`, `gen_id_field`; detects `tools_array_wrapped`; extracts `section_start`/`section_end`314  - `analyze_tool_call_format_non_json()`: uses PEG parsers on the haystack to find up to two opening markers (section + per-call) then up to two closing markers315 316**T2 — `check_per_call_markers()`**: Compares 1 call vs 2 calls.317 318- Computes a secondary diff of the second call portion vs the common suffix319- If the second call content starts with `section_start` → the section marker is actually per-call → moves `section_start/end` to `per_call_start/end` and clears the section markers320 321**T3 — `extract_function_markers()`**: Compares function name `FUN_FIRST` vs `FUN_SECOND` (two different named functions).322 323- Finds where the function name appears in `diff.left`324- Extracts `function.name_prefix` from the common prefix up to the function marker, and `function.name_suffix` from after the name up to the next marker325- Extends `name_suffix` into `diff.suffix` (to the first marker for TAG_WITH_TAGGED; to the first `{` or `[` for TAG_WITH_JSON)326- Extracts `function.close` from after the last argument value up to the per-call/section end marker327 328**T4 — `analyze_arguments()`** (TAG_WITH_TAGGED only):329 330- **A1 `extract_argument_name_markers()`**: Compares `arg_name_A` vs `arg_name_B` (two different argument names).331  - Finds shared surrounding structure → `arguments.name_prefix`, `arguments.name_suffix`332- **A2 `extract_argument_value_markers()`**: Compares argument value `"XXXX"` vs `"YYYY"` (same arg, different value).333  - Finds markers surrounding the value → `arguments.value_prefix`, `arguments.value_suffix`334 335**T5 — `extract_argument_separator()`**: Compares 1 argument vs 2 arguments (same function).336 337- Uses `until_common_prefix(diff.right, ARG_FIRST, ARG_SECOND)` to find what separates the two argument blocks338 339**T6 — `extract_args_markers()`**: Compares 0 arguments vs 1 argument.340 341- Uses `until_common_prefix()` and `after_common_suffix()` with the empty and single-arg JSON strings as anchors to find container markers (`arguments.start`, `arguments.end`)342 343**T7 — `extract_call_id_markers()`**: Compares call IDs `"call00001"` vs `"call99999"`.344 345- Determines whether function name appears in `diff.prefix` or `diff.suffix` to classify position:346  - Function name in prefix only → `BETWEEN_FUNC_AND_ARGS` or `POST_ARGS` (further distinguished by where `{` appears)347  - Function name in suffix only → `PRE_FUNC_NAME`348- Extracts `call_id.prefix` and `call_id.suffix` markers around the call ID value349- Clears `per_call_end` if it incorrectly incorporated the call ID suffix350 351### Workarounds352 353A workaround array in `common/chat-diff-analyzer.cpp` applies post-hoc patches after analysis. Each workaround is a lambda that inspects the template source and overrides analysis results. Current workarounds:354 3551. **Old Qwen/DeepSeek thinking templates** — source contains `content.split('</think>')` but not `<SPECIAL_12>`: sets `reasoning.mode = TAG_BASED` with `<think>`/`</think>` markers if no reasoning was detected3562. **Granite 3.3** — source contains specific "Write your thoughts" text: forces `TAG_BASED` reasoning with `<think>`/`</think>` and `WRAPPED_WITH_REASONING` content with `<response>`/`</response>`3573. **Cohere Command R+** — source contains `<|CHATBOT_TOKEN|>`: sets `ALWAYS_WRAPPED` content mode if no content start is already set3584. **Functionary 3.1** — source contains `set has_code_interpreter`: forces `PLAIN` content, specific `per_call_start/end`, clears preserved tokens to only keep Functionary-specific markers3595. **DeepSeek-R1-Distill-Qwen** — source contains `tool▁calls▁begin` markers: overrides tool section/per-call markers with the correct Unicode block characters360 361### Parser Building362 363Each analyzer struct (`analyze_reasoning`, `analyze_content`, `analyze_tools`) implements `build_parser(parser_build_context&)`. They share a `parser_build_context` that carries the PEG builder, inference inputs, the pre-built reasoning parser, and a pointer to the content analyzer.364 365#### Reasoning Parser (`analyze_reasoning::build_parser`)366 367| Mode                                          | Parser                                                                    |368|-----------------------------------------------|---------------------------------------------------------------------------|369| Not extracting reasoning                      | `eps()`                                                                   |370| `TAG_BASED` or `TOOLS_ONLY` (non-empty start) | `optional(start + reasoning(until(end)) + end + space())`                 |371| `TAG_BASED` or `TOOLS_ONLY` (empty start)     | `optional(reasoning(until(end)) + end + space())` — delimiter-style       |372 373Note: The start marker may be empty either because the analyzer detected delimiter-style reasoning, or because `generate_parser()` cleared a template artifact start marker (see Generation Prompt & Reasoning Prefill above). Whitespace-only reasoning content (e.g. from a `<think></think>` prefill) is discarded by the mapper.374 375#### Content Parser (`analyze_content::build_parser`)376 377| Condition                              | Parser                                                                          |378|----------------------------------------|---------------------------------------------------------------------------------|379| `json_schema` present                  | `reasoning + space() + content(schema(json(), "response-format", ...)) + end()` |380| Tools present                          | Dispatches to `analyze_tools::build_parser()`                                   |381| `ALWAYS_WRAPPED` with reasoning        | `reasoning + start + content(until(end)) + end + end()`                         |382| `ALWAYS_WRAPPED` without reasoning     | `content(until(start)) + start + content(until(end)) + end + end()`             |383| Default (PLAIN)                        | `reasoning + content(rest()) + end()`                                           |384 385#### Tool Parsers (`analyze_tools::build_parser`)386 387Dispatches by `format.mode`:388 389**`build_tool_parser_json_native()`**: Calls `p.standard_json_tools()` which internally dispatches to:390 391- `build_json_tools_function_is_key()` — function name is the JSON key: `{"get_weather": {...}}`392- `build_json_tools_nested_keys()` — nested: `{"function": {"name": "X", "arguments": {...}}}`393- `build_json_tools_flat_keys()` — flat: `{"name": "X", "arguments": {...}}`394 395Handles content wrappers, array wrapping (`tools_array_wrapped`), parallel calls, and `parameter_order`.396 397**`build_tool_parser_tag_json()`**: For each tool function:398 399```text400tool_open(name_prefix + tool_name(literal(name)) + name_suffix) +401    call_id_section +402    tool_args(schema(json(), tool_schema))403  [+ function.close if non-empty]404```405 406Wrapped in per-call markers (with optional parallel call repetition) then optionally in section markers.407 408**`build_tool_parser_tag_tagged()`**: For each tool function, builds one parser per argument:409 410- String types: `tool_arg_string_value(schema(until(value_suffix), ...))`411- JSON types: `tool_arg_json_value(schema(json(), ...))`412- Required args are plain; optional args wrapped in `optional()`413- Arguments joined with `space()` between consecutive parsers414 415For closing: uses `function.close` if present; otherwise uses `peek(per_call_end)` to avoid premature close during partial streaming; falls back to `tool_close(space())` to trigger mapper callbacks.416 417All three tool parsers return:418 419```text420reasoning + optional(content(until(trigger_marker))) + tool_calls + end()421```422 423Each returned parser is wrapped by `wrap_for_generation_prompt()`, which prepends a literal for any boilerplate prefix of the generation prompt (the portion before the reasoning start marker).424 425## Mapper426 427`common_chat_peg_mapper` maps PEG parse results (AST nodes) into `common_chat_msg` structures. Key design:428 429- **Buffered arguments**: Before `tool_name` is known, argument text goes to `args_buffer`; once the name is set, the buffer is flushed to `current_tool->arguments`430- **`args_target()`**: Returns a reference to whichever destination is currently active (buffer or tool args), eliminating branching431- **`closing_quote_pending`**: Tracks whether a closing `"` needs to be appended when a string argument value is finalized (for schema-declared string types in tagged format)432- **Whitespace-only reasoning**: Reasoning content that consists entirely of whitespace (e.g. from a `<think></think>` prefill) is cleared so the message shows no reasoning433- **Brace auto-closing**: At tool close, unclosed `{` braces are closed automatically434 435## Files436 437| File                                      | Purpose                                                                         |438|-------------------------------------------|---------------------------------------------------------------------------------|439| `common/chat-auto-parser.h`               | All analysis structs, enums, `autoparser`, `peg_generator`, `generation_params` |440| `common/chat-auto-parser-generator.cpp`   | Parser generator: `generate_parser()` and `build_parser()` methods              |441| `common/chat-diff-analyzer.cpp`           | Differential analysis implementation and workarounds                            |442| `common/chat-auto-parser-helpers.h/cpp`   | `calculate_diff_split()`, `segmentize_markers()`, `compare_variants()`,         |443|                                           | `wrap_for_generation_prompt()`, string helpers                                  |444| `common/chat-peg-parser.h/cpp`            | `common_chat_peg_builder`, `common_chat_peg_mapper`, and helpers                |445| `common/chat.cpp`                         | Entry point: `common_chat_templates_apply_jinja()`                              |446| `tests/test-chat-auto-parser.cpp`         | Auto-parser unit tests; also a debug tool when given a template path            |447| `tests/test-chat-analysis.cpp`            | Template differential analysis debug tool                                       |448 449## Testing & Debugging450 451### Debug Tools452 453**Template Debugger**: `tests/test-chat-auto-parser.cpp`454 455- Usage: `./bin/test-chat-auto-parser path/to/template.jinja` (without a path, it runs the automated tests)456- Shows detected format, markers, generated parser, and GBNF grammar457 458**Template Analysis**: `tests/test-chat-analysis.cpp`459 460- Usage: `./bin/test-chat-analysis --template-file path/to/template.jinja` (without arguments, it runs on all templates from the test suite)461 462**Debug Logging**: Enable with `LLAMA_ARG_LOG_VERBOSITY=2`463 464- Shows detailed analysis steps, pattern extraction results, and generated parser structure465 466**PEG Test Builder**: Fluent API for creating test cases — see [tests/test-chat.cpp:947-1043](tests/test-chat.cpp#L947-L1043). Example usage:467 468```cpp469auto tst = peg_tester("models/templates/Template.jinja");470tst.test("input text")471   .reasoning_format(COMMON_REASONING_FORMAT_AUTO)472   .tools({tool_json})473   .parallel_tool_calls(true)474   .enable_thinking(true)475   .expect(expected_message)476   .run();477```478 479### Tested Templates480 481The following templates have active tests in `tests/test-chat.cpp`:482 483| Template | Format | Notes |484| -------- | ------ | ----- |485| Ministral-3-14B-Reasoning | Reasoning | `[THINK]...[/THINK]` tags (specialized handler) |486| NVIDIA-Nemotron-3-Nano-30B | TAG_WITH_TAGGED | Reasoning + tools |487| CohereForAI Command-R7B | JSON_NATIVE | `<\|START_THINKING\|>`/`<\|START_RESPONSE\|>` markers |488| Google Gemma 2 2B | Content only | No tool support |489| Qwen-QwQ-32B | Reasoning | Forced-open thinking |490| NousResearch Hermes 2 Pro | JSON_NATIVE | `<tool_call>` wrapper |491| IBM Granite 3.3 | JSON_NATIVE | `<think></think>` + `<response></response>` |492| IBM Granite 4.0 | JSON_NATIVE | `<tool_call>` wrapper (same template used by 4.1) |493| ByteDance Seed-OSS | TAG_WITH_TAGGED | Custom `<seed:think>` and `<seed:tool_call>` tags |494| Qwen3-Coder | TAG_WITH_TAGGED | XML-style tool format |495| DeepSeek V3.1 | JSON_NATIVE | Forced thinking mode |496| GLM-4.6 | TAG_WITH_TAGGED | `<tool_call>name\n<arg_key>...<arg_value>...` format |497| GLM-4.7-Flash | TAG_WITH_TAGGED | Updated GLM format |498| Kimi-K2-Thinking | JSON_NATIVE | Reasoning + JSON tools |499| Apertus-8B-Instruct | JSON_NATIVE | Function name as JSON key |500| MiniMax-M2 | TAG_WITH_JSON | XML invoke with JSON args |501| NVIDIA-Nemotron-Nano-v2 | JSON_NATIVE | `<TOOLCALL>` wrapper (nested) |502| CohereForAI Command-R Plus | JSON_NATIVE | Markdown code block format |503| Mistral-Nemo-Instruct-2407 | JSON_NATIVE | `[TOOL_CALLS]` wrapper with ID field |504| Functionary v3.1 | TAG_WITH_JSON | `<function=X>` format |505| Functionary v3.2 | Specialized | `>>>` recipient delimiter (dedicated handler) |506| Fireworks Firefunction v2 | TAG_WITH_JSON | Fireworks tool format |507| DeepSeek R1 Distill (Llama/Qwen) | Reasoning | Forced-open thinking |508| llama-cpp-deepseek-r1 | Reasoning | Forced-open thinking |509| Kimi-K2 / Kimi-K2-Instruct | JSON_NATIVE | JSON tools with special markers |510| Llama 3.1/3.2/3.3 | JSON_NATIVE | Standard Llama tool format |511| OpenAI GPT-OSS | Specialized | Channel-based (dedicated handler) |512| Apriel 1.5 | JSON_NATIVE | `<tool_calls>` wrapper with JSON array |513| Apriel 1.6 Thinker | Reasoning | Implicit reasoning start |514| Mistral Small 3.2 | JSON_NATIVE | `[TOOL_CALLS]func[ARGS]{...}` with call ID |515| Devstral | JSON_NATIVE | `[TOOL_CALLS]func[ARGS]{...}` without call ID |516| StepFun 3.5 Flash | TAG_WITH_TAGGED | `<function=X><parameter=Y>` format |517| Spark2.5 | TAG_WITH_TAGGED | `<tool_call>name<arg_key>...<arg_value>...` format |518 519## Adding Support for New Templates520 521To support a new template format:522 5231. **If it follows standard patterns** — The auto-parser should detect it automatically. Run `test-chat-auto-parser <template_path>` to verify markers are correctly extracted.5242. **If differential analysis extracts incorrect markers** — Add a workaround lambda to the `workarounds` vector in `common/chat-diff-analyzer.cpp`. Inspect the template source for a unique identifying substring.5253. **If it needs fundamentally different handling** — Add a dedicated handler function in `chat.cpp` before the auto-parser block (as done for GPT-OSS, Functionary v3.2, and Ministral).526 527## Edge Cases and Quirks528 5291. **Generation Prompt & Reasoning Prefill**: The generation prompt is extracted by diffing `add_generation_prompt=false` vs `true` in `common_chat_templates_apply_jinja`, so it contains exactly what the template appends — avoiding false positives from prior conversation turns.5302. **Per-Call vs Per-Section Markers**: Some templates wrap each tool call individually (`per_call_start/end`); others wrap the entire section (`section_start/end`). T2 (`check_per_call_markers()`) disambiguates by checking if the second call in a two-call output starts with the section marker.5313. **Tag Boundary Fixing**: `calculate_diff_split()` iteratively adjusts prefix/suffix boundaries to avoid splitting `<tag>` or `[marker]` tokens, ensuring clean extraction.5324. **Call ID Side Effects**: When a call ID is detected, `per_call_end` may have been incorrectly set to include the call ID suffix. T7 clears `per_call_end` in this case.5335. **Tool Analysis Gating**: `analyze_tools` is only constructed (and all tool analysis phases run) when `jinja_caps.supports_tool_calls` is true. Within tool analysis, `check_per_call_markers()` (T2) only runs if `jinja_caps.supports_parallel_tool_calls`.5346. **`analyze_arguments()` Gating**: Within tool analysis, A1 and A2 (argument name/value marker extraction) only run for `TAG_WITH_TAGGED` format. `extract_argument_separator()` and `extract_args_markers()` run for all non-`JSON_NATIVE` formats.5357. **Undetected Tool Format**: If `analyze_tools` concludes tool calling is supported but cannot determine the format, `build_parser()` logs an error and returns `eps()` (graceful degradation) rather than aborting.536